SPECS is a project to allow dysarthric speech users to do environmental control. It is a collaboration between Sheffield University, Toby Churchill Ltd (TCL), The NHS, and Elpedium Technologies.
The same hardware is being used for the follow-on Vivoca2 project which is using the same dysarthric speech recogniser to generate speech instead of environmental control.
Most speech recognisers are designed to be speaker independent. Dysarthric speech is highly speaker-dependent to the point that often only people who spend a lot of time with the speaker can understand them. This requires a completely different approach to recognition: models bespoke to each individual.
Balloon3 is being used as the recogniser engine and controller in a system based on TCL SL40 hardware.
For development work a serial connection is necessary. To do this unscrew the back from the box and plug in a serial adaptor:
to the little IO board with the yellow switch on it:
That is a bottom-contact connector so the yellow backing must be upwards, contact side downwards.
You can connect to this with the usual balloon serial settings. See Balloon3Serial
Making it work
Build from balloonboard.org svn setting
in your toplevel Makefile.local
make dist will produce a set of kernels, debian and emdebian bootstrap images, etc.
Use BalloonJTAGing, BalloonOpenOCD, and BalloonSingleSidedRootfs to get those images into the machine.
Machines are supplied preinstalled, so normally no-one needs to do all the guff below. But it is documented for future reference.
A tarball of the result of the below works (386MB installed) is http://www.balloonboard.org/files/balloon3/vivoca2/specsroot.tgz. All the code is in balloonboard.org svn under the -tcl-specs variant.
Software setup from Debian base image
After installing and second-staging Debian Lenny rootfs do this:
(need to remove commented-out lines in /etc/apt/preferences to avoid error)
apt-get --no-install-recommends install libglademm-2.4-1c2a libxerces-c28 oss-compat bluez-utils xserver-xorg-video-fbdev xinit locales ttf-dejavu-core bluez-audio xfonts-base
and extra stuff to make the bluetooth pairing applet work
apt-get --no-install-recommends install bluez-gnome hal dbus-x11
Other packages installed for development convenience but not actually needed:
openssh-server mplayer jed mc screen alsa-utils rxvt
Install svg reader, it may install other dependencies
apt-get install librsvg2-common
SetUpBluetooth to make headset mic work
SetUpXserver to make xserver work with framebuffer
add auto usb0 to /etc/network/interfaces
- to install en_GB-UTF8 and make it the default
apt-get install --reinstall ttf-dejavu-core fc-cache -f
- to fix missing fonts
Audio setup: You don't actually need alsa-utils to run the app (only alsa-base is needed) but it's inconvenient not having it if you need to test anything. The fact that it brings 20MB of python in is tiresome.
The settings needed are: volume 90, not muted, headphone amp, DC block and bass boost off, smart power saving 100
These can be set using amixer, alsamixer(if you have alsa-utils installed) or copying this into /var/lib/alsa/asound.rc
amixer sset Master 90% unmute amixer sset 'Headphone Driver' mute amixer sset 'DC Filter' mute amixer sset 'Loudness (bass boost)' mute amixer sset 'Smart Low Power Mode' 100%
To use a 3.5mm jack plug-in microphone/headset you need to set it as an input:
amixer sset 'Line' cap amixer sset 'Capture' 60%
then arecord will use it as the default capture device. The appropriate 'Capture' gain will depend on your mic.
GEWA prog module. In order for the GEWA serial port (/dev/ttyS0 on the Loon) to work, RUN_GSM must be asserted on the motherboard. The easiest way to do this is to send the appropriate command down the Pinko serial port (/dev/ttyS1):
stty -F /dev/ttyS1 115200 echo "gsm:1" > /dev/ttyS1 stty -F /dev/ttyS1 9600
To test this do
cat /dev/ttyS0 & echo '!V' > /dev/ttyS0 fg
which is the 'print version' comand and should return
(Note that minicom is not generally useful here, use the commands directly or use the lw-infrared test package from the vivoca repository.)
setting up userinterface app
adduser specs cp libasrtest.so to /usr/local/lib/ mkdir -p ~specs/userinterface/bin cp test ~specs/userinterface/bin/ chmod 755 ~specs/userinterface/bin/test cp models_bk specs/userinterface/ cp asrtest_balloon.glade /home/user/userinterface/
with those files in the right places you should be able to do
building test app
There is not space to install all this dev stuff on a standard machine with 512Mb flash. You need to add a usb stick and mount /usr from it (make sure it is formatted ext2)
Install all the necessary stuff:
apt-get install build-essential libsndfile1-dev libxerces-c2-dev libglademm-2.4-dev pkg-config
then build libartest.so and test
copy lib/libartest.so into /usr/local/lib
First build the library libasrtest.so and copy it into /usr/local/lib using the ./build script in casapi/libaudiotest. Then build the test program also using ./build in castapi. Copy this binary into userterface/bin. Run it with DISPLAY=:0 test