Overview

SPECS is a project to allow dysarthric speech users to do environmental control. It is a collaboration between Sheffield University, Toby Churchill Ltd (TCL), The NHS, and Elpedium Technologies.

The same hardware is being used for the follow-on Vivoca2 project which is using the same dysarthric speech recogniser to generate speech instead of environmental control.

Most speech recognisers are designed to be speaker independent. Dysarthric speech is highly speaker-dependent to the point that often only people who spend a lot of time with the speaker can understand them. This requires a completely different approach to recognition: models bespoke to each individual.

Hardware

Balloon3 is being used as the recogniser engine and controller in a system based on TCL SL40 hardware.

http://balloonboard.org/gallery/SPECS/SPECSinternals.jpg

Serial connection

For development work a serial connection is necessary. To do this unscrew the back from the box and plug in a serial adaptor:

http://balloonboard.org/gallery/SL40serialcable.jpg

to the little IO board with the yellow switch on it:

http://balloonboard.org/gallery/SPECS/SPECSserialconn.jpg

That is a bottom-contact connector so the yellow backing must be upwards, contact side downwards.

You can connect to this with the usual balloon serial settings. See Balloon3Serial

Making it work

Initial installation

Build from balloonboard.org svn setting

$VARIANT=-specs

in your toplevel Makefile.local

make dist will produce a set of kernels, debian and emdebian bootstrap images, etc.

Use BalloonJTAGing, BalloonOpenOCD, and BalloonSingleSidedRootfs to get those images into the machine.

Machines are supplied preinstalled, so normally no-one needs to do all the guff below. But it is documented for future reference.

A tarball of the result of the below works (386MB installed) is http://www.balloonboard.org/files/balloon3/vivoca2/specsroot.tgz. All the code is in balloonboard.org svn under the -tcl-specs variant.

Software setup from Debian base image

After installing and second-staging Debian Lenny rootfs do this:

apt-get update

(need to remove commented-out lines in /etc/apt/preferences to avoid error)

apt-get --no-install-recommends install libglademm-2.4-1c2a libxerces-c28 oss-compat bluez-utils xserver-xorg-video-fbdev xinit locales ttf-dejavu-core bluez-audio xfonts-base

and extra stuff to make the bluetooth pairing applet work

apt-get --no-install-recommends install bluez-gnome hal dbus-x11 

Other packages installed for development convenience but not actually needed:

openssh-server mplayer jed mc screen alsa-utils rxvt

Install svg reader, it may install other dependencies

apt-get install librsvg2-common 

SetUpBluetooth to make headset mic work

SetUpXserver to make xserver work with framebuffer

add auto usb0 to /etc/network/interfaces

dpkg-reconfigure locales

- to install en_GB-UTF8 and make it the default

apt-get install --reinstall ttf-dejavu-core
fc-cache -f

- to fix missing fonts

Audio setup: You don't actually need alsa-utils to run the app (only alsa-base is needed) but it's inconvenient not having it if you need to test anything. The fact that it brings 20MB of python in is tiresome.

The settings needed are: volume 90, not muted, headphone amp, DC block and bass boost off, smart power saving 100

These can be set using amixer, alsamixer(if you have alsa-utils installed) or copying this into /var/lib/alsa/asound.rc

amixer sset Master 90% unmute
amixer sset 'Headphone Driver' mute
amixer sset 'DC Filter' mute
amixer sset 'Loudness (bass boost)' mute
amixer sset 'Smart Low Power Mode' 100%

To use a 3.5mm jack plug-in microphone/headset you need to set it as an input:

amixer sset 'Line' cap
amixer sset 'Capture' 60%

then arecord will use it as the default capture device. The appropriate 'Capture' gain will depend on your mic.

GEWA prog module. In order for the GEWA serial port (/dev/ttyS0 on the Loon) to work, RUN_GSM must be asserted on the motherboard. The easiest way to do this is to send the appropriate command down the Pinko serial port (/dev/ttyS1):

stty -F /dev/ttyS1 115200
echo "gsm:1" > /dev/ttyS1
stty -F /dev/ttyS1 9600

To test this do

cat /dev/ttyS0 &
echo '!V' > /dev/ttyS0
fg 

which is the 'print version' comand and should return

01:319 

(Note that minicom is not generally useful here, use the commands directly or use the lw-infrared test package from the vivoca repository.)

setting up userinterface app

adduser specs
cp libasrtest.so to /usr/local/lib/
mkdir -p ~specs/userinterface/bin
cp test ~specs/userinterface/bin/
chmod 755 ~specs/userinterface/bin/test
cp models_bk specs/userinterface/
cp asrtest_balloon.glade /home/user/userinterface/

with those files in the right places you should be able to do

startx ~specs/userinterface/bin/test

building test app

There is not space to install all this dev stuff on a standard machine with 512Mb flash. You need to add a usb stick and mount /usr from it (make sure it is formatted ext2)

Install all the necessary stuff:

apt-get install build-essential libsndfile1-dev libxerces-c2-dev  libglademm-2.4-dev pkg-config

then build libartest.so and test

copy lib/libartest.so into /usr/local/lib

First build the library libasrtest.so and copy it into /usr/local/lib using the ./build script in casapi/libaudiotest. Then build the test program also using ./build in castapi. Copy this binary into userterface/bin. Run it with DISPLAY=:0 test

Balloonboard: SPECSBAlloon (last edited 2011-01-31 14:38:08 by NeilWilliams)