1. Introduction

Electroencephalography is a technique used to detect the electrical impulses formed in the brain.In the early days it was used for monitoring a person's health for signs of epilepsy or any head injury.Electroencephalography enabled the development of Brain Computer Interfaces (BCI) based AAC devices, but these devices are expensive costing thousands of dollars and are out of reach of the common population due to its high cost.Moreover, these BCI based AAC devices are bulky and difficult to carry around, making them quite a hassle for a disabled person to use.

There have been few AAC devices built on top of EEG headsets which uses Rapid Serial Visual Presentation techniques such as the commonly used matrix speller[8] based input method, but these devices often requires the user to wear large number of EEG electrodes and tend to be slow due to the usage of a matrix speller based technique.There has been a similar approach to build an AAC device using Eye blinks[1], the work relies on the usage of IR switches to accomplish the task.Using IR switches can make the AAC device low cost, but on the other hand it introduces unreliable data due to the interference of a background IR source with the IR switch, moreover in such a device a natural eye blink would trigger actions that the user unintended to make.

 

This inspired to build Synaspeak by detecting neural electrical impulses associated with eye blinks and taking input from the user in the form of morse code and then translating it to english alphabets.Such an approach ensures faster input speeds and also brings down the overall cost of the AAC device to $100.

In this report, we utilize Synaspeak to demonstrate methods to convert EEG Eye blink data to speech and showcase the efficiency of a morse code based AAC input system.In Synaspeak,we incorporated a Command mode wherein the user can input large and common english sentences by using commands like 'HAY' for 'How are you?' and 'IAF' for 'I am fine'.Several of these common phrases have been pre programmed with their commands in the Synaspeak python program.We observe a satisfactory increase in the overall efficiency of the device, compared to existing AAC devices.

In the past decade, there has been a great growth in the development of affordable consumer EEG headsets by companies like Neurosky, Emotiv and Muse.These devices provide a platform for developers to build low cost EEG enabled solutions to various real world problems.

2. Methods

2.1. Hardware.

Synaspeak is based on the Neurosky Mindwave Mobile EEG Headset as seen in Fig 2.0.The neurosky mindwave mobile is an affordable consumer EEG headset.It is a single electrode based EEG Headset with its reference and ground electrodes on an ear clip attachment.The single EEG electrode rests in front of the users forehead (FP1 position) and can pick up Delta, Theta, alpha, beta and gamma brain waves.The neurosky blink detection algorithm enables users to monitor electrical impulses from FP1 region corresponding to eye blinks with an accuracy above 90%.

2.2. Software.

In Synaspeak we've written a program to correlate eye blinks with morse code.For this, two eye blinks are mapped to a 'dot' and three eye blinks to a 'dash'.The process is illustrated in Fig 2.0.

 

We've designed the algorithm to ignore naturally occuring blinks , which are commonly single blinks. To enter a word and speak it using the a Text To Speech Engine the user can blink four times and finally to clear an incorrect input the user needs to blink five times.One of our main focus was to avoid any cause of stress on the users eye, the algorithm is fine tuned to detect eye blinks even if the user applies less effort/pressure to close the eyelids.Therefore users can blink with ease and still input in morse code.Synaspeak enables the user to enter words of any length and the command mode incorporated into the Synaspeak program enables a user to input large common english phrases using smaller commands like 'WT' for I want water and 'FD' for I want food.We've open sourced the Synaspeak program on Github to enable developers from all over the world to innovate using Synaspeak and incorporate it for different applications, the source code is available at : https://github.com/GeeveGeorge/SynaSpeak

3. Results

As Synaspeak is mainly a software based solution to building a novel AAC device, the main hardware component used is the neurosky mindwave mobile as shown in Fig 2.0.The overall cost of the setup is approximately $100.

Video : https://www.youtube.com/watch?v=o99KV7MKYeY

3.1. EEG Headset Operation.

The EEG headset is powered by a single AAA battery.It connects to a laptop via bluetooth.Once paired with a laptop via bluetooth the Synaspeak python program is run on the laptop to connect to the neurosky mindwave headset.

Once a connection has been established between the Synaspeak program and the mindwave headset,  the program starts collecting raw EEG data from the headset.The synaspeak program specifically looks out for two blink and three blink commands that corresponds to a 'dot' and 'dash' respectively.Using these unique blink patterns the user can forms words, once the word has been formed the user can blink four times to read it aloud using the e-speak Text To Speech (TTS) engine.

3.2. Input speed testing.

A common drawback in existing AAC devices are their slow input speeds due to which users are often unable to convey their thoughts quickly.These drawbacks are often the result of using a common input method which is a matrix based character selection approach.With Synaspeak we explore a novel approach of morse code based character selection and have put to test the time taken for user character input on such a system.

In Fig 3.0 we can see the time taken to input various english alphabets.The least time taken for inputting a character is 1.6 seconds in the case of the letter 'E' which in morse code is a single dot '.'

The maximum time taken for inputting a character is 10.6 seconds in the case of 'X' which in morse code is '-..-'.

After testing single character speeds, we tested the performance of Synaspeak in forming words.In these tests we compared Synaspeak with a P300 Speller based EEG Method.In a paper[8] comparing speeds of P300 speller based EEG methods, the authors recorded the time taken by test subjects to input a word 'WATER' using their P300 speller based BCI device.The time taken to input 'WATER' was rounded up to 5 minutes in the paper.Using Synaspeak we were able to input the same word 'WATER' in 52 seconds without the use of any automatic text prediction algorithms.This shows that Synaspeak is 82% faster than the mentioned P300 Speller based BCI technique.

3.3. Command mode testing.

Table 1.0: Table showing the time taken to input common english phrases using the command mode featured in Synaspeak.

Command	Sentence	Time Taken (in seconds)
FD (..-. , -..)	I want food	25.14
IAF (.. , .- , ..-.)	I am fine	33.39
HI (.... , ..)	Hi, Nice to meet you	22.33
WR (.-- , .-.)	I would like to use the washroom	21.07
WT (.-- , -)	I want water	16.78
SP (... , .--.)	I would like to sleep	26.46
6 blinks	Yes	2.26
7 blinks	No	2.29

Synaspeak includes a command mode to input large common english phrases using relatively smaller commands.

The commands currently included in Synaspeak, their equivalent sentences and the time taken to input the commands can be seen in Table 1.0

These english phrases were included in the Synaspeak program so as to enable the user to inform the caregivers about their needs quickly.Common phrases like 'I would like to use the washroom' which is 32 characters long takes only 22 seconds to input.Such a feature ensures that the caregiver need not be constantly asking the user about his/her needs, rather the user can make known their needs quickly using short commands.

3.4. Performance comparison with commercial AAC devices.

Synaspeak was compared with commercial AAC devices like Eyespeak and Dynavox i-15+ to contrast the difference in durability , speed and cost.From Table 2.0 it can be observed that Synaspeak is the only wireless and low cost AAC device when compared to commercial AAC devices available in the market today.

Table 2.0 : Comparing commercial Eye tracking/BCI based AAC devices with Synaspeak.

Product	Method of Input	Speed	Durability	Cost
MegaBee AAC Tablet	Eye gaze tracking. (Manual Method which requires aid of a second individual )	Dependent on the speaker and listener.	Requires caregiver holding the device.	$2400 approx
Eyespeak 18HD	Eye Tracking (Rapid Serial Visual Presentation)	Fast, about 100 characters per minute with Auto-Prediction	Requires external mount	$15,800 approx
P300 Speller based BCI AAC Device	EEG Signals (Rapid Serial Visual Presentation)	Slow , less than 20 characters per minute. (without auto prediction)	Bulky Setup, Wired	Expensive (Cost varies)
Dynavox I-15+	Eye Tracking (Rapid Serial Visual Presentation)	Fast, about 100 characters per minute with Auto-Prediction	Requires external mount	$13,700 approx
Synaspeak	Eye Blink Detection (Morse Code)	Fast , about 100 characters per minute with command mode.	Wearable Headset	$100

 

4. Discussion

Synaspeak is a proof of concept that a wireless, portable AAC device which is fast and accurate can be produced for users with motor neuron disabilities inexpensively.It shows how the Eye blink to speech conversion algorithm could pave a new path for building faster and accurate AAC devices for paraplegic users.This is largely possible due to the development of consumer grade EEG headsets which are low cost and lightweight.Such an AAC device built on top of a consumer EEG headset could have a big impact on paraplegic users living in developing countries who often aren't able to afford the existing expensive alternatives.On the other hand, existing BCI based AAC devices rely on the need for multiple EEG electrodes to enable Thought to speech conversion techniques, but this makes the device bulky and hard to maintain.Hence, having a system with fewer(1-4) electrodes could prove useful for accomplishing tasks like blink detection in a much more inexpensive and portable form factor.

Synaspeak owes its existence to the Neurosky Mindwave EEG headset, which has enabled to make a first of its kind morse code based Eye blink to speech AAC device. Building Synaspeak on top of the Neurosky Mindwave Mobile EEG headset has made the AAC device to be built within $100. Existing AAC devices use Rapid Serial Visual Presentation techniques to acquire input from the user, such techniques requires the user to have a high attention and concentration level to make the right selection and the right time.This often causes an increase in the input error rate.Synaspeak gives the freedom to its users to input directly in morse code using Eye blinks without having any timing constraints, this enables the user to have a more natural interaction using the AAC device.

One limitation of Synaspeak is the learning curve associated with using the device , the user might have to take some time to learn morse code and input it using different eye blink patterns.As morse code contains a maximum of 4 character long commands for english alphabets it would be easier for the user to have the morse code and its equivalent characters displayed on a screen at first and as time progresses the user can learn morse code. Synaspeak being connected to a device through bluetooth can be easily paired with any PC device if the user requires the AAC device to be fixed to a wheelchair, Synaspeak can also be paired and used with a smartphone.The lightweight form factor makes Synaspeak readily available for use both indoors and outdoors.

Currently Synaspeak includes two modes, a normal input mode that enables a user to freely input any word or sentence using eye blink patterns in morse code and a command mode which includes common english phrases that can read out faster using smaller commands like 'WT' for I want water and 'FD' for I want food.In the future we will be incorporating automatic text prediction algorithms with the current system to improve the time taken to input words in the normal mode.We're also focussing on building Synaspeak for android to make it low cost and easily available to the people in developing countries.A device like Synaspeak enables day to day conversations with one's loved ones possible.We're also exploring into enabling users to explore the Internet using Eye blinks in a way that it doesn't stress the user's eyes in any way.To enable the further development of Synaspeak in innovative ways, we're open sourcing the Synaspeak python program which can be found on Github, the link can be found under the Methods section.

5. Conclusion

Synaspeak takes full advantage of the recent developments in wearable consumer EEG headsets and is a proof of concept that fast and accurate EEG based AAC devices can be built at a much lower cost and lighter form factor when compared to the existing expensive and bulky BCI based alternatives.Using the Neurosky Mindwave mobile as the central unit Synaspeak costs a maximum of only $100.With addition of automatic text prediction algorithms and other fine refinements Synaspeak would help a large population of motor neuron disabled users in developing countries have access to a fast and accurate AAC device.