Introduction

An estimated two million people with complex communication needs (CCN) worldwide depend on some sort of low or high tech augmentative and alternative communication (AAC) (Beukelman & Miranda 2005; National Joint Committee, 2017). Like the general population, most AAC users require multiple technologies, including eye tracking, to meet their needs across tasks and across the course of their injury or disease (Scherer, 2005).

People affected by severe motor disorders such as amyotrophic lateral sclerosis (ALS) and brainstem stroke may not be able to use even the most basic conventional assistive technologies, which all rely in one way or another on muscle control. A brain-computer interface (BCI) can give such individuals communication and control technology that does not depend on neuromuscular output (Wolpaw and Wolpaw, 2012).

A BCI records brain signals, extracts specific measures (or features) from them, and converts (or translates) these features into commands that operate applications that replace, restore, enhance, supplement, or improve natural CNS outputs (Wolpaw and Wolpaw, 2012). For people diagnosed with ALS, a BCI using the P300 event-related potential allows users to choose among items in a matrix and provides the means for communication. (Sellers et al., 2006).

The P300 is a positive deflection in the electroencephalogram (EEG) that occurs 200 to 700 ms after stimulus onset and is typically recorded over central-parietal scalp locations. The response is evoked by attention to rare stimuli in a random series of stimulus events (i.e., the oddball paradigm). Farwell and Donchin (1988) first discussed using the oddball paradigm for communication almost three decades ago.

More recently, researchers at the National Center for Adaptive Neurotechnologies and Helen Hayes Hospital of the New York State Department of Health, have shown that people with limited eye control, can learn to control and use distinct features of scalp-recorded EEG activity including the P300 and other event-related potentials, to move a computer cursor in one or more dimensions, to select letters or icons, or even to move a robotic arm (reviewed in, (Vaughan et al., 2006)). Further, we have demonstrated that people with ALS can use EEG-based BCIs for communication and control autonomously in their homes (Sellers et al., 2010; Wolpaw et al., 2013).

Tobii Dynavox Communicator 5 (C5) is a highly configurable software package designed for use with an eye-gaze device and other conventional switches. It can be used for communication and computer access including e-mail, text messaging, telephone, and environmental control (Tobii Dynavox, 2016).

Methods

Fourteen individuals (seven males, two with ALS) (average age 35(+24); range, 16-73) took part in this study. It was reviewed and approved by the Institutional Review Board of New York State Department of Health; and all subjects gave informed consent.

All aspects of BCI use were controlled by BCI2000 (Schalk et al., 2004). The C5 dynamic screens were presented via a Perl script and the program was accessed using the Microsoft Active Accessibility (MSAA) software development kit (SDK). MSAA is an Application Programming Interface (API) originally designed to improve computer access for physically and cognitively impaired individuals. It reports on the location and purpose of elements on the current Microsoft Windows screen through a graphical user interface (GUI). (https://msdn.microsoft.com/en-us/library/ms697707.aspx.).

We recorded eight channels of EEG from frontal, central, and posterior scalp locations (Fz, Cz, P3, Pz, P4, Po7, Po8, Oz), referenced to the right and grounded to the left mastoid respectively. Signals were amplified using a a g.USBamp biosignal amplifier (g-tec Medical). Signals were sampled at a rate of 256 Hz, high and low-pass filtered at 0.1 Hz and 60 Hz, respectively and a 60Hz notch filter (58-62 Hz). A review of the method can be found in (McCane et al., 2014).

TABLE 1 Performance during BCI/C5 operation

Subj	S1 % correct	S1 Selections /Min	S2 Correct Selections	S3 Total selections
1	100	7.3	4	7
2	100	8	4	6
3	100	6.7	4	10
4	100	7.3	4	7
5	100	5.6	4	-
6	100	6	4	-
7	100	5.1	4	10
8	100	8	4	7
9	100	3.7	4	-
10	100	8	4	6
11	100	5.4	4	6
12	100	7.4	4	7
13*	79	1.2	3	-
14*	79	1.5	3	-
AVE	97+8	5.8+2.3	3.9	7.3+1.6
Performance data for all subjects. S1 represents the optimized performance, i.e., results calculated offline using a 5-fold cross validation. S2 represents the number of correct selections (max=4) classified correctly during Validation. S3 represents the number of selections needed to complete the sentence: “I want to watch TV.”

The subject sat in a comfortable chair or his/her own wheelchair at a comfortable viewing distance from a 50 cm screen (Fig 1A). The task had three steps. During the Calibration Step (S1) subjects attended to 21 cued targets as the letters in the words “THE,” “QUICK,” “BROWN,” “FOX,” “JUMPS” with several minutes break between each word. In the Validation Step (S2), subjects selected four cued targets displayed as the letters in the word “JULY” with feedback. During Navigation (S3) the static keyboard (Fig 1C) was replaced with the C5 dynamic screen (Fig 1D). The subjects were instructed to complete the following sentence “I want to watch TV.” To do this successfully, they needed to navigate among three different C5 screens and make a total of four selections from them. Results were displayed at the top of the Core Screen.

Each step required the subject to attend to a picture of a face flashed over the target item (Kaufmann et al., 2011) while all items on the screen flashed in groups of 4-6 items at a rate of four times per second (Townsend et al., 2010). Each step was carefully explained and illustrated. After each run, the subject was asked if she/he wished to continue. The experiment, including consent, step instructions, cap application and removal, and data collection took 45-60 min.

Personalized settings, i.e., correlation coefficients, derived using a stepwise linear discriminate analysis (SWLDA) from data collected during S1, were applied and validated in S2 and S3. Accuracies and optimized selections/min found in Table 1 represent results of a five-fold cross-correlation performed offline. Results reported from S2 and S3 are in real time where selection rates were held constant at 3 selections/min.

Results

All subjects were able to access the TOBII pages using the BCI, making a minimum of three correct selections during Validation (S2). (acceptable criteria for BCI use (Sellers et al., 2006)). Calibration (S1) accuracy, calculated offline using a five-fold cross validation, and average selections/min were 97(+8) and 5.8(+2.3) (range 1.2-8.0), respectively. The nine subjects who completed Navigation (S3) were able to move back and forth between the Communicator 5 screens making an average of seven selections where six selections represented a trial with no errors. (TABLE 1).

Discussion

These early results indicate that the Tobii Communicator 5 software can be accessed using a P300-based BCI. In future work, we will ask the BCI home users to trial the full functionality of the software. Future research will address streamlining the system for easier setup and use by caregivers; developing a hybrid system that allows the user to transfer effortlessly between access methods; and the development of a truly integrated system using both eye tracking and EEG control.