Applying Machine Learning In The Classification Of Visuospatial Attention Signal Patterns For Communication And Control
Olawunmi George
2, Maysam M. Ardehali 1, Qussai M. Obiedat 1, Brooke C. Follansbee 1, Sheikh Iqbal Ahamed 2, Roger O. Smith 1
1 R2D2 Center, University of Wisconsin-Milwaukee; 2 Ubicomp Lab, Marquette University, Wisconsin
ABSTRACT
The detection and classification of the direction of covert attention can be used in building applications for people with motor impairments. This makes the study of attention of high importance. In this study, we investigated the use of machine learning in classifying the direction of attention of subjects performing a covert attention task. This paper presents the comparative results from the use of nine machine learning algorithms in the classification of covert attention. We introduce the need for communication devices targeted towards a specific population; those in the locked-in state. These tools may rely upon the detection and prediction of covert attention for successful communication with such people. With reference to previous works where machine learning has been explored, we applied nine machine learning algorithms on data obtained from four healthy subjects. The data was recorded from electrodes mostly placed in the parietal and occipital regions of the brain. From the study, our results show that tree-based algorithms performed best overall and that an approach based on the top distinguishing electrodes yielded better performances compared to others. We explain the results, stating our future goal for the improvement of the results.
INTRODUCTION
Detecting neurological patterns in the brain is not a new endeavour. Several efforts have been made to detect and understand brain patterns resulting from different cognitive activities. Of great interest has been the investigation of visuospatial attention [1, 2] and its use in providing a means of communication and control, particularly for people with severe motor impairments, such as locked-in syndrome. Overt visuospatial attention detection is targeted at detecting the focus of attention on an object, while having the eyes fixated upon it. Covert visuospatial attention detection, on another hand, is targeted towards identifying patterns in brain signals, when a subject is fixated on an object but actually paying attention to another. From the earlier efforts by Posner [3] to more recent endeavours [4], we see a variety of data collection paradigms and a variety of statistical and machine learning techniques being used in analysing visuospatial attention from data acquired.
Machine learning has been seen to yield very successful results in electroencephalography-based prediction tasks [4, 5]. The robustness of machine learning algorithms makes them very suitable for capturing the relationships between the variables in a dataset for prediction tasks. In this study, electroencephalography (EEG) was used to acquire brain signals from subjects performing a covert attention task. Machine learning algorithms were applied in predicting the direction of attention of the subject. All analyses were done offline.
METHODS
Participants
Four healthy female students, in their twenties, volunteered as subjects. All subjects had little to no experience with Brain-Computer Interface (BCI) experiments.
Procedure
The experiment was based on attention tasks, involving left and right directions. Participants were asked to fixate upon a cross, placed at the centre of the screen and to attend to either the left or right direction, based on auditory cues. For each recorded epoch of the data, 1000 ms elapsed before the presentation of a cue. After the cue presentation, subjects attended to the specified direction for a duration of 3000 ms. The 1000 ms before cue presentation was used for baseline correction; the data was bandpass filtered for frequencies between 0.1 and 40Hz; Common Average Referencing (CAR) was done and epoch extraction retaining only the 3000 ms of attention activity.
The number of data recordings from participants was 200, on the average; with some subjects having more data than others. The total recordings equalled 817 data points. A gTec 16-channel 256Hz EEG device was used with the electrodes (C5, C3, C4, C6, P7, P5, P3, PZ, P4, P6, P8, PO7, PO3, PO4, PO8, Oz) mostly in the parietal and occipital regions and placed according to the international 10-20 standard. BCI2000 software [6] was used for stimulus presentation and data acquisition. The data for each 3000 ms period was averaged and fed into the machine learning algorithms.
Machine Learning
Nine algorithms were applied to the data. The algorithms were XGBoost (XG), Logistic regression (LR), Quadratic Discriminant Analysis (QDA), Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Decision tree (DT), Random forests (RF), Naive Bayes (NB) and Deep Neural Networks (DNN) [4, 7]. Three feature selection approaches were used, based on electrode grouping: a) Basic approach, using all electrodes; b) Thut approach [8], using the following electrodes: P7, P5, PO7, P6, P8, PO8; c) Correlation approach, using electrodes P7, PO4, P5, C3, PZ, C4, PO8, P3, C5 and PO7, that represented the most distinction between the left and right targets. The data were split using an 80:20 split ratio.
RESULTS
The results of the analyses showed the XGBoost algorithm [9] had the highest accuracy of classification, closely followed by the DNN, QDA and NB, using the basic approach. It also, in general, performed better as one of the top two using the correlation approach.
| Algorithm | Basic approach % | Thut approach [8] % | Correlation approach % |
|---|---|---|---|
| XG | 61.59 | 58.54 | 63.41 |
| LR | 43 | 46 | 45 |
| QDA | 60 | 58 | 56 |
| LDA | 45 | 46 | 45 |
| SVM | 54 | 58 | 52 |
| DT | 55 | 58 | 64 |
| RF | 58 | 57 | 59 |
| NB | 60 | 60 | 58 |
| DNN | 60 | 58 | 61 |
DISCUSSION
In this study, we investigated the use of machine learning in classifying the direction of covert attention. The results show that accuracies of over 60% can be achieved using some algorithms. Also, the results show that using features which are correlated to target class yields better performance in classification. For each data recording, the extracted data for the 3000 ms of interest was averaged and used. This means the 768 recordings of 16 electrodes was smoothed to become just one. While this approach yielded over 60% accuracy in some cases, the smoothing might have been too intense in levelling out the data. That is perhaps why the accuracies did not go past 64%, as seen in the case of the decision tree, when the top ten correlated electrodes were used. Averaging the whole 3000 ms, though reducing the amount of computation would cause some variations in the signal to be averaged out, as well. It would be good to see how a smoothing approach which averages portions of the data and not the whole, at once, will affect the results.
Also, we realized that while the approach based on [8] yielded results mostly less than 60%, other approaches had more algorithms yield accuracies greater than or equal to 60%. These different parameters have to be taken into account in building an online system for real-time classification of signals, in providing a means of communication. The model performance must be improved to be suitable for an online communication and control system for healthy or unhealthy subjects.
CONCLUSION
The ability to classify these signals can help build an online tool for real-time communication with people with severe motor impairments. The current study was performed offline. In future studies, factors such as the operational speed of the system (total time from data acquisition to issuing a communication command and from the release of the command to the reception of feedback), accuracy of the system, usability and other factors must be considered in choosing the optimal algorithms and approaches. We plan to address factors such as these in future studies.
REFERENCES
- Posner, M. I., and Petersen, S. E. (1990). The attention system of the human brain. Annu. Rev. Neurosci. 13, 25–42. doi:10.1146/annurev.ne.13.030190.000325
- Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulus-driven attention in the brain. Nature reviews neuroscience, 3(3), 201.
- Posner, M. I. (1980). Orienting of attention. Quarterly journal of experimental psychology, 32(1), 3-25.
- Agrawal et.al, 2019, A Scalable Automated Diagnostic Feature Extraction System for EEGs, in 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC), July 2019, Milwaukee, Wisconsin.
- Farahat, A., Reichert, C., Sweeney-Reed, C. M., & Hinrichs, H. (2019). Convolutional neural networks for decoding of covert attention focus and saliency maps for EEG feature visualization. Journal of neural engineering, 16(6), 066010.
- Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., & Wolpaw, J. R. (2004). BCI2000: a general-purpose brain-computer interface (BCI) system. IEEE Transactions on biomedical engineering, 51(6), 1034-1043.
- Amanpreet Singh, Narina Thakur, Aakanksha Sharma "Review of Supervised Machine Learning Algorithms." 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), 2016.
- Thut, G., Nietzel, A., Brandt, S. A., & Pascual-Leone, A. (2006). α-Band electroencephalographic activity over occipital cortex indexes visuospatial attention bias and predicts visual target detection. Journal of Neuroscience, 26(37), 9494-9502.
- Tianqi Chen , Carlos Guestrin, XGBoost: A Scalable Tree Boosting System, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 13-17, 2016, San Francisco, California, USA [doi>10.1145/2939672.2939785]