RESNA 26th International Annual Confence
FEEDBACK EXPERIMENTS TO IMPROVE THE DETECTION OF EVENT-RELATED POTENTIALS IN ELECTROCORTICOGRAM SIGNALS.
We report here our early attempts to use real-time feedback to improve the detection of event-related potentials (ERPs) in human electrocorticogram (ECoG) signals. Both first and second generation online feedback systems are reported. In the first generation system, feedback to subjects was based on the incremental change in the signal-to-noise ratio (SNR) of the ERP template. Our second generation system provides the subject with online feedback based on the cross-correlation values (a direct measure of detection accuracy) generated by correlating an ERP template with the continuous ECoG signal. Our results show that with feedback subjects can learn to improve the detection accuracy of ERPs, which is required for the operation of a direct brain interface.
A direct brain interface (DBI) accepts voluntary commands directly from the human brain without requiring physical movement and can be used to operate a computer or other assistive technology. The University of Michigan Direct Brain Interface (UM-DBI) is based on the detection of voluntarily generated event-related potentials (ERPs) in human electrocorticograms (ECoG) (1). Work to date has employed actual movements to accurately assess ERP detection accuracy. Migration to the use of imagined movements or other purely cognitive activities will occur in the near future. The short-term goal of this work is an interface capable of consistently operating single-switch assistive technologies. The longer-term goal is to increase the accuracy of the interface and the number of useable control channels.
Human subjects given appropriate feedback can learn to modify mu rhythms (2) or increase event-related desynchronization/synchronization (ERD/ERS) (3) in their electroencephalogram (EEG) in order to operate a DBI. Similar experiments have not been attempted with ECoGs. Our previously reported off-line detection experiments (1) relied solely on off-line processing and provided no feedback to subjects during data collection.
Our research question is whether subjects, given real-time feedback, can learn to alter their ERPs in order to improve the detection accuracy for ERPs in ECoG signals. Restrictions included lack of control over electrode placement (electrodes are solely placed on the basis of clinical considerations outside the scope of this research) and the state of mind of the subject.
Research subjects were patients in epilepsy surgery programs who have electrodes implanted on the surface of the cortex for clinical purposes unrelated to our research objectives.
Data was collected from the subjects as they performed one or more template collection blocks of 50 repetitions of an action. Triggered averaging was used to create a signal template from the ECoG for every electrode location (1). A single ERP template was selected based on a signal to noise (SNR) measure (4) and the Peak-Baseline ratio (a quantitative amplitude measure) (1). The template extended 2 seconds before and 1 second after the trigger point. This template collection method was common to both the first and second-generation feedback systems.
This system provided the subject positive feedback when they generated clean and stable ERPs that were consistent with ERPs in the previous trials (4). Feedback was based on a comparison between the incremental change in SNR from the addition of the current ERP and the average change in SNR during the previous set of ERPs (4). The larger the increase in SNR value, the higher the level of positive feedback. However, the SNR value of the ERP template was only an indirect measure of detection accuracy and thus was sub-optimal. Visual feedback was given in the form of a deflection of a vertical green bar on a computer screen approximately 2.2 seconds after the onset of each movement (4). The feedback session involved 6 feedback blocks each containing 25 repetitions of the action. Sessions lasted less than 2 hours.
This system provided online feedback based on the cross-correlation value, a more direct measure of detection accuracy than the SNR value. Unlike the first generation system, where feedback was provided only after an action, this system provided feedback approximately every 1.5 seconds. The specific point in time at which feedback was given was independent of the trigger points. The feedback had an inherent 1 second delay because the ERP template extended 1 second after movement onset. The subject was presented with alternating "ready" and "not-ready" windows of at least 4 seconds. Detection points were called "hits" only if they occurred in the "ready" window; otherwise they were called "false positives" (FP). Detection accuracy was quantified as the difference between the hit% and false positive% (HF-difference). The correlation value was represented to the subject on-screen as the height attained by a caricature of a high jumper, who was attempting to jump over a high bar. The height of the bar represented the detection threshold. Subjects were instructed to make the athlete jump over the bar during a "ready" window and keep the high jumper below the bar during the "not-ready" windows. This design was employed to promote both an increase in hit% as well as a decrease in false positive%. An average of four feedback blocks, each comprising 50 repetitions of the action, was completed in 2 hours.
Three of six subjects showed dramatic improvements in the template SNR between the baseline template collection block and subsequent feedback blocks. The SNRs improved from 3.5 to 7.8, 4.8 to 10.5 and 5.1 to 8.0. One subject had a corresponding improvement in the ERP detection accuracy from 79% hits and 22% false positives to 100% hits and 0% false positives.
The second-generation system has been used with three subjects to date. A detection threshold of 0.4 was used. A template with an SNR of only 1.21 was found for the first subject (subject E50) and feedback did not improve the template SNR or the HF-difference. For the second subject (E51), template SNR increased from 7.63 to 18.98 and detection accuracy increased from 90% hits with 44% false positives to 90% hits with 10% false positives (see Table 1). For the third subject (F52) template SNR increased from 1.94 to a maximum of 5.03. Overall, the HF-difference actually decreased for this subject (see Table 2).
FP % |
|||||
|---|---|---|---|---|---|
11.34 |
|||||
The results of the first generation feedback system indicated that improvements in the SNR values of ERPs related to actual movement as well as improvements in the resultant detection accuracy are possible over a relatively short period of time given only simple feedback. Likewise the results from subject E51 clearly showed that the subject learned to control ERPs to improve the detection accuracy. The reduction in false positives also showed that the subject learned to suppress background ECoG that caused false positives, when in the "not-ready" window.
Several subjects had ERP templates with very low SNR values. This was most likely due to the lack of control over electrode location which often leads to electrode locations that do not overlap any cortical region for motor activity and more specifically the action being performed by the subject.
The results from subject F52 showed that the template SNR increased from 1.94 to 5.03 and then decreased again. The HF-difference also decreased in the last two feedback blocks (Table 2). While the number of false positives remained consistently low, the number of hits actually decreased. A possible explanation is that the correlation threshold could have been too high, resulting in valid detections being missed and inaccurate feedback given. Subsequent offline testing, of the ECoG signals from the 4 feedback blocks, using a lower threshold of 0.3, showed improved HF-differences of 65.1%, 53.5%, 15.8% and 32% respectively. These were higher than the HF-differences obtained using a threshold of 0.4 (Table 2). This shows that because several valid detections were missed, the subject did not get accurate feedback across all feedback blocks and can explain the lack of learning demonstrated by this subject.
In order to operate a DBI, high detection accuracy will be required. The results presented here show that with feedback, subjects can learn to improve the performance of the DBI. There are several variables like the electrode location, the cross-correlation threshold, the subject's state of mind, and the use of imagined movements that could impact the results of these feedback experiments. The impact of these unknowns needs to be explored further and quantified.