Steady-State Visual Evoked Potential-Based Brain–Computer Interface System for Enhanced Human Activity Monitoring and Assessment
Abstract
:1. Introduction
2. Related Work
3. Research Methodology
3.1. Common Spatial Pattern Algorithm
3.2. Independent Component Analysis
4. Experimental Setup
4.1. Participants
4.2. EEG Setup
4.3. Experimental Procedure
- Score each stimulus on a five-point scale based on their level of comfort of the experiment, with 1 to 5 corresponding to very uncomfortable, uncomfortable, slightly uncomfortable, comfortable, and very comfortable, respectively.
- Score each stimulus on a five-point scale based on their perception of the effect of the stimulus flicker, with 1 to 5 corresponding to very annoying, annoying, slightly annoying, noticeable, and imperceptible, respectively.
- Score each stimulus on a five-point scale based on their preference of the stimuli, with 1 to 5 corresponding to very annoying, annoying, neutral, like, and very like, respectively.
4.4. Virtual Environment for Assisted Vehicle Maneuvring
5. Data Processing
5.1. Noise Reduction
- Alpha waves: 8–12 Hz;
- Beta 1 waves: 12–20 Hz;
- Beta 2 waves: 20–30 Hz;
- Gamma waves: 30–50 Hz.
5.2. Removal of Artifacts with ICA
5.3. Feature Extraction with CSP
6. Results and Discussion
6.1. Classification of Brain Activity
6.2. Comparison with Similar Methods
6.3. Limitations
6.4. Future Work
7. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Wolpaw, J.R.; Birbaumer, N.; Heetderks, W.J.; McFarland, D.J.; Peckham, P.H.; Schalk, G.; Donchin, E.; Quatrano, L.A.; Robinson, C.J.; Vaughan, T.M.; et al. Brain-computer interface technology: A review of the first international meeting. IEEE Trans. Rehabil. Eng. 2000, 8, 164–173. [Google Scholar] [CrossRef] [PubMed]
- Paszkiel, S.; Paszkiel, S. Using BCI and VR technology in neurogaming. In Analysis and Classification of EEG Signals for Brain– Computer Interfaces; Springer: Cham, Switzerland, 2020; pp. 93–99. [Google Scholar]
- Moore, M.M. Real-world applications for brain-computer interface technology. IEEE Trans. Neural Syst. Rehabil. Eng. 2003, 11, 162–165. [Google Scholar] [CrossRef]
- Müller, S.M.T.; Bastos, T.F.; Filho, M.S. Proposal of a SSVEP-BCI to command a robotic wheelchair. J. Control. Autom. Electr. Syst. 2013, 24, 97–105. [Google Scholar] [CrossRef]
- Arpaia, P.; Duraccio, L.; Moccaldi, N.; Rossi, S. Wearable brain–computer interface instrumentation for robot-based rehabilitation by augmented reality. IEEE Trans. Instrum. Meas. 2020, 69, 6362–6371. [Google Scholar] [CrossRef]
- Veena, N.; Anitha, N. A review of non-invasive BCI devices. Int. J. Biomed. Eng. Technol. 2020, 34, 205–233. [Google Scholar]
- Angrisani, L.; Arpaia, P.; Esposito, A.; Moccaldi, N. A wearable brain–computer interface instrument for augmented reality-based inspection in industry 4.0. IEEE Trans. Instrum. Meas. 2019, 69, 1530–1539. [Google Scholar] [CrossRef]
- Rakotomamonjy, A.; Guigue, V. BCI competition III: Dataset II-ensemble of SVMs for BCI P300 speller. IEEE Trans. Biomed. Eng. 2008, 55, 1147–1154. [Google Scholar] [CrossRef]
- Thomas, E.; Dyson, M.; Clerc, M. An analysis of performance evaluation for motor-imagery based BCI. J. Neural Eng. 2013, 10, 031001. [Google Scholar] [CrossRef]
- Liu, B.; Huang, X.; Wang, Y.; Chen, X.; Gao, X. BETA: A large benchmark database toward SSVEP-BCI application. Front. Neurosci. 2020, 14, 544547. [Google Scholar] [CrossRef]
- Yin, E.; Zhou, Z.; Jiang, J.; Yu, Y.; Hu, D. A dynamically optimized SSVEP brain–computer interface (BCI) speller. IEEE Trans. Biomed. Eng. 2014, 62, 1447–1456. [Google Scholar] [CrossRef]
- Lin, B.S.; Wang, H.A.; Huang, Y.K.; Wang, Y.L.; Lin, B.S. Design of SSVEP enhancement-based brain computer interface. IEEE Sens. J. 2020, 21, 14330–14338. [Google Scholar] [CrossRef]
- Vialatte, F.B.; Maurice, M.; Dauwels, J.; Cichocki, A. Steady-state visually evoked potentials: Focus on essential paradigms and future perspectives. Prog. Neurobiol. 2010, 90, 418–438. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Gao, X.; Hong, B.; Jia, C.; Gao, S. Brain-computer interfaces based on visual evoked potentials. IEEE Eng. Med. Biol. Mag. 2008, 27, 64–71. [Google Scholar] [CrossRef] [PubMed]
- Herrmann, C.S. Human EEG responses to 1–100 Hz flicker: Resonance phenomena in visual cortex and their potential correlation to cognitive phenomena. Exp. Brain Res. 2001, 137, 346–353. [Google Scholar] [CrossRef]
- Nakanishi, M.; Wang, Y.; Wang, Y.T.; Mitsukura, Y.; Jung, T.P. A high-speed brain speller using steady-state visual evoked potentials. Int. J. Neural Syst. 2014, 24, 1450019. [Google Scholar] [CrossRef]
- Ha, J.; Park, S.; Im, C.H. Novel hybrid brain-computer interface for virtual reality applications using steady-state visual-evoked potential-based brain–computer interface and electrooculogram-based eye tracking for increased information transfer rate. Front. Neuroinformatics 2022, 16, 758537. [Google Scholar] [CrossRef]
- Yin, G.; Gong, L. Direction control and speed control combined model of motor-imagery based brain-actuated vehicle. In Proceedings of the 2017 36th Chinese Control Conference (CCC), Dalian, China, 26–28 July 2017; pp. 2210–2214. [Google Scholar]
- Acharya, D.; Das, D.K. Design of a fuzzy-based proportional integral derivative controller with optimal membership function scaling for respiratory ventilation system. Biomed. Signal Process. Control 2022, 78, 103938. [Google Scholar] [CrossRef]
- Liu, R.; Wang, Y.X.; Zhang, L. An FDES-based shared control method for asynchronous brain-actuated robot. IEEE Trans. Cybern. 2015, 46, 1452–1462. [Google Scholar] [CrossRef]
- Schmidt, K.W.; Boutalis, Y.S. Fuzzy discrete event systems for multiobjective control: Framework and application to mobile robot navigation. IEEE Trans. Fuzzy Syst. 2012, 20, 910–922. [Google Scholar] [CrossRef]
- Milekovic, T.; Sarma, A.A.; Bacher, D.; Simeral, J.D.; Saab, J.; Pandarinath, C.; Sorice, B.L.; Blabe, C.; Oakley, E.M.; Tringale, K.R.; et al. Stable long-term BCI-enabled communication in ALS and locked-in syndrome using LFP signals. J. Neurophysiol. 2018, 120, 343–360. [Google Scholar] [CrossRef]
- Combaz, A.; Chatelle, C.; Robben, A.; Vanhoof, G.; Goeleven, A.; Thijs, V.; Van Hulle, M.M.; Laureys, S. A comparison of two spelling brain-computer interfaces based on visual P3 and SSVEP in locked-in syndrome. PLoS ONE 2013, 8, e73691. [Google Scholar] [CrossRef]
- Cao, L.; Liu, T.; Hou, L.; Wang, Z.; Fan, C.; Li, J.; Wang, H. A novel real-time multi-phase BCI speller based on sliding control paradigm of SSVEP. IEEE Access 2019, 7, 133974–133981. [Google Scholar] [CrossRef]
- Chen, W.; Chen, S.K.; Liu, Y.H.; Chen, Y.J.; Chen, C.S. An electric wheelchair manipulating system using SSVEP-based BCI system. Biosensors 2022, 12, 772. [Google Scholar] [CrossRef] [PubMed]
- Peters, B.; Bedrick, S.; Dudy, S.; Eddy, B.; Higger, M.; Kinsella, M.; McLaughlin, D.; Memmott, T.; Oken, B.; Quivira, F.; et al. SSVEP BCI and eye tracking use by individuals with late-stage ALS and visual impairments. Front. Hum. Neurosci. 2020, 14, 595890. [Google Scholar] [CrossRef] [PubMed]
- Cao, T.; Wang, X.; Wang, B.; Wong, C.M.; Wan, F.; Mak, P.U.; Mak, P.I.; Vai, M.I. A high rate online SSVEP based brain-computer interface speller. In Proceedings of the 2011 5th International IEEE/EMBS Conference on Neural Engineering, Cancun, Mexico, 27 April–1 May 2011; pp. 465–468. [Google Scholar]
- Shyu, K.K.; Lee, P.L.; Lee, M.H.; Lin, M.H.; Lai, R.J.; Chiu, Y.J. Development of a low-cost FPGA-based SSVEP BCI multimedia control system. IEEE Trans. Biomed. Circuits Syst. 2010, 4, 125–132. [Google Scholar] [CrossRef] [PubMed]
- Wang, M.; Li, R.; Zhang, R.; Li, G.; Zhang, D. A wearable SSVEP-based BCI system for quadcopter control using head-mounted device. IEEE Access 2018, 6, 26789–26798. [Google Scholar] [CrossRef]
- Shyu, K.K.; Lee, P.L.; Liu, Y.J.; Sie, J.J. Dual-frequency steady-state visual evoked potential for brain computer interface. Neurosci. Lett. 2010, 483, 28–31. [Google Scholar] [CrossRef]
- Asheri, B.; Haratian, A.; Mohamadi, M.; Asadi, F.; Yasini, P.; Zarepak, N.; Samiei, D.S.; Menhaj, M.B. Enhancing detection of steady-state visual evoked potentials using frequency and harmonics of that frequency in openvibe. Biomed. Eng. Adv. 2021, 2, 100022. [Google Scholar] [CrossRef]
- Wang, Y.; Chen, X.; Gao, X.; Gao, S. A benchmark dataset for SSVEP-based brain–computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 2016, 25, 1746–1752. [Google Scholar] [CrossRef]
- Choi, G.Y.; Han, C.H.; Jung, Y.J.; Hwang, H.J. A multi-day and multi-band dataset for a steady-state visual-evoked potential–based brain-computer interface. GigaScience 2019, 8, giz133. [Google Scholar] [CrossRef]
- Renton, A.I.; Painter, D.R.; Mattingley, J.B. Optimising the classification of feature-based attention in frequency-tagged electroencephalography data. Sci. Data 2022, 9, 296. [Google Scholar] [CrossRef] [PubMed]
- Lim, H.; Ku, J. Multiple-command single-frequency SSVEP-based BCI system using flickering action video. J. Neurosci. Methods 2019, 314, 21–27. [Google Scholar] [CrossRef] [PubMed]
- Wen, D.; Fan, Y.; Hsu, S.H.; Xu, J.; Zhou, Y.; Tao, J.; Lan, X.; Li, F. Combining brain–computer interface and virtual reality for rehabilitation in neurological diseases: A narrative review. Ann. Phys. Rehabil. Med. 2021, 64, 101404. [Google Scholar] [CrossRef]
- Zehra, S.R.; Mu, J.; Syiem, B.V.; Burkitt, A.N.; Grayden, D.B. Evaluation of optimal stimuli for ssvep-based augmented reality brain-computer interfaces. IEEE Access 2023, 11, 87305–87315. [Google Scholar] [CrossRef]
- Wang, S.; Mao, Z.; Zeng, C.; Gong, H.; Li, S.; Chen, B. A new method of virtual reality based on Unity3D. In Proceedings of the 2010 18th International Conference on Geoinformatics, Beijing, China, 18–20 June 2010; pp. 1–5. [Google Scholar]
- Haas, J.K. A History of the Unity Game Engine. Interactive Qualifying Project, Worcester Polytechnic Institute: Worcester, MA, USA, 2014; Available online: http://www.daelab.cn/wp-content/uploads/2023/09/A_History_of_the_Unity_Game_Engine.pdf (accessed on 29 October 2024).
- Subasi, A.; Gursoy, M.I. EEG signal classification using PCA, ICA, LDA and support vector machines. Expert Syst. Appl. 2010, 37, 8659–8666. [Google Scholar] [CrossRef]
- Lotte, F.; Guan, C. Regularizing common spatial patterns to improve BCI designs: Unified theory and new algorithms. IEEE Trans. Biomed. Eng. 2010, 58, 355–362. [Google Scholar] [CrossRef] [PubMed]
- Wang, Y.; Gao, S.; Gao, X. Common spatial pattern method for channel selelction in motor imagery based brain-computer interface. In Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China, 17–18 January 2006; pp. 5392–5395. [Google Scholar]
- Blankertz, B.; Tomioka, R.; Lemm, S.; Kawanabe, M.; Muller, K.R. Optimizing spatial filters for robust EEG single-trial analysis. IEEE Signal Process. Mag. 2007, 25, 41–56. [Google Scholar] [CrossRef]
- Ramoser, H.; Muller-Gerking, J.; Pfurtscheller, G. Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehabil. Eng. 2000, 8, 441–446. [Google Scholar] [CrossRef]
- Viola, F.C.; Debener, S.; Thorne, J.; Schneider, T.R. Using ICA for the analysis of multi-channel EEG data. InSimultaneous EEG and fMRI: Recording, Analysis, and Application: Recording, Analysis, and Application; Oxford University Press: New York, NY, USA, 2010; pp. 121–133. [Google Scholar]
- Atti, I.; Belardinelli, P.; Ilmoniemi, R.J.; Metsomaa, J. Measuring the accuracy of ICA-based artifact removal from TMS-evoked potentials. Brain Stimul. 2024, 17, 10–18. [Google Scholar] [CrossRef]
- Mammone, N.; La Foresta, F.; Morabito, F.C. Automatic artifact rejection from multichannel scalp EEG by wavelet ICA. IEEE Sens. J. 2011, 12, 533–542. [Google Scholar] [CrossRef]
- Yi, Y.; Billor, N.; Ekstrom, A.; Zheng, J. CW_ICA: An efficient dimensionality determination method for independent component analysis. Sci. Rep. 2024, 14, 143. [Google Scholar] [CrossRef] [PubMed]
- Durka, P.; Kuś, R.; Żygierewicz, J.; Michalska, M.; Milanowski, P.; Łabęcki, M.; Sputek, T.; Laszuk, D.; Duszyk, A.; Kruszyński, M. User-centered design of brain-computer interfaces: OpenBCI. pl and BCI Appliance. Bull. Pol. Acad. Sci. Tech. Sci. 2012, 60, 427–431. [Google Scholar] [CrossRef]
- Homan, R.W.; Herman, J.; Purdy, P. Cerebral location of international 10–20 system electrode placement. Electroencephalogr. Clin. Neurophysiol. 1987, 66, 376–382. [Google Scholar] [CrossRef]
- McFarland, D.J.; McCane, L.M.; Wolpaw, J.R. EEG-based communication and control: Short-term role of feedback. IEEE Trans. Rehabil. Eng. 1998, 6, 7–11. [Google Scholar] [CrossRef]
- Kothe, C.; Shirazi, S.Y.; Stenner, T.; Medine, D.; Boulay, C.; Crivich, M.I.; Mullen, T.; Delorme, A.; Makeig, S. The lab streaming layer for synchronized multimodal recording. bioRxiv 2024. [Google Scholar] [CrossRef]
- Brainard, D.H.; Vision, S. The psychophysics toolbox. Spat. Vis. 1997, 10, 433–436. [Google Scholar] [CrossRef]
- The MathWorks, Inc. Matlab; The MathWorks: Natick, MA, USA, 2012; Volume 9. [Google Scholar]
- Zhang, Y.; Zhou, G.; Jin, J.; Wang, X.; Cichocki, A. Frequency recognition in SSVEP-based BCI using multiset canonical correlation analysis. Int. J. Neural Syst. 2014, 24, 1450013. [Google Scholar] [CrossRef] [PubMed]
- Banach, K.; Małecki, M.; Rosół, M.; Broniec, A. Brain–computer interface for electric wheelchair based on alpha waves of EEG signal. Bio-Algorithms Med-Syst. 2021, 17, 165–172. [Google Scholar] [CrossRef]
- Ban, N.; Xie, S.; Qu, C.; Chen, X.; Pan, J. Multifunctional robot based on multimodal brain-machine interface. Biomed. Signal Process. Control 2024, 91, 106063. [Google Scholar] [CrossRef]
- Pan, Y.; Chen, J.; Zhang, Y.; Zhang, Y. An efficient CNN-LSTM network with spectral normalization and label smoothing technologies for SSVEP frequency recognition. J. Neural Eng. 2022, 19, 056014. [Google Scholar] [CrossRef]
- Zhang, R.; Xu, Z.; Zhang, L.; Cao, L.; Hu, Y.; Lu, B.; Shi, L.; Yao, D.; Zhao, X. The effect of stimulus number on the recognition accuracy and information transfer rate of SSVEP–BCI in augmented reality. J. Neural Eng. 2022, 19, 036010. [Google Scholar] [CrossRef] [PubMed]
Accuracy (%) | ||||
---|---|---|---|---|
Action | _target Frequency (Hz) | LDA | MLP | SVM |
Forward | 10 | 89.68 | 90.43 | 90.72 |
Backward | 12 | 87.59 | 90.83 | 90.20 |
Left | 15 | 88.27 | 88.67 | 90.33 |
Right | 20 | 86.06 | 87.82 | 88.27 |
Average | 87.90 | 89.43 | 89.88 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Chen, Y.; Shi, X.; De Silva, V.; Dogan, S. Steady-State Visual Evoked Potential-Based Brain–Computer Interface System for Enhanced Human Activity Monitoring and Assessment. Sensors 2024, 24, 7084. https://doi.org/10.3390/s24217084
Chen Y, Shi X, De Silva V, Dogan S. Steady-State Visual Evoked Potential-Based Brain–Computer Interface System for Enhanced Human Activity Monitoring and Assessment. Sensors. 2024; 24(21):7084. https://doi.org/10.3390/s24217084
Chicago/Turabian StyleChen, Yuankun, Xiyu Shi, Varuna De Silva, and Safak Dogan. 2024. "Steady-State Visual Evoked Potential-Based Brain–Computer Interface System for Enhanced Human Activity Monitoring and Assessment" Sensors 24, no. 21: 7084. https://doi.org/10.3390/s24217084
APA StyleChen, Y., Shi, X., De Silva, V., & Dogan, S. (2024). Steady-State Visual Evoked Potential-Based Brain–Computer Interface System for Enhanced Human Activity Monitoring and Assessment. Sensors, 24(21), 7084. https://doi.org/10.3390/s24217084