Parameter transfer learning based on shallow visual geometry group network and its application in motor imagery classification_Journal of Biomedical Engineering

Authors：

XU Dongqin ¹ ,  LI Ming’ai ^1,2,3

1. Faculty of Information Technology, Beijing University of Technology, Beijing 100124, P. R. China;
2. Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing 100124, P. R. China;
3. Engineering Research Center of Digital Community, Ministry of Education, Beijing 100124, P. R. China;

Corresponding author：

LI Ming’ai, Email: limingai@bjut.edu.cn

Keywords：

Motor imagery; Parameter transfer learning; Visual geometry group network; Pearson correlation coefficient; Transfer strategy

DOI：

10.7507/1001-5515.202108060

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Transfer learning is provided with potential research value and application prospect in motor imagery electroencephalography (MI-EEG)-based brain-computer interface (BCI) rehabilitation system, and the source domain classification model and transfer strategy are the two important aspects that directly affect the performance and transfer efficiency of the target domain model. Therefore, we propose a parameter transfer learning method based on shallow visual geometry group network (PTL-sVGG). First, Pearson correlation coefficient is used to screen the subjects of the source domain, and the short-time Fourier transform is performed on the MI-EEG data of each selected subject to acquire the time-frequency spectrogram images (TFSI). Then, the architecture of VGG-16 is simplified and the block design is carried out, and the modified sVGG model is pre-trained with TFSI of source domain. Furthermore, a block-based frozen-fine-tuning transfer strategy is designed to quickly find and freeze the block with the greatest contribution to sVGG model, and the remaining blocks are fine-tuned by using TFSI of target subjects to obtain the target domain classification model. Extensive experiments are conducted based on public MI-EEG datasets, the average recognition rate and Kappa value of PTL-sVGG are 94.9% and 0.898, respectively. The results show that the subjects’ optimization is beneficial to improve the model performance in source domain, and the block-based transfer strategy can enhance the transfer efficiency, realizing the rapid and effective transfer of model parameters across subjects on the datasets with different number of channels. It is beneficial to reduce the calibration time of BCI system, which promote the application of BCI technology in rehabilitation engineering.

Citation： XU Dongqin, LI Ming’ai. Parameter transfer learning based on shallow visual geometry group network and its application in motor imagery classification. Journal of Biomedical Engineering, 2022, 39(1): 28-38. doi: 10.7507/1001-5515.202108060 Copy

1.	Saha S, Baumert M. Intra- and inter-subject variability in EEG-based sensorimotor brain computer interface: a review. Front Comput Neurosci, 2019, 13: 87.
2.	霍首君, 郝琰, 石慧宇, 等. 基于深度卷积网络的运动想象脑电信号模式识别. 计算机应用, 2021, 41(4): 1042-1048.
3.	Shajil N, Mohan S, Srinivasan P, et al. Multiclass classification of spatially filtered motor imagery EEG signals using convolutional neural network for BCI based applications. J Med Biol Eng, 2020, 40(5): 663-672.
4.	Hou Y, Zhou L, Jia S, et al. A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. J Neural Eng, 2020, 17(1): 016048.
5.	Ahn M, Cho H, Ahn S, et al. High theta and low alpha powers may be indicative of BCI-illiteracy in motor imagery. PLoS One, 2013, 8(11): e80886.
6.	Lotte F, Bougrain L, Cichocki A, et al. A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update. J Neural Eng, 2018, 15: 031005.
7.	庄福振, 罗平, 何清, 等. 迁移学习研究进展. 软件学报, 2015, 26(1): 26-39.
8.	Zhang K, Xu G, Chen L, et al. Instance transfer subject-dependent strategy for motor imagery signal classification using deep convolutional neural networks. Comput Math Methods Med, 2020: 1683013.
9.	Ju Ce, Gao Dashan, Mane R, et al. Federated transfer learning for EEG signal classification//2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Montreal, Canada, 2020: 3040-3045.
10.	Zhu Xuyang, Li Peiyang, Li Cunbo, et al. Separated channel convolutional neural network to realize the training free motor imagery BCI systems. Biomedical Signal Processing and Control, 2019, 49: 396-403.
11.	Wang Peitao, Lu Jun, Zhang Bin, et al. A review on transfer learning for brain-computer interface classification//2015 5th International Conference on Information Science and Technology (ICIST). Changsha, China: IEEE, 2015: 315-322.
12.	Zhang K, Xu G, Zheng X, et al. Application of transfer learning in EEG decoding based on brain-computer interfaces: a review. Sensors, 2020, 20(21): 6321.
13.	Zhao D, Tang F, Si B, et al. Learning joint space-time-frequency features for EEG decoding on small labeled data. Neural Netw, 2019, 114: 67-77.
14.	Zanini P, Congedo M, Jutten C, et al. Transfer learning: a riemannian geometry framework with applications to brain-computer interfaces. IEEE Trans Biomed Eng, 2018, 65(5): 1107-1116.
15.	He H, Wu D. Transfer learning for brain-computer interfaces: a euclidean space data alignment approach. IEEE Trans Biomed Eng, 2020, 67(2): 399-410.
16.	Xu L, Xu M, Ke Y, et al. Cross-dataset variability problem in EEG decoding with deep learning. Front Hum Neurosci, 2020, 14: 103.
17.	Kostas D, Rudzicz F. Thinker invariance: enabling deep neural networks for BCI across more people. J Neural Eng, 2020, 17(5): 056008.
18.	Parvan M, Ghiasi A R, Rezaii T Y, et al. Transfer learning based motor imagery classification using convolutional neural networks//2019 27th Iranian Conference on Electrical Engineering (ICEE). Yazd, Iran, 2019: 1825-1828.
19.	Sakhavi S, Guan Cuntai. Convolutional neural network-based transfer learning and knowledge distillation using multi-subject data in motor imagery BCI//2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), IEEE, 2017: 588-591.
20.	Wu H, Niu Y, Li F, et al. A parallel multiscale filter bank convolutional neural networks for motor imagery EEG classification. Front Neurosci, 2019, 13: 1275.
21.	Roy S, Chowdhury A, Mccreadie K, et al. Deep learning based inter-subject continuous decoding of motor imagery for practical brain-computer interfaces. Front Neurosci, 2020, 14: 918.
22.	Zhang Ruilong, Zong Qun, Dou Liqian, et al. Hybrid deep neural network using transfer learning for EEG motor imagery decoding. Biomedical Signal Processing and Control, 2021, 63: 102144.
23.	Özdenizci O, WANG Y e, Koike-Akino T, et al. Transfer learning in brain-computer interfaces with adversarial variational autoencoders//2019 9th International IEEE/EMBS Conference on Neural Engineering (NER). San Francisco, USA, 2019: 207-210.
24.	Kant P, Laskar S H, Hazarika J, et al. CWT based transfer learning for motor imagery classification for brain computer interfaces. J Neurosci Methods, 2020, 345(2020): 108886.
25.	Xu Gaowei, Shen Xiaoang, Chen Sirui, et al. A deep transfer convolutional neural network framework for EEG signal classification. IEEE Access, 2019, 7(1): 112767-112776.
26.	Vidaurre C, Blankertz B. Towards a cure for BCI illiteracy. Brain Topogr, 2010, 23(2): 194-198.
27.	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014, arXiv:1409.1556v6.
28.	Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks?// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge: IEEE, 2014, arXiv: 1411.1792v1.

1. Saha S, Baumert M. Intra- and inter-subject variability in EEG-based sensorimotor brain computer interface: a review. Front Comput Neurosci, 2019, 13: 87.
2. 霍首君, 郝琰, 石慧宇, 等. 基于深度卷积网络的运动想象脑电信号模式识别. 计算机应用, 2021, 41(4): 1042-1048.
3. Shajil N, Mohan S, Srinivasan P, et al. Multiclass classification of spatially filtered motor imagery EEG signals using convolutional neural network for BCI based applications. J Med Biol Eng, 2020, 40(5): 663-672.
4. Hou Y, Zhou L, Jia S, et al. A novel approach of decoding EEG four-class motor imagery tasks via scout ESI and CNN. J Neural Eng, 2020, 17(1): 016048.
5. Ahn M, Cho H, Ahn S, et al. High theta and low alpha powers may be indicative of BCI-illiteracy in motor imagery. PLoS One, 2013, 8(11): e80886.
6. Lotte F, Bougrain L, Cichocki A, et al. A review of classification algorithms for EEG-based brain-computer interfaces: a 10 year update. J Neural Eng, 2018, 15: 031005.
7. 庄福振, 罗平, 何清, 等. 迁移学习研究进展. 软件学报, 2015, 26(1): 26-39.
8. Zhang K, Xu G, Chen L, et al. Instance transfer subject-dependent strategy for motor imagery signal classification using deep convolutional neural networks. Comput Math Methods Med, 2020: 1683013.
9. Ju Ce, Gao Dashan, Mane R, et al. Federated transfer learning for EEG signal classification//2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Montreal, Canada, 2020: 3040-3045.
10. Zhu Xuyang, Li Peiyang, Li Cunbo, et al. Separated channel convolutional neural network to realize the training free motor imagery BCI systems. Biomedical Signal Processing and Control, 2019, 49: 396-403.
11. Wang Peitao, Lu Jun, Zhang Bin, et al. A review on transfer learning for brain-computer interface classification//2015 5th International Conference on Information Science and Technology (ICIST). Changsha, China: IEEE, 2015: 315-322.
12. Zhang K, Xu G, Zheng X, et al. Application of transfer learning in EEG decoding based on brain-computer interfaces: a review. Sensors, 2020, 20(21): 6321.
13. Zhao D, Tang F, Si B, et al. Learning joint space-time-frequency features for EEG decoding on small labeled data. Neural Netw, 2019, 114: 67-77.
14. Zanini P, Congedo M, Jutten C, et al. Transfer learning: a riemannian geometry framework with applications to brain-computer interfaces. IEEE Trans Biomed Eng, 2018, 65(5): 1107-1116.
15. He H, Wu D. Transfer learning for brain-computer interfaces: a euclidean space data alignment approach. IEEE Trans Biomed Eng, 2020, 67(2): 399-410.
16. Xu L, Xu M, Ke Y, et al. Cross-dataset variability problem in EEG decoding with deep learning. Front Hum Neurosci, 2020, 14: 103.
17. Kostas D, Rudzicz F. Thinker invariance: enabling deep neural networks for BCI across more people. J Neural Eng, 2020, 17(5): 056008.
18. Parvan M, Ghiasi A R, Rezaii T Y, et al. Transfer learning based motor imagery classification using convolutional neural networks//2019 27th Iranian Conference on Electrical Engineering (ICEE). Yazd, Iran, 2019: 1825-1828.
19. Sakhavi S, Guan Cuntai. Convolutional neural network-based transfer learning and knowledge distillation using multi-subject data in motor imagery BCI//2017 8th International IEEE/EMBS Conference on Neural Engineering (NER), IEEE, 2017: 588-591.
20. Wu H, Niu Y, Li F, et al. A parallel multiscale filter bank convolutional neural networks for motor imagery EEG classification. Front Neurosci, 2019, 13: 1275.
21. Roy S, Chowdhury A, Mccreadie K, et al. Deep learning based inter-subject continuous decoding of motor imagery for practical brain-computer interfaces. Front Neurosci, 2020, 14: 918.
22. Zhang Ruilong, Zong Qun, Dou Liqian, et al. Hybrid deep neural network using transfer learning for EEG motor imagery decoding. Biomedical Signal Processing and Control, 2021, 63: 102144.
23. Özdenizci O, WANG Y e, Koike-Akino T, et al. Transfer learning in brain-computer interfaces with adversarial variational autoencoders//2019 9th International IEEE/EMBS Conference on Neural Engineering (NER). San Francisco, USA, 2019: 207-210.
24. Kant P, Laskar S H, Hazarika J, et al. CWT based transfer learning for motor imagery classification for brain computer interfaces. J Neurosci Methods, 2020, 345(2020): 108886.
25. Xu Gaowei, Shen Xiaoang, Chen Sirui, et al. A deep transfer convolutional neural network framework for EEG signal classification. IEEE Access, 2019, 7(1): 112767-112776.
26. Vidaurre C, Blankertz B. Towards a cure for BCI illiteracy. Brain Topogr, 2010, 23(2): 194-198.
27. Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. 2014, arXiv:1409.1556v6.
28. Yosinski J, Clune J, Bengio Y, et al. How transferable are features in deep neural networks?// Proceedings of the 27th International Conference on Neural Information Processing Systems. Cambridge: IEEE, 2014, arXiv: 1411.1792v1.

Journal of Biomedical Engineering

Parameter transfer learning based on shallow visual geometry group network and its application in motor imagery classification

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content