Electrophysiological characteristics of emotion arousal difference between stereoscopic and non-stereoscopic virtual reality films_Journal of Biomedical Engineering

Authors：

TIAN Feng ^1,2 , ZHANG Wenrui ¹ ,  LI Yingjie ^3,4

1. Shanghai Film Academy, Shanghai University, Shanghai 200072, P. R. China;
2. Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai University, Shanghai 200072, P. R. China;
3. College of International Education, Shanghai University, Shanghai 200444, P. R. China;
4. Institute of Biomedical Engineering, School of Life Science, Shanghai University, Shanghai 200444, P. R. China;

Corresponding author：

LI Yingjie, Email: liyj@i.shu.edu.cn

Keywords：

Non-stereoscopic virtual reality; Stereoscopic virtual reality; Continuous display; Emotional arousal; Electroencephalogram; Neurocinematics

DOI：

10.7507/1001-5515.202101010

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

There are two modes to display panoramic movies in virtual reality (VR) environment: non-stereoscopic mode (2D) and stereoscopic mode (3D). It has not been fully studied whether there are differences in the activation effect between these two continuous display modes on emotional arousal and what characteristics of the related neural activity are. In this paper, we designed a cognitive psychology experiment in order to compare the effects of VR-2D and VR-3D on emotional arousal by analyzing synchronously collected scalp electroencephalogram signals. We used support vector machine (SVM) to verify the neurophysiological differences between the two modes in VR environment. The results showed that compared with VR-2D films, VR-3D films evoked significantly higher electroencephalogram (EEG) power (mainly reflected in α and β activities). The significantly improved β wave power in VR-3D mode showed that 3D vision brought more intense cortical activity, which might lead to higher arousal. At the same time, the more intense α activity in the occipital region of the brain also suggested that VR-3D films might cause higher visual fatigue. By the means of neurocinematics, this paper demonstrates that EEG activity can well reflect the effects of different vision modes on the characteristics of the viewers’ neural activities. The current study provides theoretical support not only for the future exploration of the image language under the VR perspective, but for future VR film shooting methods and human emotion research.

Citation： TIAN Feng, ZHANG Wenrui, LI Yingjie. Electrophysiological characteristics of emotion arousal difference between stereoscopic and non-stereoscopic virtual reality films. Journal of Biomedical Engineering, 2022, 39(1): 56-66. doi: 10.7507/1001-5515.202101010 Copy

1.	赵沁平. 虚拟现实综述. 中国科学: 信息科学, 2009, 39(1): 2-46.
2.	Bartsch A, Viehoff R. The use of media entertainment and emotional gratification. Procedia Soc Behav Sci, 2010, 5: 2247-2255.
3.	Tussyadiah I P, Wang D, Jung T H, et al. Virtual reality, presence, and attitude change: Empirical evidence from tourism. Tourism Manage, 2018, 66: 140-154.
4.	Diemer J, Alpers G W, Peperkorn H M, et al. The impact of perception and presence on emotional reactions: a review of research in virtual reality. Front Psychol, 2015, 6: 26.
5.	Chao C J, Wu S Y, Yau Y J, et al. Effects of three-dimensional virtual reality and traditional training methods on mental workload and training performance. Hum Factor Ergon Man, 2017, 27(4): 187-196.
6.	徐小萍, 吕健, 金昱潼, 等. 用户认知驱动的VR自然交互认知负荷研究. 计算机应用研究, 2020, 37(7): 1958-1963.
7.	Montana J I, Matamala-Gomez M, Maisto M, et al. The benefits of emotion regulation interventions in virtual reality for the improvement of wellbeing in adults and older adults: a systematic review. J Clin Med, 2020, 9(2): 500.
8.	Li B J, Bailenson J N, Pines A, et al. A public database of immersive VR videos with corresponding ratings of arousal, valence, and correlations between head movements and self report measures. Front Psychol, 2017, 8: 2116.
9.	Hasson U, Landesman O, Knappmeyer B, et al. Neurocinematics: The neuroscience of film. Projections, 2008, 2(1): 1-26.
10.	Schubring D, Kraus M, Stolz C, et al. Virtual reality potentiates emotion and task effects of alpha/beta brain oscillations. Brain Sci, 2020, 10(8): 537.
11.	He L, Li H, Xue T, et al. Am I in the theater?: usability study of live performance based virtual reality// The 24th ACM Symposium. Tokyo: ACM, 2018: 1-11.
12.	Bilgin P, Agres K, Robinson N, et al. A comparative study of mental states in 2D and 3D virtual environments using EEG// 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Bari: IEEE, 2019: 2833-2838.
13.	Kweon S H, Kweon H J, Kim S, et al. A brain wave research on VR (virtual reality) usage: comparison between VR and 2D video in EEG measurement// International Conference on Applied Human Factors and Ergonomics. Los Angeles: Springer, Cham, 2017: 194-203.
14.	Kober S E, Kurzmann J, Neuper C. Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study. Int J Psychophysiol, 2011, 83(3): 365-374.
15.	Slobounov S M, Ray W, Johnson B, et al. Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. Int J Psychophysiol, 2015, 95(3): 254-260.
16.	Tian Feng, Hua Minlei, Zhang Wenrui, et al. Emotional arousal in 2D versus 3D virtual reality environments. PLoS One, 2021, 16(9): e0256211.
17.	Bradley M M, Lang P J. Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry, 1994, 25(1): 49-59.
18.	Yao D, Qin Y, Hu S, et al. Which reference should we use for EEG and ERP practice?. Brain Topogr, 2019, 32(4): 530-549.
19.	刘冲, 赵海滨, 李春胜, 等. 脑电信号频带能量特征的提取方法及分类研究. 系统仿真学报, 2012, 24(12): 2496-2499.
20.	奉国和. SVM分类核函数及参数选择比较. 计算机工程与应用, 2011, 47(3): 123-124, 128.
21.	章悦, 夏春明, 谢佳智, 等. 特征与分类算法在基于肌音信号的头部运动分类中的对比研究. 数据采集与处理, 2020, 35(4): 711-719.
22.	侯晓菲. 面向影视的2D与VR正负性情绪比较研究. 上海: 上海大学, 2020.
23.	Watson D, Clark L A, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol, 1988, 54(6): 1063-1070.
24.	Kunaharan S, Halpin S, Sitharthan T, et al. Do EEG and startle reflex modulation vary with self-reported aggression in response to violent images?. Brain Sci, 2019, 9(11): 298.
25.	Perrett D I, Mistlin A J, Chitty A J. Visual neurones responsive to faces. Trends Neurosci, 1987, 10(9): 358-364.
26.	Roy J P, Wurtz R H. The role of disparity-sensitive cortical neurons in signalling the direction of self-motion. Nature, 1990, 348(6297): 160-162.
27.	刘烨, 付秋芳, 傅小兰. 认知与情绪的交互作用. 科学通报, 2009, 54(18): 2783-2796.
28.	George M S, Ketter T A, Parekh P I, et al. Brain activity during transient sadness and happiness in healthy women. Am J Psychiat, 1995, 152(3): 341-351.
29.	Lane R D, Reiman E M, Ahern G L, et al. Neuroanatomical correlates of happiness, sadness, and disgust. Am J Psychiat, 1997, 154(7): 926-933.
30.	Lane R D, Reiman E M, Axelrod B, et al. Neural correlates of levels of emotional awareness: Evidence of an interaction between emotion and attention in the anterior cingulate cortex. J Cogn Neurosci, 1998, 10(4): 525-535.
31.	Damasio A R, Grabowski T J, Bechara A, et al. Subcortical and cortical brain activity during the feeling of self-generated emotions. Nat Neurosci, 2000, 3(10): 1049-1056.
32.	Rooney B, Hennessy E. Actually in the cinema: A field study comparing real 3D and 2D movie patrons' attention, emotion, and film satisfaction. Media Psychol, 2013, 16(4): 441-460.
33.	Tan E S H. Entertainment is emotion: The functional architecture of the entertainment experience. Media Psychol, 2008, 11(1): 28-51.
34.	Dennis T A, Solomon B. Frontal EEG and emotion regulation: Electrocortical activity in response to emotional film clips is associated with reduced mood induction and attention interference effects. Biol Psychol, 2010, 85(3): 456-464.
35.	陈曾, 刘光远. 脑电信号在情感识别中的应用. 计算机工程, 2010, 36(9): 168-170.
36.	Jurewicz K, Paluch K, Kublik E, et al. EEG-neurofeedback training of beta band (12-22Hz) affects alpha and beta frequencies - A controlled study of a healthy population. Neuropsychologia, 2017, 108: 13-24.
37.	陈成明, 王慧, 周健, 等. VDT视觉疲劳及其测量方法综述. 人类工效学, 2013, 19(2): 92-95.
38.	Zou B, Liu Y, Guo M, et al. EEG-based assessment of stereoscopic 3D visual fatigue caused by vergence-accommodation conflict. J Disp Technol, 2015, 11(12): 1076-1083.
39.	王嘉辉, 程义, 李焜阳, 等. 3D显示方式与视差对视觉疲劳的影响研究. 中山大学学报(自然科学版), 2013, 52(5): 1-5.
40.	赵彦富, 随力, 李月如. 基于脑电图的脑疲劳检测研究进展. 中国医学物理学杂志, 2019, 36(11): 1312-1316.

1. 赵沁平. 虚拟现实综述. 中国科学: 信息科学, 2009, 39(1): 2-46.
2. Bartsch A, Viehoff R. The use of media entertainment and emotional gratification. Procedia Soc Behav Sci, 2010, 5: 2247-2255.
3. Tussyadiah I P, Wang D, Jung T H, et al. Virtual reality, presence, and attitude change: Empirical evidence from tourism. Tourism Manage, 2018, 66: 140-154.
4. Diemer J, Alpers G W, Peperkorn H M, et al. The impact of perception and presence on emotional reactions: a review of research in virtual reality. Front Psychol, 2015, 6: 26.
5. Chao C J, Wu S Y, Yau Y J, et al. Effects of three-dimensional virtual reality and traditional training methods on mental workload and training performance. Hum Factor Ergon Man, 2017, 27(4): 187-196.
6. 徐小萍, 吕健, 金昱潼, 等. 用户认知驱动的VR自然交互认知负荷研究. 计算机应用研究, 2020, 37(7): 1958-1963.
7. Montana J I, Matamala-Gomez M, Maisto M, et al. The benefits of emotion regulation interventions in virtual reality for the improvement of wellbeing in adults and older adults: a systematic review. J Clin Med, 2020, 9(2): 500.
8. Li B J, Bailenson J N, Pines A, et al. A public database of immersive VR videos with corresponding ratings of arousal, valence, and correlations between head movements and self report measures. Front Psychol, 2017, 8: 2116.
9. Hasson U, Landesman O, Knappmeyer B, et al. Neurocinematics: The neuroscience of film. Projections, 2008, 2(1): 1-26.
10. Schubring D, Kraus M, Stolz C, et al. Virtual reality potentiates emotion and task effects of alpha/beta brain oscillations. Brain Sci, 2020, 10(8): 537.
11. He L, Li H, Xue T, et al. Am I in the theater?: usability study of live performance based virtual reality// The 24th ACM Symposium. Tokyo: ACM, 2018: 1-11.
12. Bilgin P, Agres K, Robinson N, et al. A comparative study of mental states in 2D and 3D virtual environments using EEG// 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC). Bari: IEEE, 2019: 2833-2838.
13. Kweon S H, Kweon H J, Kim S, et al. A brain wave research on VR (virtual reality) usage: comparison between VR and 2D video in EEG measurement// International Conference on Applied Human Factors and Ergonomics. Los Angeles: Springer, Cham, 2017: 194-203.
14. Kober S E, Kurzmann J, Neuper C. Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study. Int J Psychophysiol, 2011, 83(3): 365-374.
15. Slobounov S M, Ray W, Johnson B, et al. Modulation of cortical activity in 2D versus 3D virtual reality environments: An EEG study. Int J Psychophysiol, 2015, 95(3): 254-260.
16. Tian Feng, Hua Minlei, Zhang Wenrui, et al. Emotional arousal in 2D versus 3D virtual reality environments. PLoS One, 2021, 16(9): e0256211.
17. Bradley M M, Lang P J. Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry, 1994, 25(1): 49-59.
18. Yao D, Qin Y, Hu S, et al. Which reference should we use for EEG and ERP practice?. Brain Topogr, 2019, 32(4): 530-549.
19. 刘冲, 赵海滨, 李春胜, 等. 脑电信号频带能量特征的提取方法及分类研究. 系统仿真学报, 2012, 24(12): 2496-2499.
20. 奉国和. SVM分类核函数及参数选择比较. 计算机工程与应用, 2011, 47(3): 123-124, 128.
21. 章悦, 夏春明, 谢佳智, 等. 特征与分类算法在基于肌音信号的头部运动分类中的对比研究. 数据采集与处理, 2020, 35(4): 711-719.
22. 侯晓菲. 面向影视的2D与VR正负性情绪比较研究. 上海: 上海大学, 2020.
23. Watson D, Clark L A, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol, 1988, 54(6): 1063-1070.
24. Kunaharan S, Halpin S, Sitharthan T, et al. Do EEG and startle reflex modulation vary with self-reported aggression in response to violent images?. Brain Sci, 2019, 9(11): 298.
25. Perrett D I, Mistlin A J, Chitty A J. Visual neurones responsive to faces. Trends Neurosci, 1987, 10(9): 358-364.
26. Roy J P, Wurtz R H. The role of disparity-sensitive cortical neurons in signalling the direction of self-motion. Nature, 1990, 348(6297): 160-162.
27. 刘烨, 付秋芳, 傅小兰. 认知与情绪的交互作用. 科学通报, 2009, 54(18): 2783-2796.
28. George M S, Ketter T A, Parekh P I, et al. Brain activity during transient sadness and happiness in healthy women. Am J Psychiat, 1995, 152(3): 341-351.
29. Lane R D, Reiman E M, Ahern G L, et al. Neuroanatomical correlates of happiness, sadness, and disgust. Am J Psychiat, 1997, 154(7): 926-933.
30. Lane R D, Reiman E M, Axelrod B, et al. Neural correlates of levels of emotional awareness: Evidence of an interaction between emotion and attention in the anterior cingulate cortex. J Cogn Neurosci, 1998, 10(4): 525-535.
31. Damasio A R, Grabowski T J, Bechara A, et al. Subcortical and cortical brain activity during the feeling of self-generated emotions. Nat Neurosci, 2000, 3(10): 1049-1056.
32. Rooney B, Hennessy E. Actually in the cinema: A field study comparing real 3D and 2D movie patrons' attention, emotion, and film satisfaction. Media Psychol, 2013, 16(4): 441-460.
33. Tan E S H. Entertainment is emotion: The functional architecture of the entertainment experience. Media Psychol, 2008, 11(1): 28-51.
34. Dennis T A, Solomon B. Frontal EEG and emotion regulation: Electrocortical activity in response to emotional film clips is associated with reduced mood induction and attention interference effects. Biol Psychol, 2010, 85(3): 456-464.
35. 陈曾, 刘光远. 脑电信号在情感识别中的应用. 计算机工程, 2010, 36(9): 168-170.
36. Jurewicz K, Paluch K, Kublik E, et al. EEG-neurofeedback training of beta band (12-22Hz) affects alpha and beta frequencies - A controlled study of a healthy population. Neuropsychologia, 2017, 108: 13-24.
37. 陈成明, 王慧, 周健, 等. VDT视觉疲劳及其测量方法综述. 人类工效学, 2013, 19(2): 92-95.
38. Zou B, Liu Y, Guo M, et al. EEG-based assessment of stereoscopic 3D visual fatigue caused by vergence-accommodation conflict. J Disp Technol, 2015, 11(12): 1076-1083.
39. 王嘉辉, 程义, 李焜阳, 等. 3D显示方式与视差对视觉疲劳的影响研究. 中山大学学报(自然科学版), 2013, 52(5): 1-5.
40. 赵彦富, 随力, 李月如. 基于脑电图的脑疲劳检测研究进展. 中国医学物理学杂志, 2019, 36(11): 1312-1316.

Previous Article
Research on depression recognition based on brain function network
Next Article
Effect of lower limb amputation level on aortic hemodynamics: a numerical study

Journal of Biomedical Engineering

Electrophysiological characteristics of emotion arousal difference between stereoscopic and non-stereoscopic virtual reality films

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content