Research on classification of benign and malignant lung nodules based on three-dimensional multi-view squeeze-and-excitation convolutional neural network_Journal of Biomedical Engineering

Authors：

YANG Yang ,  LI Xiaoqin , HAN Zhenbo , FU Jipeng , GAO Bin

Faculty of Environment and Life, Beijing University of Technology, Beijing 100124, P. R. China;

Corresponding author：

LI Xiaoqin, Email: lxq0811@bjut.edu.cn

Keywords：

Image processing; Deep learning; Attention mechanism; Lung nodule; Consistency analysis

DOI：

10.7507/1001-5515.202110059

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Lung cancer is the most threatening tumor disease to human health. Early detection is crucial to improve the survival rate and recovery rate of lung cancer patients. Existing methods use the two-dimensional multi-view framework to learn lung nodules features and simply integrate multi-view features to achieve the classification of benign and malignant lung nodules. However, these methods suffer from the problems of not capturing the spatial features effectively and ignoring the variability of multi-views. Therefore, this paper proposes a three-dimensional (3D) multi-view convolutional neural network (MVCNN) framework. To further solve the problem of different views in the multi-view model, a 3D multi-view squeeze-and-excitation convolution neural network (MVSECNN) model is constructed by introducing the squeeze-and-excitation (SE) module in the feature fusion stage. Finally, statistical methods are used to analyze model predictions and doctor annotations. In the independent test set, the classification accuracy and sensitivity of the model were 96.04% and 98.59% respectively, which were higher than other state-of-the-art methods. The consistency score between the predictions of the model and the pathological diagnosis results was 0.948, which is significantly higher than that between the doctor annotations and the pathological diagnosis results. The methods presented in this paper can effectively learn the spatial heterogeneity of lung nodules and solve the problem of multi-view differences. At the same time, the classification of benign and malignant lung nodules can be achieved, which is of great significance for assisting doctors in clinical diagnosis.

Citation： YANG Yang, LI Xiaoqin, HAN Zhenbo, FU Jipeng, GAO Bin. Research on classification of benign and malignant lung nodules based on three-dimensional multi-view squeeze-and-excitation convolutional neural network. Journal of Biomedical Engineering, 2022, 39(3): 452-461. doi: 10.7507/1001-5515.202110059 Copy

1.	Sung H, Ferlay J, Siegel R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2021, 71(3): 209-249.
2.	Siegel R L, Miller K D, Jemal A. Cancer statistics, 2020. CA Cancer J Clin, 2020, 70(1): 7-30.
3.	LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition. Neural Comput, 1989, 1(4): 541-551.
4.	Hua K L, Hsu C H, Hidayati H C, et al. Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther, 2015, 8: 2015-2022.
5.	Liu Shuang, Xie Yiting, Jirapatnakul A, et al. Pulmonary nodule classification in lung cancer screening with three-dimensional convolutional neural networks. J Med Imaging (Bellingham), 2017, 4(4): 041308.
6.	Marques S, Schiavo F, Ferreira C A, et al. A multi-task CNN approach for lung nodule malignancy classification and characterization. Expert Syst Appl, 2021, 184: 115469.
7.	Astaraki M, Zakko Y, Toma D I, et al. Benign-malignant pulmonary nodule classification in low-dose CT with convolutional features. Phys Med, 2021, 83: 146-153.
8.	Tang Siyuan, Ma Rong, Li Qingqian, et al. Classification of benign and malignant pulmonary nodules based on the multiresolution 3D DPSECN model and semisupervised clustering. IEEE Access, 2021, 9: 43397-43410.
9.	Setio A A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging, 2016, 35(5): 1160-1169.
10.	Roth H R, Lu L, Seff A, et al. A new 2. 5 D representation for lymph node detection using random sets of deep convolutional neural network observations//Medical Image Computing and Computer-Assisted Intervention, Boston: MICCAI, 2014: 520-527.
11.	Zhai Penghua, Tao Yaling, Chen Hao, et al. Multi-task learning for lung nodule classification on chest CT. IEEE Access, 2020, 8: 180317-180327.
12.	Xie Y, Xia Y, Zhang J, et al. Knowledge-based collaborative deep learning for benign-malignant lung nodule classification on chest CT. IEEE Trans Med Imaging, 2019, 38(4): 991-1004.
13.	Armato S G, Mclennan G, Bidaut L, et al. The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys, 2011, 38(2): 915-931.
14.	Manos D, Seely J M, Taylor J, et al. The lung reporting and data system (LU-RADS): a proposal for computed tomography screening. Canadian Association of Radiologists Journal, 2014, 65(2): 121-134.
15.	Reeves A P, Biancardi A M. The lung image database consortium (lidc) nodule size report. Cornell Univ, 2011. (2011-10-27) [2021-09-15]. http: //www.via.cornell.edu/lidc/list3.2.html.
16.	Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res, 2014, 15(1): 1929-1958.
17.	Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.03167, 2015.
18.	Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018: 7132-7141.
19.	Cantor A B. Sample-size calculations for Cohen's kappa. Psychol Methods, 1996, 1(2): 150.
20.	Fleiss J L, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas, 1973, 33(3): 613-619.
21.	Randolph J J. Free-marginal multirater Kappa(multirater K_free): an alternative to Fleiss' fixed-marginal multirater Kappa//the Joensuu Learning and Instruction Symposium, Joensuu, Finland: ERIC, 2005: ED490661.
22.	Willerman B. The adaptation and use of Kendall's coefficient of concordance (W) to sociometric-type rankings. Psychol Bull, 1955, 52(2): 132-133.
23.	Shewaye T N, Mekonnen A A. Benign-malignant lung nodule classification with geometric and appearance histogram features. arXiv preprint arXiv: 1605.08350, 2016.
24.	Kumar D, Chung A G, Shafiee M J, et al. Discovery radiomics for pathologically-proven computed tomography lung cancer prediction. arXiv preprint arXiv: 1509.00117, 2017.
25.	Kumar D, Wong A, Clausi D A. Lung nodule classification using deep features in CT images//2015 12th Conference on Computer and Robot Vision, Halifax: IEEE, 2015: 133-138.
26.	Kang Guixia, Liu Kui, Hou Beibei, et al. 3D multi-view convolutional neural networks for lung nodule classification. PLoS One, 2017, 12(11): 12-22.
27.	Lampert T, Stumpf A, Gancarski P. An empirical study into annotator agreement, ground truth estimation, and algorithm evaluation. IEEE Trans Image Process, 2016, 25(6): 2557-2572.

1. Sung H, Ferlay J, Siegel R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin, 2021, 71(3): 209-249.
2. Siegel R L, Miller K D, Jemal A. Cancer statistics, 2020. CA Cancer J Clin, 2020, 70(1): 7-30.
3. LeCun Y, Boser B, Denker J S, et al. Backpropagation applied to handwritten zip code recognition. Neural Comput, 1989, 1(4): 541-551.
4. Hua K L, Hsu C H, Hidayati H C, et al. Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther, 2015, 8: 2015-2022.
5. Liu Shuang, Xie Yiting, Jirapatnakul A, et al. Pulmonary nodule classification in lung cancer screening with three-dimensional convolutional neural networks. J Med Imaging (Bellingham), 2017, 4(4): 041308.
6. Marques S, Schiavo F, Ferreira C A, et al. A multi-task CNN approach for lung nodule malignancy classification and characterization. Expert Syst Appl, 2021, 184: 115469.
7. Astaraki M, Zakko Y, Toma D I, et al. Benign-malignant pulmonary nodule classification in low-dose CT with convolutional features. Phys Med, 2021, 83: 146-153.
8. Tang Siyuan, Ma Rong, Li Qingqian, et al. Classification of benign and malignant pulmonary nodules based on the multiresolution 3D DPSECN model and semisupervised clustering. IEEE Access, 2021, 9: 43397-43410.
9. Setio A A, Ciompi F, Litjens G, et al. Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans Med Imaging, 2016, 35(5): 1160-1169.
10. Roth H R, Lu L, Seff A, et al. A new 2. 5 D representation for lymph node detection using random sets of deep convolutional neural network observations//Medical Image Computing and Computer-Assisted Intervention, Boston: MICCAI, 2014: 520-527.
11. Zhai Penghua, Tao Yaling, Chen Hao, et al. Multi-task learning for lung nodule classification on chest CT. IEEE Access, 2020, 8: 180317-180327.
12. Xie Y, Xia Y, Zhang J, et al. Knowledge-based collaborative deep learning for benign-malignant lung nodule classification on chest CT. IEEE Trans Med Imaging, 2019, 38(4): 991-1004.
13. Armato S G, Mclennan G, Bidaut L, et al. The lung image database consortium (LIDC) and image database resource initiative (IDRI): a completed reference database of lung nodules on CT scans. Med Phys, 2011, 38(2): 915-931.
14. Manos D, Seely J M, Taylor J, et al. The lung reporting and data system (LU-RADS): a proposal for computed tomography screening. Canadian Association of Radiologists Journal, 2014, 65(2): 121-134.
15. Reeves A P, Biancardi A M. The lung image database consortium (lidc) nodule size report. Cornell Univ, 2011. (2011-10-27) [2021-09-15]. http: //www.via.cornell.edu/lidc/list3.2.html.
16. Srivastava N, Hinton G, Krizhevsky A, et al. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res, 2014, 15(1): 1929-1958.
17. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv: 1502.03167, 2015.
18. Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation networks//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City: IEEE, 2018: 7132-7141.
19. Cantor A B. Sample-size calculations for Cohen's kappa. Psychol Methods, 1996, 1(2): 150.
20. Fleiss J L, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas, 1973, 33(3): 613-619.
21. Randolph J J. Free-marginal multirater Kappa(multirater K_free): an alternative to Fleiss' fixed-marginal multirater Kappa//the Joensuu Learning and Instruction Symposium, Joensuu, Finland: ERIC, 2005: ED490661.
22. Willerman B. The adaptation and use of Kendall's coefficient of concordance (W) to sociometric-type rankings. Psychol Bull, 1955, 52(2): 132-133.
23. Shewaye T N, Mekonnen A A. Benign-malignant lung nodule classification with geometric and appearance histogram features. arXiv preprint arXiv: 1605.08350, 2016.
24. Kumar D, Chung A G, Shafiee M J, et al. Discovery radiomics for pathologically-proven computed tomography lung cancer prediction. arXiv preprint arXiv: 1509.00117, 2017.
25. Kumar D, Wong A, Clausi D A. Lung nodule classification using deep features in CT images//2015 12th Conference on Computer and Robot Vision, Halifax: IEEE, 2015: 133-138.
26. Kang Guixia, Liu Kui, Hou Beibei, et al. 3D multi-view convolutional neural networks for lung nodule classification. PLoS One, 2017, 12(11): 12-22.
27. Lampert T, Stumpf A, Gancarski P. An empirical study into annotator agreement, ground truth estimation, and algorithm evaluation. IEEE Trans Image Process, 2016, 25(6): 2557-2572.

Journal of Biomedical Engineering

Research on classification of benign and malignant lung nodules based on three-dimensional multi-view squeeze-and-excitation convolutional neural network

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content