Automatic three-dimensional segmentation of liver and tumors regions based on conditional generative adversarial networks_Journal of Biomedical Engineering

Authors：

ZHANG Zelin , LI Baoming ,  XU Jun

Jiangsu Key Laboratory of Big Data Analysis, Nanjing University of Information Science and Technology, Nanjing 210044, P.R.China;

Corresponding author：

XU Jun, Email: xujung@gmail.com

Keywords：

three-dimensional automatic segmentation of liver and tumor regions; conditional generative adversarial networks; deep learning; computed tomography

DOI：

10.7507/1001-5515.201912077

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

The three-dimensional (3D) liver and tumor segmentation of liver computed tomography (CT) has very important clinical value for assisting doctors in diagnosis and prognosis. This paper proposes a tumor 3D conditional generation confrontation segmentation network (T3scGAN) based on conditional generation confrontation network (cGAN), and at the same time, a coarse-to-fine 3D automatic segmentation framework is used to accurately segment liver and tumor area. This paper uses 130 cases in the 2017 Liver and Tumor Segmentation Challenge (LiTS) public data set to train, verify and test the T3scGAN model. Finally, the average Dice coefficients of the validation set and test set segmented in the 3D liver regions were 0.963 and 0.961, respectively, while the average Dice coefficients of the validation set and test set segmented in the 3D tumor regions were 0.819 and 0.796, respectively. Experimental results show that the proposed T3scGAN model can effectively segment the 3D liver and its tumor regions, so it can better assist doctors in the accurate diagnosis and treatment of liver cancer.

Citation： ZHANG Zelin, LI Baoming, XU Jun. Automatic three-dimensional segmentation of liver and tumors regions based on conditional generative adversarial networks. Journal of Biomedical Engineering, 2021, 38(1): 80-88. doi: 10.7507/1001-5515.201912077 Copy

1.	Stawiaski J, Decenciere E, Bidault F. Interactive liver tumor segmentation using graph-cuts and watershed//11th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2008), New York: Springer, 2008.
2.	Smeets D, Loeckx D, Stijnen B, et al. Semi-automatic level set segmentation of liver tumors combining a spiral-scanning technique with supervised fuzzy pixel classification. Med Image Anal, 2010, 14(1): 13-20.
3.	Li Bingnan, Chui C K, Chang S, et al. A new unified level set method for semi-automatic liver tumor segmentation on contrast-enhanced CT images. Expert Syst Appl, 2012, 39(10): 9661-9668.
4.	Li C, Wang X, Eberl S, et al. A likelihood and local constraint level set model for liver tumor segmentation from CT volumes. IEEE Trans Biomed Eng, 2013, 60(10): 2967-2977.
5.	Zhang Xing, Tian Jie, Xiang Dehui, et al. Interactive liver tumor segmentation from CT scans using support vector classification with watershed//2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston: IEEE, 2011: 6005-6008.
6.	Evgeniou T, Pontil M. Support vector machines: theory and applications//Proceedings of the 1999 Advanced Course on Artificial Intelligence, Berlin: Springer, 1999: 249-257.
7.	Li W. Automatic segmentation of liver tumor in CT images with deep convolutional neural networks. Journal of Computer and Communications, 2015, 3(11): 146.
8.	Li X, Chen H, Qi X, et al. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging, 2018, 37(12): 2663-2674.
9.	Bi L, Kim J, Kumar A, et al. Automatic liver lesion detection using cascaded deep residual networks, arXiv preprint, 2017. arXiv: 1704.02703.
10.	He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016: 770-778.
11.	Gruber N, Antholzer S, Jaschke W, et al. A joint deep learning approach for automated liver and tumor segmentation, arXiv preprint, 2019. arXiv: 1902.07971.
12.	Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation//International Conference on Medical image computing and computer-assisted intervention, Munich: Springer, 2015: 234-241.
13.	Dey R, Hong Y. Hybrid cascaded neural network for liver lesion segmentation, arXiv preprint, 2019. arXiv: 1909.04797.
14.	Deng Z, Guo Q, Zhu Z. Dynamic regulation of level set parameters using 3D convolutional neural network for liver tumor segmentation. J Healthc Eng, 2019: 4321645.
15.	Lu Fang, Wu Fa, Hu Peijun, et al. Automatic 3D liver location and segmentation via convolutional neural network and graph cut. Int J Comput Assist Radiol Surg, 2017, 12(2): 171-182.
16.	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Advances in neural information processing systems, Montreal: NIPS, 2014: 2672-2680.
17.	Gauthier J. Conditional generative adversarial nets for convolutional face generation. Class Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter Semester, 2014, 2014(5): 2.
18.	Isola P, Zhu Junyan, Zhou Tinghui, et al. Image-to-Image translation with conditional adversarial networks//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii: IEEE, 2017: 5967-5976.
19.	Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston: IEEE, 2015: 3431-3440.
20.	Çiçek Ö, Abdulkadir A, Lienkamp S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation//International Conference on Medical Image Computing and Computer-Assisted Intervention, Greece: Springer, 2016: 424-432.
21.	Zhou Zongwei, Siddiquee M M R, Tajbakhsh N, et al. Unet++: a nested u-net architecture for medical image segmentation// Stoyanov D. et al. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support (DLMIA). Springer, 2018: 3-11.
22.	Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift, arXiv preprint, 2015. arXiv: 1502.03167.
23.	Wu Y, He K. Group normalization//Proceedings of the European Conference on Computer Vision(ECCV), Munich: IEEE, 2018: 3-19.
24.	Moore R C, DeNero J. L1 and L2 regularization for multiclass hinge loss models//Symposium on Machine Learning in Speech and Language Processing, Bellevue: ISCA, 2013. CiteSeerX. psu: 10.1. 1.296. 5923.
25.	Tustison N J, Gee J C. Introducing dice, jaccard, and other label overlap measures to ITK, Insight J, 2009. http://hdl.handle.net/10380/3141.
26.	Yushkevich P A, GAO Yang, Gerig G. ITK-SNAP: an interactive tool for semi-automatic segmentation of multi-modality biomedical images//2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Florida: IEEE, 2016: 3342-3345.
27.	Yushkevich P A, Gerig G. ITK-SNAP: an intractive medical image segmentation tool to meet the need for expert-guided segmentation of complex medical images. IEEE Pulse, 2017, 8(4): 54-57.
28.	Kingma D P, Ba J. Adam: a method for stochastic optimization, arXiv preprint, 2014. arXiv: 1412.6980.
29.	LeCun Y, Bengio Y, Hinton G. Deep learning. nature, 2015, 521(7553): 436-444.
30.	Milletari F, Navab N, Ahmadi S A. V-net: fully convolutional neural networks for volumetric medical image segmentation//2016 Fourth International Conference on 3D Vision (3DV), Stanford: IEEE, 2016: 565-571.

1. Stawiaski J, Decenciere E, Bidault F. Interactive liver tumor segmentation using graph-cuts and watershed//11th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2008), New York: Springer, 2008.
2. Smeets D, Loeckx D, Stijnen B, et al. Semi-automatic level set segmentation of liver tumors combining a spiral-scanning technique with supervised fuzzy pixel classification. Med Image Anal, 2010, 14(1): 13-20.
3. Li Bingnan, Chui C K, Chang S, et al. A new unified level set method for semi-automatic liver tumor segmentation on contrast-enhanced CT images. Expert Syst Appl, 2012, 39(10): 9661-9668.
4. Li C, Wang X, Eberl S, et al. A likelihood and local constraint level set model for liver tumor segmentation from CT volumes. IEEE Trans Biomed Eng, 2013, 60(10): 2967-2977.
5. Zhang Xing, Tian Jie, Xiang Dehui, et al. Interactive liver tumor segmentation from CT scans using support vector classification with watershed//2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Boston: IEEE, 2011: 6005-6008.
6. Evgeniou T, Pontil M. Support vector machines: theory and applications//Proceedings of the 1999 Advanced Course on Artificial Intelligence, Berlin: Springer, 1999: 249-257.
7. Li W. Automatic segmentation of liver tumor in CT images with deep convolutional neural networks. Journal of Computer and Communications, 2015, 3(11): 146.
8. Li X, Chen H, Qi X, et al. H-DenseUNet: hybrid densely connected UNet for liver and tumor segmentation from CT volumes. IEEE Trans Med Imaging, 2018, 37(12): 2663-2674.
9. Bi L, Kim J, Kumar A, et al. Automatic liver lesion detection using cascaded deep residual networks, arXiv preprint, 2017. arXiv: 1704.02703.
10. He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas: IEEE, 2016: 770-778.
11. Gruber N, Antholzer S, Jaschke W, et al. A joint deep learning approach for automated liver and tumor segmentation, arXiv preprint, 2019. arXiv: 1902.07971.
12. Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation//International Conference on Medical image computing and computer-assisted intervention, Munich: Springer, 2015: 234-241.
13. Dey R, Hong Y. Hybrid cascaded neural network for liver lesion segmentation, arXiv preprint, 2019. arXiv: 1909.04797.
14. Deng Z, Guo Q, Zhu Z. Dynamic regulation of level set parameters using 3D convolutional neural network for liver tumor segmentation. J Healthc Eng, 2019: 4321645.
15. Lu Fang, Wu Fa, Hu Peijun, et al. Automatic 3D liver location and segmentation via convolutional neural network and graph cut. Int J Comput Assist Radiol Surg, 2017, 12(2): 171-182.
16. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets//Advances in neural information processing systems, Montreal: NIPS, 2014: 2672-2680.
17. Gauthier J. Conditional generative adversarial nets for convolutional face generation. Class Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter Semester, 2014, 2014(5): 2.
18. Isola P, Zhu Junyan, Zhou Tinghui, et al. Image-to-Image translation with conditional adversarial networks//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii: IEEE, 2017: 5967-5976.
19. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston: IEEE, 2015: 3431-3440.
20. Çiçek Ö, Abdulkadir A, Lienkamp S S, et al. 3D U-Net: learning dense volumetric segmentation from sparse annotation//International Conference on Medical Image Computing and Computer-Assisted Intervention, Greece: Springer, 2016: 424-432.
21. Zhou Zongwei, Siddiquee M M R, Tajbakhsh N, et al. Unet++: a nested u-net architecture for medical image segmentation// Stoyanov D. et al. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support (DLMIA). Springer, 2018: 3-11.
22. Ioffe S, Szegedy C. Batch normalization: accelerating deep network training by reducing internal covariate shift, arXiv preprint, 2015. arXiv: 1502.03167.
23. Wu Y, He K. Group normalization//Proceedings of the European Conference on Computer Vision(ECCV), Munich: IEEE, 2018: 3-19.
24. Moore R C, DeNero J. L1 and L2 regularization for multiclass hinge loss models//Symposium on Machine Learning in Speech and Language Processing, Bellevue: ISCA, 2013. CiteSeerX. psu: 10.1. 1.296. 5923.
25. Tustison N J, Gee J C. Introducing dice, jaccard, and other label overlap measures to ITK, Insight J, 2009. http://hdl.handle.net/10380/3141.
26. Yushkevich P A, GAO Yang, Gerig G. ITK-SNAP: an interactive tool for semi-automatic segmentation of multi-modality biomedical images//2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Florida: IEEE, 2016: 3342-3345.
27. Yushkevich P A, Gerig G. ITK-SNAP: an intractive medical image segmentation tool to meet the need for expert-guided segmentation of complex medical images. IEEE Pulse, 2017, 8(4): 54-57.
28. Kingma D P, Ba J. Adam: a method for stochastic optimization, arXiv preprint, 2014. arXiv: 1412.6980.
29. LeCun Y, Bengio Y, Hinton G. Deep learning. nature, 2015, 521(7553): 436-444.
30. Milletari F, Navab N, Ahmadi S A. V-net: fully convolutional neural networks for volumetric medical image segmentation//2016 Fourth International Conference on 3D Vision (3DV), Stanford: IEEE, 2016: 565-571.

Journal of Biomedical Engineering

Automatic three-dimensional segmentation of liver and tumors regions based on conditional generative adversarial networks

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content