A generative adversarial network-based unsupervised domain adaptation method for magnetic resonance image segmentation_Journal of Biomedical Engineering

Authors：

SUN Yubo ^1,2 , LIU Jianan ^1,2 , SUN Zewen ³ , HAN Jianda ^1,2,4 ,  YU Ningbo ^1,2,4

1. College of Artificial Intelligence, Nankai University, Tianjin 300350, P. R. China;
2. Tianjin Key Laboratory of Intelligent Robotics, Nankai University, Tianjin 300350, P. R. China;
3. Institute of Sports Medicine, Peking University Third Hospital, Beijing 100083, P. R. China;
4. Institute of Intelligence Technology and Robotic Systems, Shenzhen Research Institute of Nankai University, Shenzhen, Guangdong 518083, P. R. China;

Corresponding author：

YU Ningbo, Email: nyu@nankai.edu.cn

Keywords：

Medical image segmentation; Domain shift; Generative adversarial networks; Image level domain adaptation; Feature level domain adaptation

DOI：

10.7507/1001-5515.202203009

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Intelligent medical image segmentation methods have been rapidly developed and applied, while a significant challenge is domain shift. That is, the segmentation performance degrades due to distribution differences between the source domain and the target domain. This paper proposed an unsupervised end-to-end domain adaptation medical image segmentation method based on the generative adversarial network (GAN). A network training and adjustment model was designed, including segmentation and discriminant networks. In the segmentation network, the residual module was used as the basic module to increase feature reusability and reduce model optimization difficulty. Further, it learned cross-domain features at the image feature level with the help of the discriminant network and a combination of segmentation loss with adversarial loss. The discriminant network took the convolutional neural network and used the labels from the source domain, to distinguish whether the segmentation result of the generated network is from the source domain or the target domain. The whole training process was unsupervised. The proposed method was tested with experiments on a public dataset of knee magnetic resonance (MR) images and the clinical dataset from our cooperative hospital. With our method, the mean Dice similarity coefficient (DSC) of segmentation results increased by 2.52% and 6.10% to the classical feature level and image level domain adaptive method. The proposed method effectively improves the domain adaptive ability of the segmentation method, significantly improves the segmentation accuracy of the tibia and femur, and can better solve the domain transfer problem in MR image segmentation.

Citation： SUN Yubo, LIU Jianan, SUN Zewen, HAN Jianda, YU Ningbo. A generative adversarial network-based unsupervised domain adaptation method for magnetic resonance image segmentation. Journal of Biomedical Engineering, 2022, 39(6): 1181-1188. doi: 10.7507/1001-5515.202203009 Copy

1.	Chowdhary C L, Acharjya D P. Segmentation and feature extraction in medical imaging: a systematic review. Procedia Comput Sci, 2020, 167: 26-36.
2.	Hesamian M H, Jia W, He X, et al. Deep learning techniques for medical image segmentation: achievements and challenges. J Digit Imaging, 2019, 32(4): 582-596.
3.	吴玉超, 林岚, 王婧璇, 等. 基于卷积神经网络的语义分割在医学图像中的应用. 生物医学工程学杂志, 2020, 37(3): 533-540.
4.	王玉丽, 赵子健. 基于深度学习的脑图像分割算法研究综述. 生物医学工程学杂志, 2020, 37(4): 721-729.
5.	于宁波, 刘嘉男, 高丽, 等. 基于深度学习的膝关节MR影像自动分割方法. 仪器仪表学报, 2020, 41(6): 140-149.
6.	Aslan M F, Unlersen M F, Sabanci K, et al. CNN-based transfer learning–BiLSTM network: A novel approach for COVID-19 infection detection. Appl Soft Comput, 2021, 98: 106912.
7.	Torralba A, Efros A A. Unbiased look at dataset bias// CVPR 2011. Colorado Springs: IEEE, 2011: 1521-1528.
8.	Guan H, Liu M. Domain adaptation for medical image analysis: a survey. IEEE Trans Biomed Eng, 2021, 69(3): 1173-1185.
9.	Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: A review. Med Image Anal, 2019, 58: 101552.
10.	Patgiri R, Biswas A, Roy P. Health informatics: A computational perspective in healthcare. Berlin: Springer, 2021: 77-96.
11.	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks. Commun ACM, 2020, 63(11): 139-144.
12.	Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2223-2232.
13.	Wollmann T, Eijkman C S, Rohr K. Adversarial domain adaptation to improve automatic breast cancer grading in lymph nodes// 2018 IEEE 15th International Symposium on Biomedical Imaging. Washington: IEEE, 2018: 582-585.
14.	Hiasa Y, Otake Y, Takao M, et al. Cross-modality image synthesis from unpaired data using CycleGAN// Gooya A, Goksel O, Oguz I, et al. International workshop on simulation and synthesis in medical imaging. Cham: Springer, 2018: 31-41.
15.	Liu F. SUSAN: segment unannotated image structure using adversarial network. Magn Reson Med, 2019, 81(5): 3330-3345.
16.	Javanmardi M, Tasdizen T. Domain adaptation for biomedical image segmentation using adversarial training// 2018 IEEE 15th International Symposium on Biomedical Imaging. Washington: IEEE, 2018: 554-558.
17.	Panfilov E, Tiulpin A, Klein S, et al. Improving robustness of deep learning based knee MRI segmentation: Mixup and adversarial domain adaptation// Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. Seoul: IEEE/CVF, 2019: 450-459.
18.	Paszke A, Gross S, Massa F, et al. Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst, 2019, 32: 1-12.
19.	Razmjoo A, Caliva F, Lee J, et al. T2 analysis of the entire osteoarthritis initiative dataset. J Orthop Res, 2021, 39(1): 74-85.
20.	Hayashi D, Roemer F W, Guermazi A. Magnetic resonance imaging assessment of knee osteoarthritis: current and developing new concepts and techniques. Clin Exp Rheumatol, 2019, 37(Suppl 120): 88-95.
21.	Gao L, Ding K, Liu J, et al. Design and Development of A Knee Surgery Planning System// 2020 Chinese Control And Decision Conference (CCDC). Hefei: IEEE, 2020: 2053-2059.
22.	Ambellan F, Tack A, Ehlke M, et al. Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative. Med Image Anal, 2019, 52: 109-118.
23.	Yushkevich P A, Gerig G. ITK-SNAP: an intractive medical image segmentation tool to meet the need for expert-guided segmentation of complex medical images. IEEE Pulse, 2017, 8(4): 54-57.
24.	Cherry J M, Adler C, Ball C, et al. SGD: Saccharomyces genome database. Nucleic Acids Res, 1998, 26(1): 73-79.
25.	Miyato T, Kataoka T, Koyama M, et al. Spectral normalization for generative adversarial networks. arXiv preprint arXiv: 2018: 1802.05957.
26.	Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv: 2014: 1412.6980.
27.	Ma J, Chen J, Ng M, et al. Loss odyssey in medical image segmentation. Med Image Anal, 2021, 71: 102035.

1. Chowdhary C L, Acharjya D P. Segmentation and feature extraction in medical imaging: a systematic review. Procedia Comput Sci, 2020, 167: 26-36.
2. Hesamian M H, Jia W, He X, et al. Deep learning techniques for medical image segmentation: achievements and challenges. J Digit Imaging, 2019, 32(4): 582-596.
3. 吴玉超, 林岚, 王婧璇, 等. 基于卷积神经网络的语义分割在医学图像中的应用. 生物医学工程学杂志, 2020, 37(3): 533-540.
4. 王玉丽, 赵子健. 基于深度学习的脑图像分割算法研究综述. 生物医学工程学杂志, 2020, 37(4): 721-729.
5. 于宁波, 刘嘉男, 高丽, 等. 基于深度学习的膝关节MR影像自动分割方法. 仪器仪表学报, 2020, 41(6): 140-149.
6. Aslan M F, Unlersen M F, Sabanci K, et al. CNN-based transfer learning–BiLSTM network: A novel approach for COVID-19 infection detection. Appl Soft Comput, 2021, 98: 106912.
7. Torralba A, Efros A A. Unbiased look at dataset bias// CVPR 2011. Colorado Springs: IEEE, 2011: 1521-1528.
8. Guan H, Liu M. Domain adaptation for medical image analysis: a survey. IEEE Trans Biomed Eng, 2021, 69(3): 1173-1185.
9. Yi X, Walia E, Babyn P. Generative adversarial network in medical imaging: A review. Med Image Anal, 2019, 58: 101552.
10. Patgiri R, Biswas A, Roy P. Health informatics: A computational perspective in healthcare. Berlin: Springer, 2021: 77-96.
11. Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks. Commun ACM, 2020, 63(11): 139-144.
12. Zhu J Y, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks// Proceedings of the IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2223-2232.
13. Wollmann T, Eijkman C S, Rohr K. Adversarial domain adaptation to improve automatic breast cancer grading in lymph nodes// 2018 IEEE 15th International Symposium on Biomedical Imaging. Washington: IEEE, 2018: 582-585.
14. Hiasa Y, Otake Y, Takao M, et al. Cross-modality image synthesis from unpaired data using CycleGAN// Gooya A, Goksel O, Oguz I, et al. International workshop on simulation and synthesis in medical imaging. Cham: Springer, 2018: 31-41.
15. Liu F. SUSAN: segment unannotated image structure using adversarial network. Magn Reson Med, 2019, 81(5): 3330-3345.
16. Javanmardi M, Tasdizen T. Domain adaptation for biomedical image segmentation using adversarial training// 2018 IEEE 15th International Symposium on Biomedical Imaging. Washington: IEEE, 2018: 554-558.
17. Panfilov E, Tiulpin A, Klein S, et al. Improving robustness of deep learning based knee MRI segmentation: Mixup and adversarial domain adaptation// Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops. Seoul: IEEE/CVF, 2019: 450-459.
18. Paszke A, Gross S, Massa F, et al. Pytorch: An imperative style, high-performance deep learning library. Adv Neural Inf Process Syst, 2019, 32: 1-12.
19. Razmjoo A, Caliva F, Lee J, et al. T2 analysis of the entire osteoarthritis initiative dataset. J Orthop Res, 2021, 39(1): 74-85.
20. Hayashi D, Roemer F W, Guermazi A. Magnetic resonance imaging assessment of knee osteoarthritis: current and developing new concepts and techniques. Clin Exp Rheumatol, 2019, 37(Suppl 120): 88-95.
21. Gao L, Ding K, Liu J, et al. Design and Development of A Knee Surgery Planning System// 2020 Chinese Control And Decision Conference (CCDC). Hefei: IEEE, 2020: 2053-2059.
22. Ambellan F, Tack A, Ehlke M, et al. Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: Data from the Osteoarthritis Initiative. Med Image Anal, 2019, 52: 109-118.
23. Yushkevich P A, Gerig G. ITK-SNAP: an intractive medical image segmentation tool to meet the need for expert-guided segmentation of complex medical images. IEEE Pulse, 2017, 8(4): 54-57.
24. Cherry J M, Adler C, Ball C, et al. SGD: Saccharomyces genome database. Nucleic Acids Res, 1998, 26(1): 73-79.
25. Miyato T, Kataoka T, Koyama M, et al. Spectral normalization for generative adversarial networks. arXiv preprint arXiv: 2018: 1802.05957.
26. Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv preprint arXiv: 2014: 1412.6980.
27. Ma J, Chen J, Ng M, et al. Loss odyssey in medical image segmentation. Med Image Anal, 2021, 71: 102035.

Journal of Biomedical Engineering

A generative adversarial network-based unsupervised domain adaptation method for magnetic resonance image segmentation

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content