Medical image super-resolution reconstruction via multi-scale information distillation network under multi-scale geometric transform domain_Journal of Biomedical Engineering

Authors：

 WANG Huadong ¹ , SUN Ting ^1,2

1. School of Computer Science and Technology, Zhoukou Normal University, Zhoukou, Henan 466001, P. R. China;
2. Institute of Visualization Technology, Northwest University, Xi’an 710049, P. R. China;

Corresponding author：

WANG Huadong, Email: hdwang_zknu@163.com

Keywords：

Medical image; Super-resolution reconstruction; Convolutional neural networks; Multi-scale information distillation; Non-subsampled shearlet transform

DOI：

10.7507/1001-5515.202109057

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

High resolution (HR) magnetic resonance images (MRI) or computed tomography (CT) images can provide clearer anatomical details of human body, which facilitates early diagnosis of the diseases. However, due to the imaging system, imaging environment and human factors, it is difficult to obtain clear high-resolution images. In this paper, we proposed a novel medical image super resolution (SR) reconstruction method via multi-scale information distillation (MSID) network in the non-subsampled shearlet transform (NSST) domain, namely NSST-MSID network. We first proposed a MSID network that mainly consisted of a series of stacked MSID blocks to fully exploit features from images and effectively restore the low resolution (LR) images to HR images. In addition, most previous methods predict the HR images in the spatial domain, producing over-smoothed outputs while losing texture details. Thus, we viewed the medical image SR task as the prediction of NSST coefficients, which make further MSID network keep richer structure details than that in spatial domain. Finally, the experimental results on our constructed medical image datasets demonstrated that the proposed method was capable of obtaining better peak signal to noise ratio (PSNR), structural similarity (SSIM) and root mean square error (RMSE) values and keeping global topological structure and local texture detail better than other outstanding methods, which achieves good medical image reconstruction effect.

Citation： WANG Huadong, SUN Ting. Medical image super-resolution reconstruction via multi-scale information distillation network under multi-scale geometric transform domain. Journal of Biomedical Engineering, 2022, 39(5): 887-896. doi: 10.7507/1001-5515.202109057 Copy

1.	Chaudhari A S, Fang Z, Kogan F, et al. Super-resolution musculoskeletal MRI using deep learning. Magn Reson Med Magnetic Resonance in Medicine, 2018, 80(5): 2139-2154.
2.	Umehara K, Ota J, Ishida T. Application of super-resolution convolutional neural network for enhancing image resolution in chest CT. J Digit Imaging, 2018, 31(4): 441-450.
3.	Zhu J, Yang G, Lio P. How can we make GAN perform better in single medical image super-resolution? A lesion focused multi-scale approach// Proceedings of The 16th IEEE International Symposium on Biomedical Imaging (ISBI). Venezia: IEEE, 2019: 1-5.
4.	Cheng B, Xiao R, Wang J, et al. High frequency residual learning for multi-scale image classification// Proceedings of British Machine Vision Conference (BMVC). Cardiff: British Machine Vision Society, 2019: 1-14.
5.	Borji A, Cheng M M, Hou Q, et al. Salient object detection: A survey. Computational Visual Media, 2019, 5(2): 117-150.
6.	张宁, 王永成, 张欣, 等. 基于深度学习的单张图片超分辨率重构研究进展. 自动化学报, 2020, 46(12): 2479-2499.
7.	Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution// Proceedings of European Conference on Computer Vision (ECCV). Zurich: IEEE, 2014: 184-199.
8.	Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 1646-1654.
9.	Tai Y, Yang J, Liu X M. Image super-resolution via deep recursive residual network// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE, 2017: 3147-3155.
10.	Tai Y, Yang J, Liu X M, et al. MemNet: a persistent memory network for image restoration// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE, 2017: 4539-4547.
11.	Zhang Y L, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks// Proceedings of European Conference on Computer Vision (ECCV). Munich: IEEE, 2018: 294-310.
12.	Tong T, Li G, Liu X, et al. Image super-resolution using dense skip connections// Proceedings of IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 4809-4817.
13.	Zhang Y L, Tian Y P, Kong Y, et al. Residual dense network for image super-resolution// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 2472-2481.
14.	Hui Z, Wang X M, Gao X B. Fast and accurate single image super-resolution via information distillation network// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 723-731.
15.	Li J C, Fang F M, Mei K F, et al. Multi-scale residual network for image super-resolution// Proceedings of European Conference on Computer Vision (ECCV). Munich: IEEE, 2018: 527-542.
16.	Li Z, Yang J L, Liu Z, et al. Feedback network for image super-resolution// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). California: IEEE, 2019: 3867-3876.
17.	沈明玉, 俞鹏飞, 汪荣贵, 等. 多阶段融合网络的图像超分辨率重建. 中国图象图形学报, 2019, 24(8): 1258-1269.
18.	Shu Z, Cheng M C, Yang B, et al. Residual magnifier: a dense information flow network for super resolution// Proceedings of IEEE International Conference on Multimedia and Expo (ICME). Shanghai: IEEE, 2019: 646-651.
19.	Chen Y, Li J, Xiao H, et al. Dual path networks// Proceedings of Conference and Workshop on Neural Information Processing Systems (NeurIPS). California: IEEE, 2017: 4467-4475.
20.	Gao S H, Cheng M M, Zhao K, et al. Res2Net: A new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell, 2021, 43(2): 652-662.
21.	应自炉, 龙祥. 多尺度密集残差网络的单幅图像超分辨率重建. 中国图象图形学报, 2019, 24(3): 0410-0419.
22.	Wang C P, Wang S M, Ma B, et al. Transform domain based medical image super-resolution via deep multi-scale network// Proceedings of International Conference on Acoustics, Speech and Signal Processing (ICASSP). Brighton: IEEE, 2019: 2387-2391.
23.	Guo T T, Mousavi H S, Vu T H, et al. Deep wavelet prediction for image super-resolution// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Hawaii: IEEE, 2017: 104-113.
24.	Huang H B, He R, Sun Z N. Wavelet-SRnet: a wavelet-based CNN for multi-scale face super resolution// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE, 2017: 1689-1697.
25.	Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Hawaii: IEEE, 2017: 1-9.
26.	Glorot X, Bordes A, Bengio Y. Deep sparse rectiﬁer neural networks// Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS). Fort Lauderdale: AISTATS, 2011: 315-323.
27.	Mallat S. A wavelet tour of signal processing: the sparse way. Stanford University: Academic Press, 2008.
28.	Lim W Q. The discrete shearlet transform: a new directional transform and compactly supported shearlet frames. IEEE Trans Image Process, 2010, 19(5): 1166-1180.
29.	Hou B, Zhang X H, Bu X M, et al. SAR image despeckling based on nonsubsampled shearlet transform. IEEE J Selected Topics Appl Earth Obs Remote Sens, 2012, 5(3): 809-823.
30.	Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process, 2004, 13(4): 600-612.
31.	Seitzer M, Yang G, Schlemper J, et al. Adversarial and perceptual refinement for compressed sensing MRI reconstruction// Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Granada: Medical Image Computing and Computer-Assisted Intervention Society, 2018: 232-240.
32.	Clark K, Vendt B, Smith K, et al. The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging, 2013, 26(6): 1045-1057.

1. Chaudhari A S, Fang Z, Kogan F, et al. Super-resolution musculoskeletal MRI using deep learning. Magn Reson Med Magnetic Resonance in Medicine, 2018, 80(5): 2139-2154.
2. Umehara K, Ota J, Ishida T. Application of super-resolution convolutional neural network for enhancing image resolution in chest CT. J Digit Imaging, 2018, 31(4): 441-450.
3. Zhu J, Yang G, Lio P. How can we make GAN perform better in single medical image super-resolution? A lesion focused multi-scale approach// Proceedings of The 16th IEEE International Symposium on Biomedical Imaging (ISBI). Venezia: IEEE, 2019: 1-5.
4. Cheng B, Xiao R, Wang J, et al. High frequency residual learning for multi-scale image classification// Proceedings of British Machine Vision Conference (BMVC). Cardiff: British Machine Vision Society, 2019: 1-14.
5. Borji A, Cheng M M, Hou Q, et al. Salient object detection: A survey. Computational Visual Media, 2019, 5(2): 117-150.
6. 张宁, 王永成, 张欣, 等. 基于深度学习的单张图片超分辨率重构研究进展. 自动化学报, 2020, 46(12): 2479-2499.
7. Dong C, Loy C C, He K M, et al. Learning a deep convolutional network for image super-resolution// Proceedings of European Conference on Computer Vision (ECCV). Zurich: IEEE, 2014: 184-199.
8. Kim J, Lee J K, Lee K M. Accurate image super-resolution using very deep convolutional networks// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas: IEEE, 2016: 1646-1654.
9. Tai Y, Yang J, Liu X M. Image super-resolution via deep recursive residual network// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE, 2017: 3147-3155.
10. Tai Y, Yang J, Liu X M, et al. MemNet: a persistent memory network for image restoration// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE, 2017: 4539-4547.
11. Zhang Y L, Li K P, Li K, et al. Image super-resolution using very deep residual channel attention networks// Proceedings of European Conference on Computer Vision (ECCV). Munich: IEEE, 2018: 294-310.
12. Tong T, Li G, Liu X, et al. Image super-resolution using dense skip connections// Proceedings of IEEE International Conference on Computer Vision (ICCV). Venice: IEEE, 2017: 4809-4817.
13. Zhang Y L, Tian Y P, Kong Y, et al. Residual dense network for image super-resolution// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 2472-2481.
14. Hui Z, Wang X M, Gao X B. Fast and accurate single image super-resolution via information distillation network// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Salt Lake City: IEEE, 2018: 723-731.
15. Li J C, Fang F M, Mei K F, et al. Multi-scale residual network for image super-resolution// Proceedings of European Conference on Computer Vision (ECCV). Munich: IEEE, 2018: 527-542.
16. Li Z, Yang J L, Liu Z, et al. Feedback network for image super-resolution// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). California: IEEE, 2019: 3867-3876.
17. 沈明玉, 俞鹏飞, 汪荣贵, 等. 多阶段融合网络的图像超分辨率重建. 中国图象图形学报, 2019, 24(8): 1258-1269.
18. Shu Z, Cheng M C, Yang B, et al. Residual magnifier: a dense information flow network for super resolution// Proceedings of IEEE International Conference on Multimedia and Expo (ICME). Shanghai: IEEE, 2019: 646-651.
19. Chen Y, Li J, Xiao H, et al. Dual path networks// Proceedings of Conference and Workshop on Neural Information Processing Systems (NeurIPS). California: IEEE, 2017: 4467-4475.
20. Gao S H, Cheng M M, Zhao K, et al. Res2Net: A new multi-scale backbone architecture. IEEE Trans Pattern Anal Mach Intell, 2021, 43(2): 652-662.
21. 应自炉, 龙祥. 多尺度密集残差网络的单幅图像超分辨率重建. 中国图象图形学报, 2019, 24(3): 0410-0419.
22. Wang C P, Wang S M, Ma B, et al. Transform domain based medical image super-resolution via deep multi-scale network// Proceedings of International Conference on Acoustics, Speech and Signal Processing (ICASSP). Brighton: IEEE, 2019: 2387-2391.
23. Guo T T, Mousavi H S, Vu T H, et al. Deep wavelet prediction for image super-resolution// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Hawaii: IEEE, 2017: 104-113.
24. Huang H B, He R, Sun Z N. Wavelet-SRnet: a wavelet-based CNN for multi-scale face super resolution// Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hawaii: IEEE, 2017: 1689-1697.
25. Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Hawaii: IEEE, 2017: 1-9.
26. Glorot X, Bordes A, Bengio Y. Deep sparse rectiﬁer neural networks// Proceedings of International Conference on Artificial Intelligence and Statistics (AISTATS). Fort Lauderdale: AISTATS, 2011: 315-323.
27. Mallat S. A wavelet tour of signal processing: the sparse way. Stanford University: Academic Press, 2008.
28. Lim W Q. The discrete shearlet transform: a new directional transform and compactly supported shearlet frames. IEEE Trans Image Process, 2010, 19(5): 1166-1180.
29. Hou B, Zhang X H, Bu X M, et al. SAR image despeckling based on nonsubsampled shearlet transform. IEEE J Selected Topics Appl Earth Obs Remote Sens, 2012, 5(3): 809-823.
30. Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process, 2004, 13(4): 600-612.
31. Seitzer M, Yang G, Schlemper J, et al. Adversarial and perceptual refinement for compressed sensing MRI reconstruction// Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Granada: Medical Image Computing and Computer-Assisted Intervention Society, 2018: 232-240.
32. Clark K, Vendt B, Smith K, et al. The cancer imaging archive (TCIA): maintaining and operating a public information repository. J Digit Imaging, 2013, 26(6): 1045-1057.

Previous Article
Cascaded multi-level medical image registration method based on transformer
Next Article
Research on three-dimensional skull repair by combining residual and informer attention

Journal of Biomedical Engineering

Medical image super-resolution reconstruction via multi-scale information distillation network under multi-scale geometric transform domain

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content