The dual-stream feature pyramid network based on Mamba and convolution for brain magnetic resonance image registration_Journal of Biomedical Engineering

Authors：

FU Linjie ^1,2 , ZHU Yaoyao ^1,2 ,  YAO Yu ^1,2

1. Chengdu Institute of Computer Applications, Chinese Academy of Sciences, Chengdu 610213, P. R. China;
2. University of Chinese Academy of Sciences, Beijing 100049, P. R. China;

Corresponding author：

YAO Yu, Email: casitmed2022@163.com

Keywords：

Deformable image registration; Mamba; Feature pyramid; Multi-scale fusion

DOI：

10.7507/1001-5515.202405026

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Deformable image registration plays a crucial role in medical image analysis. Despite various advanced registration models having been proposed, achieving accurate and efficient deformable registration remains challenging. Leveraging the recent outstanding performance of Mamba in computer vision, we introduced a novel model called MCRDP-Net. MCRDP-Net adapted a dual-stream network architecture that combined Mamba blocks and convolutional blocks to simultaneously extract global and local information from fixed and moving images. In the decoding stage, we employed a pyramid network structure to obtain high-resolution deformation fields, achieving efficient and precise registration. The effectiveness of MCRDP-Net was validated on public brain registration datasets, OASIS and IXI. Experimental results demonstrated significant advantages of MCRDP-Net in medical image registration, with DSC, HD95, and ASD reaching 0.815, 8.123, and 0.521 on the OASIS dataset and 0.773, 7.786, and 0.871 on the IXI dataset. In summary, MCRDP-Net demonstrates superior performance in deformable image registration, proving its potential in medical image analysis. It effectively enhances the accuracy and efficiency of registration, providing strong support for subsequent medical research and applications.

Citation： FU Linjie, ZHU Yaoyao, YAO Yu. The dual-stream feature pyramid network based on Mamba and convolution for brain magnetic resonance image registration. Journal of Biomedical Engineering, 2024, 41(6): 1177-1184. doi: 10.7507/1001-5515.202405026 Copy

1.	Sotiras A, Davatzikos C, Paragios N. Deformable medical image registration: A survey. IEEE Trans Med Imaging, 2013, 32(7): 1153-1190.
2.	Ferrante E, Paragios N. Slice-to-volume medical image registration: A survey. Med Image Anal, 2017, 39: 101-123.
3.	Fu Y, Lei Y, Wang T, et al. Deep learning in medical image registration: a review. Phys Med Biol, 2020, 65(20): 20TR01.
4.	Jaderberg M, Simonyan K, Zisserman A, et al. Spatial Transformer Networks// Cortes C, Lee D D, Sugiyama M, et al. NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2015, 2: 2017-2025.
5.	Ungi T, Lasso A, Fichtinger G. Open-source platforms for navigated image-guided interventions. Med Image Anal, 2016, 33: 181-186.
6.	Yang Q, Atkinson D, Fu Y, et al. Cross-modality image registration using a training-time privileged third modality. IEEE Trans Med Imaging, 2022, 41(11): 3421-3431.
7.	Song X, Chao H, Xu X, et al. Cross-modal attention for multi-modal image registration. Med Image Anal, 2022, 82: 102612.
8.	Rao Y, Zhou Y, Wang Y. Salient deformable network for abdominal multiorgan registration. Med Phys, 2022, 49(9): 5953-5963.
9.	Avants B B, Epstein C L, Grossman M, et al. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal, 2008, 12(1): 26-41.
10.	Balakrishnan G, Zhao A, Sabuncu M R, et al. VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging, 2019, 38(8): 1788-1800.
11.	Arsigny V, Commowick O, Pennec X, et al. A log-euclidean framework for statistics on diffeomorphisms// Larsen R, Nielsen M, Sporring J M. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2006. MICCAI 2006. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, 2006, 4190: 924-931.
12.	Chen J, Frey E C, He Y, et al. Transmorph: transformer for unsupervised medical image registration. Med Image Anal, 2022, 82: 102615.
13.	Dey N, Schlemper J, Salehi S S M, et al. ContraReg: contrastive learning of multi-modality unsupervised deformable image registration// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 66-77.
14.	Kim B, Kim D H, Park S H, et al. CycleMorph: cycle consistent unsupervised deformable image registration. Med Image Anal, 2021, 71: 102036.
15.	Kang M, Hu X, Huang W, et al. Dual-stream pyramid registration network. Med Image Anal, 2022, 78: 102379.
16.	Liu Y, Zuo L, Han S, et al. Coordinate translator for learning deformable medical image registration// International Workshop on Multiscale Multimodal Medical Imaging. Cham: Springer Nature Switzerland, 2022: 98-109.
17.	Chen J, He Y, Frey E C, et al. Vit-V-Net: vision transformer for unsupervised volumetric medical image registration. arXiv preprint arXiv, 2021: 2104.06468.
18.	Zhu Y, Lu S. Swin-VoxelMorph: a symmetric unsupervised learning model for deformable medical image registration using swin transformer// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 78-87.
19.	Gu A. Modeling sequences with structured state spaces. Palo Alto: Stanford University, 2023.
20.	Gu A, Dao T. Mamba: linear-time sequence modeling with selective state spaces. arXiv preprint arXiv, 2023: 2312.00752.
21.	Gu A, Goel K, Ré C. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv, 2021: 2111.00396,.
22.	Liu Y, Tian Y, Zhao Y, et al. VMamba: visual state space model. arXiv preprint arXiv, 2024: 2401.10166.
23.	Ma J, Li F, Wang B. U-Mamba: enhancing long-range dependency for biomedical image segmentation. arXiv preprint arXiv, 2024: 2401.04722.
24.	Xing Z, Ye T, Yang Y, et al. SegMamba: long-range sequential modeling mamba for 3D medical image segmentation. arXiv preprint arXiv, 2024: 2401.13560.
25.	Wang Z, Zheng J Q, Zhang Y, et al. Mamba-UNet: UNet-like pure Visual Mamba for medical image segmentation. arXiv preprint arXiv, 2024: 2402.05079.
26.	Wang Z, Ma C. Semi-Mamba-UNet: Pixel-level contrastive cross-supervised Visual Mamba-based UNet for semi-supervised medical image segmentation. arXiv preprint arXiv, 2024: 2402.07245.
27.	Zhu L, Liao B, Zhang Q, et al. Vision Mamba: efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv, 2024: 2401.09417.
28.	Wang Z, Ma C. Weak-Mamba-UNet: Visual Mamba makes CNN and Vit work better for scribble-based medical image segmentation. arXiv preprint arXiv, 2024: 2402.10887.
29.	Guo T, Wang Y, Meng C. MambaMorph: a Mamba-based backbone with contrastive feature learning for deformable MR-CT registration. arXiv preprint arXiv, 2024: 2401.13934.
30.	Wang Z, Zheng J Q, Ma C, et al. VMambaMorph: a Visual Mamba-based framework with cross-scan module for deformable 3D image registration. arXiv preprint arXiv, 2024: 2404.05105.
31.	Zeiler M D, Fergus R. Visualizing and understanding convolutional networks// Fleet D, Pajdla T, Schiele B, et al. Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science. Cham: Springer, 2014, 8689: 818-833.
32.	Rao Y R, Prathapani N, Nagabhooshanam E. Application of normalized cross correlation to image registration. Intl J Res Eng Technol, 2014, 3(5): 12-16.
33.	Marcus D S, Wang T H, Parker J, et al. Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J Cogn Neurosci, 2007, 19(9): 1498-1507.

1. Sotiras A, Davatzikos C, Paragios N. Deformable medical image registration: A survey. IEEE Trans Med Imaging, 2013, 32(7): 1153-1190.
2. Ferrante E, Paragios N. Slice-to-volume medical image registration: A survey. Med Image Anal, 2017, 39: 101-123.
3. Fu Y, Lei Y, Wang T, et al. Deep learning in medical image registration: a review. Phys Med Biol, 2020, 65(20): 20TR01.
4. Jaderberg M, Simonyan K, Zisserman A, et al. Spatial Transformer Networks// Cortes C, Lee D D, Sugiyama M, et al. NIPS'15: Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 2015, 2: 2017-2025.
5. Ungi T, Lasso A, Fichtinger G. Open-source platforms for navigated image-guided interventions. Med Image Anal, 2016, 33: 181-186.
6. Yang Q, Atkinson D, Fu Y, et al. Cross-modality image registration using a training-time privileged third modality. IEEE Trans Med Imaging, 2022, 41(11): 3421-3431.
7. Song X, Chao H, Xu X, et al. Cross-modal attention for multi-modal image registration. Med Image Anal, 2022, 82: 102612.
8. Rao Y, Zhou Y, Wang Y. Salient deformable network for abdominal multiorgan registration. Med Phys, 2022, 49(9): 5953-5963.
9. Avants B B, Epstein C L, Grossman M, et al. Symmetric diffeomorphic image registration with cross-correlation: evaluating automated labeling of elderly and neurodegenerative brain. Med Image Anal, 2008, 12(1): 26-41.
10. Balakrishnan G, Zhao A, Sabuncu M R, et al. VoxelMorph: a learning framework for deformable medical image registration. IEEE Trans Med Imaging, 2019, 38(8): 1788-1800.
11. Arsigny V, Commowick O, Pennec X, et al. A log-euclidean framework for statistics on diffeomorphisms// Larsen R, Nielsen M, Sporring J M. Medical Image Computing and Computer-Assisted Intervention – MICCAI 2006. MICCAI 2006. Lecture Notes in Computer Science. Berlin, Heidelberg: Springer, 2006, 4190: 924-931.
12. Chen J, Frey E C, He Y, et al. Transmorph: transformer for unsupervised medical image registration. Med Image Anal, 2022, 82: 102615.
13. Dey N, Schlemper J, Salehi S S M, et al. ContraReg: contrastive learning of multi-modality unsupervised deformable image registration// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 66-77.
14. Kim B, Kim D H, Park S H, et al. CycleMorph: cycle consistent unsupervised deformable image registration. Med Image Anal, 2021, 71: 102036.
15. Kang M, Hu X, Huang W, et al. Dual-stream pyramid registration network. Med Image Anal, 2022, 78: 102379.
16. Liu Y, Zuo L, Han S, et al. Coordinate translator for learning deformable medical image registration// International Workshop on Multiscale Multimodal Medical Imaging. Cham: Springer Nature Switzerland, 2022: 98-109.
17. Chen J, He Y, Frey E C, et al. Vit-V-Net: vision transformer for unsupervised volumetric medical image registration. arXiv preprint arXiv, 2021: 2104.06468.
18. Zhu Y, Lu S. Swin-VoxelMorph: a symmetric unsupervised learning model for deformable medical image registration using swin transformer// International Conference on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer Nature Switzerland, 2022: 78-87.
19. Gu A. Modeling sequences with structured state spaces. Palo Alto: Stanford University, 2023.
20. Gu A, Dao T. Mamba: linear-time sequence modeling with selective state spaces. arXiv preprint arXiv, 2023: 2312.00752.
21. Gu A, Goel K, Ré C. Efficiently modeling long sequences with structured state spaces. arXiv preprint arXiv, 2021: 2111.00396,.
22. Liu Y, Tian Y, Zhao Y, et al. VMamba: visual state space model. arXiv preprint arXiv, 2024: 2401.10166.
23. Ma J, Li F, Wang B. U-Mamba: enhancing long-range dependency for biomedical image segmentation. arXiv preprint arXiv, 2024: 2401.04722.
24. Xing Z, Ye T, Yang Y, et al. SegMamba: long-range sequential modeling mamba for 3D medical image segmentation. arXiv preprint arXiv, 2024: 2401.13560.
25. Wang Z, Zheng J Q, Zhang Y, et al. Mamba-UNet: UNet-like pure Visual Mamba for medical image segmentation. arXiv preprint arXiv, 2024: 2402.05079.
26. Wang Z, Ma C. Semi-Mamba-UNet: Pixel-level contrastive cross-supervised Visual Mamba-based UNet for semi-supervised medical image segmentation. arXiv preprint arXiv, 2024: 2402.07245.
27. Zhu L, Liao B, Zhang Q, et al. Vision Mamba: efficient visual representation learning with bidirectional state space model. arXiv preprint arXiv, 2024: 2401.09417.
28. Wang Z, Ma C. Weak-Mamba-UNet: Visual Mamba makes CNN and Vit work better for scribble-based medical image segmentation. arXiv preprint arXiv, 2024: 2402.10887.
29. Guo T, Wang Y, Meng C. MambaMorph: a Mamba-based backbone with contrastive feature learning for deformable MR-CT registration. arXiv preprint arXiv, 2024: 2401.13934.
30. Wang Z, Zheng J Q, Ma C, et al. VMambaMorph: a Visual Mamba-based framework with cross-scan module for deformable 3D image registration. arXiv preprint arXiv, 2024: 2404.05105.
31. Zeiler M D, Fergus R. Visualizing and understanding convolutional networks// Fleet D, Pajdla T, Schiele B, et al. Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science. Cham: Springer, 2014, 8689: 818-833.
32. Rao Y R, Prathapani N, Nagabhooshanam E. Application of normalized cross correlation to image registration. Intl J Res Eng Technol, 2014, 3(5): 12-16.
33. Marcus D S, Wang T H, Parker J, et al. Open Access Series of Imaging Studies (OASIS): cross-sectional MRI data in young, middle aged, nondemented, and demented older adults. J Cogn Neurosci, 2007, 19(9): 1498-1507.

Journal of Biomedical Engineering

The dual-stream feature pyramid network based on Mamba and convolution for brain magnetic resonance image registration

Abstract Full text Figures/Tables Video References Cited by

Previous Article

Next Article

Format

Content