Cross modal medical image online hash retrieval based on online semantic similarity_Journal of Biomedical Engineering

Authors：

LIU Qinghai ¹ , TANG Lun ¹ , WU Qianlin ¹ ,  XU Liming ^2,3 , CHEN Qianbin ¹

1. School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications, Chongqing 400065, P. R. China;
2. School of Computer Science, China West Normal University, Nanchong, Sichuan 637009, P. R. China;
3. College of Computer Science, Sichuan University, Chengdu 610065, P. R. China;

Corresponding author：

XU Liming, Email: xulm@cwnu.edu.cn

Keywords：

Online hashing; Cross modal retrieval; Medical image; Discrete optimization

DOI：

10.7507/1001-5515.202409022

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

Online hashing methods are receiving increasing attention in cross modal medical image retrieval research. However, existing online methods often lack the learning ability to maintain semantic correlation between new and existing data. To this end, we proposed online semantic similarity cross-modal hashing (OSCMH) learning framework to incrementally learn compact binary hash codes of medical stream data. Within it, a sparse representation of existing data based on online anchor datasets was designed to avoid semantic forgetting of the data and adaptively update hash codes, which effectively maintained semantic correlation between existing and arriving data and reduced information loss as well as improved training efficiency. Besides, an online discrete optimization method was proposed to solve the binary optimization problem of hash code by incrementally updating hash function and optimizing hash code on medical stream data. Compared with existing online or offline hashing methods, the proposed algorithm achieved average retrieval accuracy improvements of 12.5% and 14.3% on two datasets, respectively, effectively enhancing the retrieval efficiency in the field of medical images.

1.	Ahmad J, Muhammad K, Baik S W. Medical image retrieval with compact binary codes generated in frequency domain using highly reactive convolutional features. J Med Syst, 2018, 42: 1-19.
2.	Fang J, Fu H, Liu J. Deep triplet hashing network for case-based medical image retrieval. Med Image Anal, 2021, 69: 101981.
3.	Li X, Hu D, Nie F. Deep binary reconstruction for cross-modal hashing// Proceedings of the 25th ACM International Conference on Multimedia. California: ACM, 2017: 1398-1406.
4.	Wang Y, Luo X, Xu X S. Label embedding online hashing for cross-modal retrieval// Proceedings of the 28th ACM International Conference on Multimedia. Seattle: ACM, 2020: 871-879.
5.	Lin M, Ji R, Sun X, et al. Fast class-wise updating for online hashing. IEEE Trans Pattern Anal Mach Intell, 2020, 44(5): 2453-2467.
6.	Xie L, Shen J, Zhu L. Online cross-modal hashing for web image retrieval// Proceedings of the AAAI Conference on Artificial Intelligence. Phoenix: AAAI, 2016, 30(1): 294-300.
7.	Wang D, Wang Q, An Y, et al. Online collective matrix factorization hashing for large-scale cross-media retrieval// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Xi’an: ACM, 2020: 1409-1418.
8.	Leng C, Wu J, Cheng J, et al. Online sketching hashing// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 2503-2511.
9.	Chen X, King I, Lyu M R. FROSH: FasteR online sketching hashing// Proceedings of the Uncertainty in Artificial Intelligence. Sydney: UAI, 2017: 89-101.
10.	Zhou J, Ding G, Guo Y. Latent semantic sparse hashing for cross-modal similarity search// Proceedings of the 37th international ACM SIGIR Conference on Research & Development in Information Retrieval. Queensland: ACM, 2014: 415-424.
11.	Chen J, Li Y, Lu H. Online self-organizing hashing// 2016 IEEE International Conference on Multimedia and Expo (ICME). Seattle: IEEE, 2016: 1-6.
12.	Li X, Wu W, Yuan Y H, et al. Online unsupervised cross-view discrete hashing for large-scale retrieval. Appl Intell, 2022, 52(13): 14905-14917.
13.	Liu X, Yi J, Cheung Y, et al. OMGH: Online manifold-guided hashing for flexible cross-modal retrieval. IEEE Trans Multimedia, 2022, 25: 3811-3824.
14.	Yao T, Wang G, Yan L, et al. Online latent semantic hashing for cross-media retrieval. Pattern Recogn, 2019, 89: 1-11.
15.	Su R, Wang D, Huang Z, et al. Online adaptive supervised hashing for large-scale cross-modal retrieval. IEEE Access, 2020, 8: 206360-206370.
16.	Lin M, Ji R, Liu H, et al. Supervised online hashing via hadamard codebook learning// Proceedings of the 26th ACM International Conference on Multimedia. Seoul: ACM, 2018: 1635-1643.
17.	Weng Z, Zhu Y. Online hashing with bit selection for image retrieval. IEEE Trans Multimedia, 2020, 23: 1868-1881.
18.	Wang Y, Chen Z D, Luo X, et al. A high-dimensional sparse hashing framework for cross-modal retrieval. IEEE Trans Circuits Syst Video Technol, 2022, 32(12): 8822-8836.
19.	Tan J, Yang X, Yang Z, et al. Extensible max-min collaborative retention for online mini-batch learning hash retrieval. IEEE Trans Multimedia, 2024, 26: 6743-6758.
20.	Lu X, Zhu L, Cheng Z, et al. Online multi-modal hashing with dynamic query-adaption// Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Paris: ACM, 2019: 715-724.
21.	Li L, Shu Z, Yu Z, et al. Robust online hashing with label semantic enhancement for cross-modal retrieval. Pattern Recogn, 2024, 145: 109972.
22.	Ding G, Guo Y, Zhou J. Collective matrix factorization hashing for multimodal data// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 2075-2082.
23.	Jiang Q Y, Li W J. Deep cross-modal hashing// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 3232-3240.
24.	Jiang Q Y, Li W J. Discrete latent factor model for cross-modal hashing. IEEE Trans Image Process, 2019, 28(7): 3490-3501.
25.	Wang Y, Chen Z D, Luo X, et al. Fast cross-modal hashing with global and local similarity embedding. IEEE Trans Cybern, 2021, 52(10): 10064-10077.
26.	Li H, Zhang C, Jia X, et al. Adaptive label correlation based asymmetric discrete hashing for cross-modal retrieval. IEEE Trans Knowl Data Eng, 2021, 35(2): 1185-1199.

1. Ahmad J, Muhammad K, Baik S W. Medical image retrieval with compact binary codes generated in frequency domain using highly reactive convolutional features. J Med Syst, 2018, 42: 1-19.
2. Fang J, Fu H, Liu J. Deep triplet hashing network for case-based medical image retrieval. Med Image Anal, 2021, 69: 101981.
3. Li X, Hu D, Nie F. Deep binary reconstruction for cross-modal hashing// Proceedings of the 25th ACM International Conference on Multimedia. California: ACM, 2017: 1398-1406.
4. Wang Y, Luo X, Xu X S. Label embedding online hashing for cross-modal retrieval// Proceedings of the 28th ACM International Conference on Multimedia. Seattle: ACM, 2020: 871-879.
5. Lin M, Ji R, Sun X, et al. Fast class-wise updating for online hashing. IEEE Trans Pattern Anal Mach Intell, 2020, 44(5): 2453-2467.
6. Xie L, Shen J, Zhu L. Online cross-modal hashing for web image retrieval// Proceedings of the AAAI Conference on Artificial Intelligence. Phoenix: AAAI, 2016, 30(1): 294-300.
7. Wang D, Wang Q, An Y, et al. Online collective matrix factorization hashing for large-scale cross-media retrieval// Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Xi’an: ACM, 2020: 1409-1418.
8. Leng C, Wu J, Cheng J, et al. Online sketching hashing// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 2503-2511.
9. Chen X, King I, Lyu M R. FROSH: FasteR online sketching hashing// Proceedings of the Uncertainty in Artificial Intelligence. Sydney: UAI, 2017: 89-101.
10. Zhou J, Ding G, Guo Y. Latent semantic sparse hashing for cross-modal similarity search// Proceedings of the 37th international ACM SIGIR Conference on Research & Development in Information Retrieval. Queensland: ACM, 2014: 415-424.
11. Chen J, Li Y, Lu H. Online self-organizing hashing// 2016 IEEE International Conference on Multimedia and Expo (ICME). Seattle: IEEE, 2016: 1-6.
12. Li X, Wu W, Yuan Y H, et al. Online unsupervised cross-view discrete hashing for large-scale retrieval. Appl Intell, 2022, 52(13): 14905-14917.
13. Liu X, Yi J, Cheung Y, et al. OMGH: Online manifold-guided hashing for flexible cross-modal retrieval. IEEE Trans Multimedia, 2022, 25: 3811-3824.
14. Yao T, Wang G, Yan L, et al. Online latent semantic hashing for cross-media retrieval. Pattern Recogn, 2019, 89: 1-11.
15. Su R, Wang D, Huang Z, et al. Online adaptive supervised hashing for large-scale cross-modal retrieval. IEEE Access, 2020, 8: 206360-206370.
16. Lin M, Ji R, Liu H, et al. Supervised online hashing via hadamard codebook learning// Proceedings of the 26th ACM International Conference on Multimedia. Seoul: ACM, 2018: 1635-1643.
17. Weng Z, Zhu Y. Online hashing with bit selection for image retrieval. IEEE Trans Multimedia, 2020, 23: 1868-1881.
18. Wang Y, Chen Z D, Luo X, et al. A high-dimensional sparse hashing framework for cross-modal retrieval. IEEE Trans Circuits Syst Video Technol, 2022, 32(12): 8822-8836.
19. Tan J, Yang X, Yang Z, et al. Extensible max-min collaborative retention for online mini-batch learning hash retrieval. IEEE Trans Multimedia, 2024, 26: 6743-6758.
20. Lu X, Zhu L, Cheng Z, et al. Online multi-modal hashing with dynamic query-adaption// Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Paris: ACM, 2019: 715-724.
21. Li L, Shu Z, Yu Z, et al. Robust online hashing with label semantic enhancement for cross-modal retrieval. Pattern Recogn, 2024, 145: 109972.
22. Ding G, Guo Y, Zhou J. Collective matrix factorization hashing for multimodal data// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 2075-2082.
23. Jiang Q Y, Li W J. Deep cross-modal hashing// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 3232-3240.
24. Jiang Q Y, Li W J. Discrete latent factor model for cross-modal hashing. IEEE Trans Image Process, 2019, 28(7): 3490-3501.
25. Wang Y, Chen Z D, Luo X, et al. Fast cross-modal hashing with global and local similarity embedding. IEEE Trans Cybern, 2021, 52(10): 10064-10077.
26. Li H, Zhang C, Jia X, et al. Adaptive label correlation based asymmetric discrete hashing for cross-modal retrieval. IEEE Trans Knowl Data Eng, 2021, 35(2): 1185-1199.

Journal of Biomedical Engineering

Latest ArticlesCross modal medical image online hash retrieval based on online semantic similarity

Abstract Full text Figures/Tables Video References Cited by

Format

Content