Interpreting the TRIPOD-LLM guideline: a reporting standard for large language model research in healthcare_Chinese Journal of Evidence-Based Medicine

Authors：

ZHENG Qingyong ^1,2,3 , WU Jingling ⁴ , LI Molan ⁵ , XU Jianguo ^1,2,3 , LIU Ming ^1,2,3 , ZHOU Yongjia ^1,2,3 , CUI Yating ^1,2,3 , LI Ruixue ⁶ , GAO Ya ^7,8 , GE Long ^2,9,10,11 ,  ZHANG Junhua ^12,13 ,  TIAN Jinhui ^1,2,3

1. Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou 730000, P. R. China;
2. Key Laboratory of Evidence-Based Medicine of Gansu Province, Lanzhou 730000, P. R. China;
3. Research Unit of Evidence-Based Evaluation and Guidelines, Chinese Academy of Medical Sciences, Lanzhou 730000, P. R. China;
4. Department of Ultrasound, 96604 Military Hospital of China People's Liberation Army, Lanzhou 730000, P. R. China;
5. The First School of Clinical Medicine, Lanzhou University, Lanzhou 730000, P. R. China;
6. People’s Hospital of Xizang Autonomous Region, Lhasa 850000, P. R. China;
7. School of Public Health, Cheeloo College of Medicine, Shandong University, Jinan 250012, P. R. China;
8. National Institute of Health and Medical Data Science, Jinan 250003, P. R. China;
9. Department of Health Policy and Management, School of Public Health, Lanzhou 730000, P. R. China;
10. Laboratory of Cross-Innovation for Evidence-based Social Sciences, Lanzhou 730000, P. R. China;
11. Research Centre for Health Management and Health Development, Lanzhou University, Lanzhou 730000, P. R. China;
12. Evidence-Based Medicine Center, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, P. R. China;
13. Key Laboratory of Evidence-Based Evaluation of Traditional Chinese Medicine, National Medical Products Administration, Tianjin 301617, P. R. China;

Corresponding author：

ZHANG Junhua, Email: zjhtcm@163.com; TIAN Jinhui, Email: tjh996@163.com

Keywords：

Large language models; Reporting guideline; TRIPOD-LLM; Healthcare AI/Medical AI

DOI：

10.7507/1672-2531.202505023

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

The burgeoning application of large language models (LLM) in healthcare demonstrates immense potential, yet simultaneously poses new challenges to the standardization of research reporting. To enhance the transparency and reliability of medical LLM research, an international expert group published the TRIPOD-LLM reporting guideline in Nature Medicine in January 2024. As an extension of the TRIPOD+AI guideline, TRIPOD-LLM provides detailed reporting items specifically tailored to the unique characteristics of LLMs, including general foundational models (e.g., GPT-4) and domain-specific fine-tuned models (e.g., Med-PaLM 2). It addresses critical aspects such as prompt engineering, inference parameters, generative evaluation, and fairness considerations. Notably, the guideline introduces an innovative modular design and a "living guideline" mechanism. This paper provides a systematic, item-by-item interpretation and example-based analysis of the TRIPOD-LLM guideline. It is intended to serve as a clear and practical handbook for researchers in this field, as well as for journal reviewers and editors responsible for assessing the quality of such studies, thereby fostering the high-quality development of medical LLM research in China.

1.	Abd-Alrazaq A, AlSaad R, Alhuwail D, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ, 2023, 9: e48291.
2.	Li Y, Zhao J, Li M, et al. RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization. J Am Med Inform Assoc, 2024, 31(9): 2030-2039.
3.	Meng X, Yan X, Zhang K, et al. The application of large language models in medicine: a scoping review. iScience, 2024, 27(5): 109713.
4.	Guo Y, Qiu W, Leroy G, et al. Retrieval augmentation of large language models for lay language generation. J Biomed Inform, 2024, 149: 104580.
5.	Tripathi S, Sukumaran R, Cook TS. Efficient healthcare with large language models: optimizing clinical workflow and enhancing patient care. J Am Med Inform Assoc, 2024, 31(6): 1436-1440.
6.	Liao QV, Vaughan JW. AI transparency in the age of llms: a human-centered research roadmap. arXiv preprint arXiv, 2023: 230601941.
7.	Franc JM, Hertelendy AJ, Cheng L, et al. Accuracy of a commercial large language model (ChatGPT) to perform disaster triage of simulated patients using the simple triage and rapid treatment (START) protocol: gage repeatability and reproducibility study. J Med Internet Res, 2024, 26: e55648.
8.	Gallifant J, Afshar M, Ameen S, et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med, 2025, 31(1): 60-69.
9.	Cohen JF, Bossuyt PMM. TRIPOD+AI: an updated reporting guideline for clinical prediction models. BMJ, 2024, 385: q824.
10.	Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 2024, 385: e078378.
11.	Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ, 2015, 350: g7594.
12.	Poole S, Sisodia N, Koshal K, et al. Detecting new lesions using a large language model: applications in real-world multiple sclerosis datasets. Ann Neurol, 2025.

1. Abd-Alrazaq A, AlSaad R, Alhuwail D, et al. Large language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ, 2023, 9: e48291.
2. Li Y, Zhao J, Li M, et al. RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization. J Am Med Inform Assoc, 2024, 31(9): 2030-2039.
3. Meng X, Yan X, Zhang K, et al. The application of large language models in medicine: a scoping review. iScience, 2024, 27(5): 109713.
4. Guo Y, Qiu W, Leroy G, et al. Retrieval augmentation of large language models for lay language generation. J Biomed Inform, 2024, 149: 104580.
5. Tripathi S, Sukumaran R, Cook TS. Efficient healthcare with large language models: optimizing clinical workflow and enhancing patient care. J Am Med Inform Assoc, 2024, 31(6): 1436-1440.
6. Liao QV, Vaughan JW. AI transparency in the age of llms: a human-centered research roadmap. arXiv preprint arXiv, 2023: 230601941.
7. Franc JM, Hertelendy AJ, Cheng L, et al. Accuracy of a commercial large language model (ChatGPT) to perform disaster triage of simulated patients using the simple triage and rapid treatment (START) protocol: gage repeatability and reproducibility study. J Med Internet Res, 2024, 26: e55648.
8. Gallifant J, Afshar M, Ameen S, et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med, 2025, 31(1): 60-69.
9. Cohen JF, Bossuyt PMM. TRIPOD+AI: an updated reporting guideline for clinical prediction models. BMJ, 2024, 385: q824.
10. Collins GS, Moons KGM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ, 2024, 385: e078378.
11. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ, 2015, 350: g7594.
12. Poole S, Sisodia N, Koshal K, et al. Detecting new lesions using a large language model: applications in real-world multiple sclerosis datasets. Ann Neurol, 2025.

Chinese Journal of Evidence-Based Medicine

Latest ArticlesInterpreting the TRIPOD-LLM guideline: a reporting standard for large language model research in healthcare

Abstract Full text Figures/Tables Video References Cited by

Format

Content