PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods—Chinese interpretation_Chinese Journal of Clinical Thoracic and Cardiovascular Surgery

Authors：

WANG Xingmeng ¹ ,  DAI Guohua ² ,  GAO Wulin ² , GUAN Hui ² , REN Lili ³ , CHEN Chen ¹ , TAN Xiaoyang ¹ , LIN Yiming ¹

1. First Clinical Medical College, Shandong University of Traditional Chinese Medicine, Jinan, 250014, P. R. China;
2. Department of Geriatric Medicine, Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan, 250014, P. R. China;
3. Department of Critical Care Medicine, Affiliated Hospital of Shandong University of Traditional Chinese Medicine, Jinan, 250014, P. R. China;

Corresponding author：

DAI Guohua, Email: daigh2004@163.com; GAO Wulin, Email: gaowulin05@sina.com

Keywords：

PROBAST+AI; artificial intelligence; machine learning; clinical prediction model; systematic review

DOI：

10.7507/1007-4848.202507088

Video：

Export PDF Favorites Scan Get Citation

Abstract Full text Figures/Tables Video References Cited by

The development and validation of clinical prediction models based on artificial intelligence (AI) and machine learning (ML) methods have become increasingly widespread. However, the prediction model bias risk and applicability evaluation tool developed in 2019 (i.e., PROBAST-2019) has shown significant limitations. Therefore, an expanded and updated version of the PROBAST-2019 tool was released in 2025, known as the PROBAST+AI tool. The tool is divided into two parts including model development and model evaluation. It aims to comprehensively and systematically evaluate potential methodological quality issues in model development, bias risks in model evaluation, and the applicability of models, regardless of the modeling method used. This paper provides a systematic interpretation of the PROBAST+AI tool's items and case analyses, with the aim of guiding and assisting researchers engaged in related studies and promoting the high-quality development of clinical predictive model research.

1.	谷鸿秋, 周支瑞, 章仲恒, 等. 临床预测模型: 基本概念、应用场景及研究思路. 中国循证心血管医学杂志, 2018, 10(12): 1454-1456+1462.Gu HQ, Zhou ZR, Zhang ZH, et al. Clinical prediction models: basic concepts, application scenarios, and research strategies. Chin J Evid Based Cardiovasc Med, 2018, 10(12): 1454-1456, 1462.
2.	孙聪, 戴国华, 侯晓铭, 等. 中医预警因素在疾病风险预后研究中的选择思路与方法. 中西医结合心脑血管病杂志, 2021, 19(17): 3033-3035.Sun C, Dai GH, Hou XM, et al. Selection criteria and methods for traditional chinese medicine risk factors in disease risk prognosis research. Chin J Integr Med Cardio-Cerebrovasc Dis, 2021, 19(17): 3033-3035.
3.	鲁小丹, 卫建华, 沈建通, 等. 预测模型系统评价的制作方法与步骤. 中国循证医学杂志, 2023, 23(5): 602-609.Lu XD, Wei JH, Shen JT, et al. Methods and processes for producing a systematic review of predictive model studies. Chin J Evid-Based Med, 2023, 23(5): 602-609.
4.	景城阳, 冯琳, 李嘉琛, 等. 中医临床预测模型研究的概况性综述. 中华中医药杂志, 2024, 39(12): 6815-6820.Jing CY, Feng L, Li JC, et al. Scoping review of study on clinical prediction model of traditional Chinese medicine. China J Tradit Chin Med, 2024, 39(12): 6815-6820.
5.	Wessler BS, Nelson J, Park JG, et al. External validations of cardiovascular clinical prediction models: a large-scale review of the literature. Circ Cardiovasc Qual Outcomes, 2021, 14(8): e007858.
6.	Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ (Clinical research ed. ), 2015, 350: g7594.
7.	Collins GS, Moons K GM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods BMJ (Clinical research ed. ), 2024, 385: e078378.
8.	周小芹, 刘慧珍, 王婷, 等. 基于大语言模型的临床预测模型研究报告指南(TRIPOD-LLM)解读. 中国胸心血管外科临床杂志, 2025, 32(7): 940-946.Zhou XQ, Liu HZ, Wang T, et al. Interpretation of the TRIPOD-LLM reporting guideline for studies using large language models. Chin J Clin Thorac Cardiovasc Surg, 2025, 32(7): 940-946.
9.	Gallifant J, Afshar M, Ameen S, et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med, 2025, 31(1): 60-69.
10.	Andaur Navarro CL, Damen JAA, Takada T, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ (Clinical research ed.), 2021, 375: n2281.
11.	Andaur Navarro CL, Damen JAA, Takada T, et al. Completeness of reporting of clinical prediction models developed using supervised machine learning: a systematic review. BMC Med Res Methodol, 2022, 22(1): 12.
12.	Cai Y, Cai YQ, Tang LY, et al. Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review. BMC Med, 2024, 22: 56.
13.	李泽宇, 詹正哲, 程嘉仪, 等. 基于人工智能的临床预测模型研究报告规范(TRIPOD+AI)中文解读. 中国循证医学杂志, 2025, 25(03): 339-343.Li ZY, Zhan ZZ, Cheng JY, et al. A Chinese introduction to TRIPOD+AI statement: transparent reporting of multivariable prediction models for individual prognosis or diagnosis that use artificial intelligence. Chin J Evid-Based Med, 2025, 25(03): 339-343.
14.	Moons KGM, Damen JAA, Kaul T, et al. PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ, 2025: e082505.
15.	Moons KGM, Wolff RF, Riley RD, et al. PROBAST: A tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med, 2019, 170(1): W1-W33.
16.	Collins GS, Dhiman P, Andaur Navarro CL, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ open, 2021, 11(7): e048008.
17.	Riley RD, Snell KI, Ensor J, et al. Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes. Stat Med, 2019, 38(7): 1276-1296.
18.	Riley RD, Snell KIE, Ensor J, et al. Minimum sample size for developing a multivariable prediction model: Part I–Continuous outcomes. Stat Med, 2019, 38(7): 1262-1275.
19.	Riley RD, Ensor J, KIE S, et al. Calculating the sample size required for developing a clinical prediction model. BMJ (Clinical research ed. ), 2020, 368: m441.
20.	Sisk R, Sperrin M, Peek N, et al. Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study. Stat Methods Med Res, 2023, 32(8): 1461-1477.
21.	Šinkovec H, Heinze G, Blagus R, et al. To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets. BMC Med Res Methodol, 2021, 21(1): 199.
22.	Riley RD, Snell KIE, Martin GP, et al. Penalization and shrinkage methods produced unreliable clinical prediction models especially when sample size was small. J Clin Epidemiol, 2021, 132: 88-96.
23.	Van Calster B, Van Smeden M, De Cock B, et al. Regression shrinkage methods for clinical prediction models do not guarantee improved performance: Simulation study. Stat Methods Med Res, 2020, 29(11): 3166-3178.
24.	曾竟, 何小龙, 胡华娟, 等. 基于机器学习构建急性心力衰竭患者易损期死亡或再入院风险预测模型. 陆军军医大学学报, 2024, 46(7): 738-745.Zeng J, He XL, Hu HJ, et al. Construction of a risk prediction model for predicting death or admission in acute heart failure patients during vulnerable phase based on machine learning. J Army Med Univ, 2024, 46(7): 738-745.

1. 谷鸿秋, 周支瑞, 章仲恒, 等. 临床预测模型: 基本概念、应用场景及研究思路. 中国循证心血管医学杂志, 2018, 10(12): 1454-1456+1462.Gu HQ, Zhou ZR, Zhang ZH, et al. Clinical prediction models: basic concepts, application scenarios, and research strategies. Chin J Evid Based Cardiovasc Med, 2018, 10(12): 1454-1456, 1462.
2. 孙聪, 戴国华, 侯晓铭, 等. 中医预警因素在疾病风险预后研究中的选择思路与方法. 中西医结合心脑血管病杂志, 2021, 19(17): 3033-3035.Sun C, Dai GH, Hou XM, et al. Selection criteria and methods for traditional chinese medicine risk factors in disease risk prognosis research. Chin J Integr Med Cardio-Cerebrovasc Dis, 2021, 19(17): 3033-3035.
3. 鲁小丹, 卫建华, 沈建通, 等. 预测模型系统评价的制作方法与步骤. 中国循证医学杂志, 2023, 23(5): 602-609.Lu XD, Wei JH, Shen JT, et al. Methods and processes for producing a systematic review of predictive model studies. Chin J Evid-Based Med, 2023, 23(5): 602-609.
4. 景城阳, 冯琳, 李嘉琛, 等. 中医临床预测模型研究的概况性综述. 中华中医药杂志, 2024, 39(12): 6815-6820.Jing CY, Feng L, Li JC, et al. Scoping review of study on clinical prediction model of traditional Chinese medicine. China J Tradit Chin Med, 2024, 39(12): 6815-6820.
5. Wessler BS, Nelson J, Park JG, et al. External validations of cardiovascular clinical prediction models: a large-scale review of the literature. Circ Cardiovasc Qual Outcomes, 2021, 14(8): e007858.
6. Collins GS, Reitsma JB, Altman DG, et al. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ (Clinical research ed. ), 2015, 350: g7594.
7. Collins GS, Moons K GM, Dhiman P, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods BMJ (Clinical research ed. ), 2024, 385: e078378.
8. 周小芹, 刘慧珍, 王婷, 等. 基于大语言模型的临床预测模型研究报告指南(TRIPOD-LLM)解读. 中国胸心血管外科临床杂志, 2025, 32(7): 940-946.Zhou XQ, Liu HZ, Wang T, et al. Interpretation of the TRIPOD-LLM reporting guideline for studies using large language models. Chin J Clin Thorac Cardiovasc Surg, 2025, 32(7): 940-946.
9. Gallifant J, Afshar M, Ameen S, et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nat Med, 2025, 31(1): 60-69.
10. Andaur Navarro CL, Damen JAA, Takada T, et al. Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review. BMJ (Clinical research ed.), 2021, 375: n2281.
11. Andaur Navarro CL, Damen JAA, Takada T, et al. Completeness of reporting of clinical prediction models developed using supervised machine learning: a systematic review. BMC Med Res Methodol, 2022, 22(1): 12.
12. Cai Y, Cai YQ, Tang LY, et al. Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review. BMC Med, 2024, 22: 56.
13. 李泽宇, 詹正哲, 程嘉仪, 等. 基于人工智能的临床预测模型研究报告规范(TRIPOD+AI)中文解读. 中国循证医学杂志, 2025, 25(03): 339-343.Li ZY, Zhan ZZ, Cheng JY, et al. A Chinese introduction to TRIPOD+AI statement: transparent reporting of multivariable prediction models for individual prognosis or diagnosis that use artificial intelligence. Chin J Evid-Based Med, 2025, 25(03): 339-343.
14. Moons KGM, Damen JAA, Kaul T, et al. PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ, 2025: e082505.
15. Moons KGM, Wolff RF, Riley RD, et al. PROBAST: A tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med, 2019, 170(1): W1-W33.
16. Collins GS, Dhiman P, Andaur Navarro CL, et al. Protocol for development of a reporting guideline (TRIPOD-AI) and risk of bias tool (PROBAST-AI) for diagnostic and prognostic prediction model studies based on artificial intelligence. BMJ open, 2021, 11(7): e048008.
17. Riley RD, Snell KI, Ensor J, et al. Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes. Stat Med, 2019, 38(7): 1276-1296.
18. Riley RD, Snell KIE, Ensor J, et al. Minimum sample size for developing a multivariable prediction model: Part I–Continuous outcomes. Stat Med, 2019, 38(7): 1262-1275.
19. Riley RD, Ensor J, KIE S, et al. Calculating the sample size required for developing a clinical prediction model. BMJ (Clinical research ed. ), 2020, 368: m441.
20. Sisk R, Sperrin M, Peek N, et al. Imputation and missing indicators for handling missing data in the development and deployment of clinical prediction models: A simulation study. Stat Methods Med Res, 2023, 32(8): 1461-1477.
21. Šinkovec H, Heinze G, Blagus R, et al. To tune or not to tune, a case study of ridge logistic regression in small or sparse datasets. BMC Med Res Methodol, 2021, 21(1): 199.
22. Riley RD, Snell KIE, Martin GP, et al. Penalization and shrinkage methods produced unreliable clinical prediction models especially when sample size was small. J Clin Epidemiol, 2021, 132: 88-96.
23. Van Calster B, Van Smeden M, De Cock B, et al. Regression shrinkage methods for clinical prediction models do not guarantee improved performance: Simulation study. Stat Methods Med Res, 2020, 29(11): 3166-3178.
24. 曾竟, 何小龙, 胡华娟, 等. 基于机器学习构建急性心力衰竭患者易损期死亡或再入院风险预测模型. 陆军军医大学学报, 2024, 46(7): 738-745.Zeng J, He XL, Hu HJ, et al. Construction of a risk prediction model for predicting death or admission in acute heart failure patients during vulnerable phase based on machine learning. J Army Med Univ, 2024, 46(7): 738-745.

Chinese Journal of Clinical Thoracic and Cardiovascular Surgery

Latest ArticlesPROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods—Chinese interpretation

Abstract Full text Figures/Tables Video References Cited by

Format

Content