User perceptions of machine learning models as decision support for colorectal cancer multidisciplinary team conferences (AID-SIM-2): a qualitative simulation study
DOI:
https://doi.org/10.2340/1651-226X.2026.45754Keywords:
multidisciplinary team conference, colorectal cancer, Prediction modelAbstract
Background: Multidisciplinary team (MDT) conferences are considered a cornerstone of decision-making in cancer diagnostics and care. However, the current literature has not demonstrated improved patient outcomes based on the decisions of the MDT conferences.
Aim: We aimed to evaluate how four different decision-support modalities impacted the decision-making process and the internal discussions in the MDTs in a multicenter simulation study, with a focus on user perceptions.
Methods: Four colorectal cancer centers with MDTs participated. We performed four simulations in each center. Each simulation used a different decision-support tool: (1) Current standard, (2) Current standard plus a prediction model, (3) A structured data presentation tool, and (4) A structured data presentation tool plus the prediction model. Clinician- and model-estimated risks were compared, the treatment suggestions from each site were compared, questionnaires about user perceptions were conducted after Simulations 2, 3, and 4 using a Google Form link, and a semi-structured interview was conducted at each site after the last simulation.
Results: Similar distributions of risk groups between clinicians and models were found; however, distinct discrepancies in predictions arose, particularly with higher-risk patients, highlighting the need for standardization for more complex clinical cases. The primary perceived benefit of decision support was increased standardization of care, independent of the individual physicians’ personal views. However, participants emphasized the necessity of clinician autonomy to overrule tool suggestions when identifying clinical nuances not captured by the model.
Conclusions: The colorectal cancer MDTs expressed a positive view regarding the use of prediction models and other forms of decision-support in their workflow. While clinicians and prediction models had similar risk score distributions, they diverged in the assessment of specific individual patients.
Downloads
References
Wille-Jørgensen P, Sparre P, Glenthøj A, Holck S, Nørgaard Petersen L, Harling H, et al. Result of the implementation of multidisciplinary teams in rectal cancer. Colorectal Dis. 2013;15(4):410–3 DOI: https://doi.org/10.1111/codi.12013
Lan YT, Lin JK, Jiang JK. Effects of a multidisciplinary team on colorectal cancer treatment. Formos J Surg. 2015;48(5):145–50 DOI: https://doi.org/10.1016/j.fjs.2015.07.003
Fehervari M, Hamrang-Yousefi S, Fadel MG, Mills SC, Warren OJ, Tekkis PP, et al. A systematic review of colorectal multidisciplinary team meetings: an international comparison. BJS Open. 2021;5(3)1-8 DOI: https://doi.org/10.1093/bjsopen/zrab044
Gouliaev A, Szejniuk WM, Fledelius J, Madsen HHT, Petersen RH, Rasmussen TR. Discrepancies in regional lung cancer multidisciplinary team decisions can be reduced through national consensus meetings. Acta Oncol (Madr). 2025;64:793–6 DOI: https://doi.org/10.2340/1651-226X.2025.43314
Gouliaev A, Berg J, Isaksson J, Andersén H, Rasmussen TR. Multidisciplinary team meetings for lung cancer in the Nordic countries: results from a Nordic survey. Acta Oncol. 2026;65:301–5 DOI: https://doi.org/10.2340/ao.v65.45213
Barberan-Garcia A, Ubré M, Roca J, Lacy AM, Burgos F, Risco R, et al. Personalised prehabilitation in high-risk patients undergoing elective major abdominal surgery: a randomized blinded controlled trial. Ann Surg. 2018;267(1):50–6 DOI: https://doi.org/10.1097/SLA.0000000000002293
Berkel AEM, Bongers BC, Kotte H, Weltevreden P, de Jongh FHC, Eijsvogel MMM, et al. Effects of community-based exercise prehabilitation for patients scheduled for colorectal surgery with high risk for postoperative complications. Ann Surg. 2021;275(2):299-6 DOI: https://doi.org/10.1097/SLA.0000000000004702
Bojesen RD, Grube C, Buzquurz F, Miedzianogora REG, Eriksen JR, Gögenur I. Effect of modifying high-risk factors and prehabilitation on the outcomes of colorectal cancer surgery: controlled before and after study. BJS Open. 2022;6(3):299-6 DOI: https://doi.org/10.1093/bjsopen/zrac029
Bojesen RD, Ravn J, Rasmus E, Vogelsang P, Grube C, Lyng J, et al. The dynamic effects of preoperative intravenous iron in anaemic patients undergoing surgery for colorectal cancer. 2021;23(10):2550–8 DOI: https://doi.org/10.1111/codi.15789
Miles LF, Sandhu RNS, Grobler AC, Heritier S, Burgess A, Burbury KL, et al. Associations between non-anaemic iron deficiency and outcomes following surgery for colorectal cancer: an exploratory study of outcomes relevant to prospective observational studies. Anaesth Intensive Care. 2019;47(2):152–9 DOI: https://doi.org/10.1177/0310057X19838899
Bojesen RD, Jørgensen LB, Grube C, Skou ST, Johansen C, Dalton SO, et al. Fit for surgery – feasibility of short-course multimodal individualized prehabilitation in high-risk frail colon cancer patients prior to surgery. Pilot Feasibility Stud. 2022;8(11):1–13 DOI: https://doi.org/10.1186/s40814-022-00967-8
Rosen AW, Ose I, Gögenur M, Andersen LPK, Bojesen RD, Vogelsang RP, et al. Clinical implementation of an AI-based prediction model for decision support for patients undergoing colorectal cancer surgery. Nat Med. 2025;31:3737-48 DOI: https://doi.org/10.1038/s41591-025-03942-x
Scott I, Carter S, Coiera E. Clinician checklist for assessing suitability of machine learning applications in healthcare. BMJ Health Care Inform. 2021;28(1):1-8 DOI: https://doi.org/10.1136/bmjhci-2020-100251
Win AK, MacInnis RJ, Hopper JL, Jenkins MA. Risk prediction models for colorectal cancer: a review. Cancer Epidemiol Biomarkers Prev. 2012;21:398–410 DOI: https://doi.org/10.1158/1055-9965.EPI-11-0771
Ferjani AM, Griffin D, Stallard N, Wong LS. A newly devised scoring system for prediction of mortality in patients with colorectal cancer: a prospective study. Lancet Oncol. 2007;8(4):317–22 DOI: https://doi.org/10.1016/S1470-2045(07)70045-1
Vogelsang RP, Bojesen RD, Hoelmich ER, Orhan A, Buzquurz F, Cai L, et al. Prediction of 90-day mortality after surgery for colorectal cancer using standardized nationwide quality-assurance data. BJS Open. 2021;5(3):1-9 DOI: https://doi.org/10.1093/bjsopen/zrab023
Ma M, Liu Y, Gotoh M, Takahashi A, Marubashi S, Seto Y, et al. Validation study of the ACS NSQIP surgical risk calculator for two procedures in Japan. Am J Surg. 2021;222(5):877–81 DOI: https://doi.org/10.1016/j.amjsurg.2021.06.008
van der Hulst HC, Dekker JWT, Bastiaannet E, van der Bol JM, van den Bos F, Hamaker ME, et al. Validation of the ACS NSQIP surgical risk calculator in older patients with colorectal cancer undergoing elective surgery. J Geriatr Oncol. 2022;13(6):788-95 DOI: https://doi.org/10.1016/j.jgo.2022.04.004
Gagliardi AR, Armstrong MJ, Bernhardsson S, Fleuren M, Pardo-Hernandez H, Vernooij RWM, et al. The Clinician Guideline Determinants Questionnaire was developed and validated to support tailored implementation planning. J Clin Epidemiol. 2021;113:129–36 DOI: https://doi.org/10.1016/j.jclinepi.2019.05.024
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57 DOI: https://doi.org/10.1093/intqhc/mzm042
Elo S, Kyngäs H. The qualitative content analysis process. J Adv Nurs. 2008;62(1):107–15 DOI: https://doi.org/10.1111/j.1365-2648.2007.04569.x
Juvani S, Isola A, Kyngäs H. The northern physical environment and the well-being of the elderly aged over 65 years. Int J Circumpolar Health. 2005;64(3):246–56 DOI: https://doi.org/10.3402/ijch.v64i3.17988
Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. 2004;24(2):105–12 DOI: https://doi.org/10.1016/j.nedt.2003.10.001
Lamb BW, Sevdalis N, Benn J, Vincent C, Green JSA. Multidisciplinary cancer team meeting structure and treatment decisions: a prospective correlational study. Ann Surg Oncol. 2013;20(3):715–22 DOI: https://doi.org/10.1245/s10434-012-2691-x
Soukup T, Lamb BW, Morbi A, Shah NJ, Bali A, Asher V, et al. A multicentre cross-sectional observational study of cancer multidisciplinary teams: analysis of team decision making. Cancer Med. 2020;9(19):7083–99 DOI: https://doi.org/10.1002/cam4.3366
Specchia ML, Frisicale EM, Carini E, Di Pilla A, Cappa D, Barbara A, et al. The impact of tumor board on cancer care: evidence from an umbrella review. BMC Health Serv Res. 2020;20(1):1-14 DOI: https://doi.org/10.1186/s12913-020-4930-3
Hammer RD, Fowler D, Sheets LR, Siadimas A, Guo C, Prime MS. Digital Tumor Board Solutions Have Significant Impact on Case Preparation. JCO Clin Cancer Inform. 2020;4:757–68 DOI: https://doi.org/10.1200/CCI.20.00029
Molnar C. Interpretable machine learning: a guide for making black box models explainable [Internet]. [cited 2026 Dec 17]. Available from: http://leanpub.com/interpretable-machine-learning
Ghassemi M, Oakden-Rayner L, Beam AL. The false hope of current approaches to explainable artificial intelligence in health care. Lancet Digit Health. 2021;3(11):e745–50 DOI: https://doi.org/10.1016/S2589-7500(21)00208-9
Lamb BW, Sevdalis N, Taylor C, Vincent C, Green JSA. Multidisciplinary team working across different tumour types: analysis of a national survey. Ann Oncol. 2012;23(5):1293–300 DOI: https://doi.org/10.1093/annonc/mdr453
Lamb BW, Green JSA, Benn J, Brown KF, Vincent CA, Sevdalis N. Improving decision making in multidisciplinary tumor boards: prospective longitudinal evaluation of a multicomponent intervention for 1,421 patients. J Am Coll Surg [Internet]. 2013;217(3). [cited 2026 Dec 17]. Available from: https://journals.lww.com/journalacs/fulltext/2013/09000/improving_decision_making_in_multidisciplinary.5.aspx DOI: https://doi.org/10.1016/j.jamcollsurg.2013.04.035
List H, Kristensen DB, Graumann O. ‘The highest decision-making level’ – multidisciplinary team meetings as boundary spaces. Soc Sci Med. 2025;371(September 2024):117886 DOI: https://doi.org/10.1016/j.socscimed.2025.117886
Cowley LE, Farewell DM, Maguire S, Kemp AM. Methodological standards for the development and evaluation of clinical prediction rules: a review of the literature. Diagn Progn Res. 2019;3(1):1–23 DOI: https://doi.org/10.1186/s41512-019-0060-y
Good ML. Patient simulation for training basic and advanced clinical skills. Med Educ. 2003;37(1):14–21 DOI: https://doi.org/10.1046/j.1365-2923.37.s1.6.x
Larue C, Pepin J, Allard É. Simulation in preparation or substitution for clinical placement: a systematic review of the literature. J Nurs Educ Pract. 2015;5(9):132-40 DOI: https://doi.org/10.5430/jnep.v5n9p132
Richard A. Krueger. Designing And Conducting Focus Group Research. Focus Group Interviewing. 2002;1:35–94
Derksen C, Walter FM, Akbar AB, Parmar AVE, Saunders TS, Round T, et al. The implementation challenge of computerised clinical decision support systems for the detection of disease in primary care: systematic review and recommendations. Implement Sci. 2025;20(33):1-52 DOI: https://doi.org/10.1186/s13012-025-01445-4
Bojesen RD, Gögenur I. Præhabilitering ved kolorektal cancer. Danish Colorectal Cancer Group; 2023.
Rony MKK, Parvin MR, Wahiduzzaman M, Debnath M, Bala SD, Kayesh I. ‘I Wonder if my Years of Training and Expertise Will be Devalued by Machines’: concerns about the replacement of medical professionals by artificial intelligence. SAGE Open Nurs. 2024;10:e1485 DOI: https://doi.org/10.1177/23779608241245220
Verdicchio M, Perin A. When doctors and AI interact: on human responsibility for artificial risks. Philos Technol. 2022;35(1):1-28 DOI: https://doi.org/10.1007/s13347-022-00506-6
Lundvall L, Patkar V, Acosta D, Davidson T, Jones A, Fox J, et al. Cancer multidisciplinary team meetings: evidence, challenges, and the role of clinical decision support technology. Int J Breast Cancer. 2011;125(1):1–7 DOI: https://doi.org/10.4061/2011/831605
Patkar V, Acosta D, Davidson T, Jones A, Fox J, Keshtgar M. Using computerised decision support to improve compliance of cancer multidisciplinary meetings with evidence-based guidance. BMJ Open. 2012;2(3):1-7 DOI: https://doi.org/10.1136/bmjopen-2011-000439
Watkins SC, de Oliveira Filho GR, Furse CM, Muffly MK, Ramamurthi RJ, Redding AT, et al. The effect of novel decision support tools on technical and non-technical performance of teams in managing emergencies. J Med Syst. 2022;46(11):75 DOI: https://doi.org/10.1007/s10916-022-01866-4
Holloway S, Sarosi G, Kim L, Nwariaku F, O’keefe G, Hynan L, et al. Health-related quality of life and postoperative length of stay for patients with colorectal cancer. J Surg Res. 2002;108(2):273–8 DOI: https://doi.org/10.1006/jsre.2002.6549
Additional Files
Published
How to Cite
Issue
Section
Categories
License
Copyright (c) 2026 Karoline Bendix Bräuner, Birgitte Bruun, Claus Anders Bertelsen, Vegar Johansen Dagenborg, Kristina Safir-Hansen, Rasmus Sanko, Ismail Gögenur, Lars Konge
This work is licensed under a Creative Commons Attribution 4.0 International License.
