Comprehensive Summary
In this study, the ability of large language models (LLMs) to be able to predict the WHO molecular subtype in adult-type diffused gliomas in 2021 solely from free text radiology MRI reports was explored. A total of 2,169 adults with pathologically confirmed gliomas from four hospitals in both Asian and European countries were evaluated, utilizing narrative radiology reports, not images, as input for the LLM. Various transformer models for natural language processing tasks, comprising both closed-source as well as open-source LLMs of different sizes, were tested with both naive and knowledge-enhanced prompting, with comparisons made between different prompting paradigms, not between models and human professionals or current clinical practice. The overall accuracy was around 76–80%, and increased up to 84% when domain knowledge was added to prompts; larger models were less sensitive to prompting. Performance was higher for reports authored by subspecialty neuroradiologists, underscoring dependence on documentation quality.
Outcomes and Implications
In terms of practical applications, these results show that LLMs could be useful during the early diagnostic stage where they might help form hypotheses regarding tumor molecular phenotype from MRI report data before receiving final molecular diagnoses. While they might improve communication and initial discussions surrounding patient management during tumor workup or help streamline patient discussions regarding test results, they do not have a place in patient assessment on their own due to their reliance on diagnostic quality, individual model, and accuracy of calibration regarding eventual patient outcomes.