Limitations of Large Language Models in Assisting PI-RADS Scoring on Prostate Biparametric MRI Text Reports

Comprehensive Summary

This study evaluated whether large language models can assist with PI-RADS v2.1 scoring using biparametric prostate MRI text reports. While LLMs demonstrated high sensitivity comparable to radiologists, they showed markedly lower specificity and positive predictive value, especially in transition and peripheral zones. Experienced radiologists achieved the best overall diagnostic performance, and results suggest LLMs are best used as adjunct tools under higher PI-RADS thresholds rather than as standalone diagnostic systems.

Outcomes and Implications

This study shows that large language models can achieve high sensitivity in PI-RADS scoring from biparametric MRI text reports, indicating potential utility as supportive screening or triage tools. However, their low specificity and positive predictive value limit their reliability for independent clinical decision-making and may increase unnecessary biopsies. The superior performance of experienced radiologists highlights that LLMs should be used as adjuncts rather than replacements.