Comprehensive Summary
Large Language Models can be very useful in any field of healthcare due to their remarkable ability to retrieve and analyze information very quickly. This study tested ChatGPT-3.5, ChatGPT-4, and Gemini Advanced in a range of neuro-oncology cases to determine the accuracy, completeness and usefulness of their respective responses. Four questions regarding differential diagnosis, diagnostic workup, provisional diagnosis, and management plan were asked of 20 common neuro-oncology cases to test the 3 LLMs. After replication (copying into a separate document) and blinding (anonymizing the responses), senior neuro-oncologists scored the responses on 3 scales: accuracy (1-6), completeness (1-3), and usefulness (1-3). To compare the performance, ANOVA and Dunn’s post hoc tests were completed. After a full assessment ChatGPT-4 was the most accurate, most complete, and most useful. This was followed by Gemini Advanced and ChatGPT-3.5 in respective order. In order to have a more complete study, there can be more neuro-oncologists reviewing the responses to a wider array of queries.
Outcomes and Implications
AI can serve as a helpful tool in clinical decision making helping doctors quickly review files and notes without forgetting any information. AI can also curate well-rounded management plans for patients. AI can aid to reduce physician workload and burnout by completing tedious tasks while the physician maintains a more patient-faced approach.