Comprehensive Summary
This study, conducted by Yasemin Denkboy Ongen, Ayla Iren Aydin, et al., explores the accuracy and performance of different large language models in answering questions about type 1 diabetes in children. A pediatric endocrinologist posted a poll on his Instagram story asking what people wanted to know about Type 1 diabetes, and compiled the 20 most frequently asked questions. These questions were then inputted to ChatGPT 3.5, 4.0, 40, Gemini and Gemini Advanced. Five different physicians evaluated the responses from the AI models by assigning a score from 1 to 5 based on factors such as empathy, accuracy, etc. ChatGPT 40 had the highest mean score of 3.78, while Gemini had the lowest mean score at around 3.40. However, no significant differences were found among the five models (p = 0.103).
Outcomes and Implications
The use of artificial intelligence in healthcare is on the rise, but there is little research in evaluating the use of LLMs in the pediatric population. Within this group, there has been an increase in type 1 diabetes and families are seeking answers about their loved ones' condition. This paper demonstrates the potential of newer LLMs, especially ChatGPT-40 to provide clear explanations about Type I diabetes. Despite the promise of AI, further evaluation and research is needed before fully incorporating large language models in pediatric diabetes care.