Comprehensive Summary
This study evaluates the quality, accuracy, and readability of ChatGPT 3.5's responses to patient questions about distal biceps tendon repair surgery. Researchers posed 27 commonly searched questions to ChatGPT and assessed the responses using the DISCERN scale for quality, JAMA benchmark criteria for accuracy, and Flesch-Kincaid readability measures. The questions were categorized using Rothwell Criteria as Fact, Policy, or Value-based. ChatGPT responses earned DISCERN scores above 50, indicating 'good' quality, but scored 0 on all JAMA criteria due to a lack of references and disclosures. Readability was at a Flesch-Kincaid Grade Level of around 15, requiring a college graduate reading level, which exceeds the recommended sixth-grade level for patient resources. The study concludes that while ChatGPT may offer high-quality content, it lacks the reliability and accessibility needed for effective patient education.
Outcomes and Implications
As AI tools become more involved in patient education, especially in orthopedics, their use must be approached with caution. This study demonstrates that ChatGPT can serve as a supplemental education tool, but its lack of citations and complex language risks misleading patients and limiting their ability to make informed decisions. The review highlights that while ChatGPT can supplement clinical education, it cannot yet replace physician-guided counseling. Future research should focus on enhancing the reliability and readability of AI models to improve their use in clinical settings, ensuring they provide accurate and accessible information to patients.