Comprehensive Summary
With the introduction of online AI bots such as ChatGPT, there remains a lingering question on the reliability and consistency of the information it provides to the public. Cai et al. investigated the utility of ChatGPT in providing accurate, reliable information on skin cancer prevention. The way research was performed was through ChatGPT4 and three board-certified dermatologists. The researchers created 25 core questions on skin cancer recommendations with an additional 15 subset questions enriched with geographic and racial demographics. Upon analysis, 24 of 25 responses were graded as “clinically appropriate” and “consistent.” In the subset analysis with variation of geography and race, 9 out of 16 questions were graded “consistent”, with 14 of 16 graded “clinically appropriate.” These findings suggest that ChatGPT’s responses were reliable for general questions but demonstrated inaccuracies when demographic factors were considered. For example, sunscreen recommendations were inconsistent across racial backgrounds. Taken as a whole, these findings suggest potential clinical applications of ChatGPT in patient education, but not without strong reservation or further modification before implementation.
Outcomes and Implications
The findings of Cai et al. suggest that ChatGPT may be a helpful tool in guiding patient’s behaviors while outside of a clinical setting. Nevertheless, these data reiterate the caution that must be exhibited when employing such tools. More safeguards and utilization of accurate clinical information by chatbots such as ChatGPT must be employed in order to ensure that the clinical or medical advice being provided to patients is both reliable and factually accurate. Until these two criteria can be met, widespread reliance by patients on such platforms—without supplementary education being sought from physicians—is highly discouraged.