Comprehensive Summary
This study examines the ability of AI and large language models (LLM) to educate the public on a minimally-invasive skin cancer removal technique, Mohs Micrographic Surgery (MMS). With the minimal retention rate that patients have after face-to-face interactions with their dermatologists, the Internet is the most accessible and used tool to further their knowledge about the procedure. Due to skeptical concerns about LLM’s accuracy and comprehensibility, this study took ten of the most searched Google questions regarding MMS and compared the responses from AI platforms and the Google Search Engine. Then, these responses were compiled into a survey that was given to doctors to evaluate the accuracy of answers based on 3 main factors: Is this appropriate information to provide to patients? Is the information accurate? Is there any practical use for such information to be incorporated into clinical practice? Additionally, the study enlisted fifteen Mohs Surgeons to read and rate the responses. The findings revealed that about 75% of the responses were rated as mostly accurate or higher. Despite this, only 33% of the responses from AI-generated platforms were rated as “sufficient” to be used in a clinical setting, with wary around its comprehensibility level for patients to easily understand the content. Overall, this study highlights the potentiality of LLM and its use to offer clarity about skin cancers, with caution.
Outcomes and Implications
With the current popularity of LLM by patients to increase their understanding of certain skin cancers, there is also an increase in skepticism surrounding how accurate and useful the information is. In this case, the reliability of AI may be questioned due to the lack of true understanding of the patient’s comprehension level and unique circumstances. The study concluded that AI is a useful tool with specific online resources like ChatGPT and Bard providing accurate responses. This study largely supports the use of LLM over traditional search engines due to the greater accuracy in its responses and its evaluation as an appropriate information source for patients to use, but not without caution. Specifically, limitations such as the inability of such AI platforms to consider the variability of each situation and comprehensibility by each patient must be rectified before we assess the merit of LLMs and promote their widespread application.