Comprehensive Summary
This study, carried out by Ito and colleagues, aimed to evaluate the overall effectiveness of Japanese language educational materials about Monkeypox (mpox) in comparison to educational texts generated by ChatGPT-40. A systematic quantitative analysis was conducted from August 20 to August 30 2024, and used 119 web pages found via the google search engine, for analysis. Additionally, 30 texts generated with ChatGPT-40 were used for comparison. Each text was evaluated in regard to three categories: understandability and actionability; readability; and credibility. It was found that texts generated by ChatGPT-40 scored significantly higher in understandability and readability, whereas the web-based articles scored higher in actionability. These results indicate that ChatGPT-40 generated texts offer the benefit of increased accessibility through a straightforward communication style, which resolves the issue of medical resources often using jargon difficult for the average person to understand. On the other hand, AI-generated texts often lack relevant visual aids, such as graphs and charts, which contribute to reader comprehension. However, these results may have been due to lack of specificity in the prompts used to generate the materials, rather than limitations in the AI itself.
Outcomes and Implications
One common issue encountered in the propagation of medical knowledge is a lack of accessibility to the general public in terms of vocabulary and reading level. This barrier can contribute to misinformation if not addressed; therefore, AI presents a useful solution by digesting difficult to comprehend medical information and communicating it at an appropriate reading level for a general population. Usage of AI in this manner has twofold benefits: one being the improvement of accessibility of educational materials, and two being a decrease in burden on medical professionals by enhancing what they have already produced. Although the texts generated by AI are generally more accessible than web-based sources, the lack of integrated visual aids hinders their serviceability. This problem can be mitigated through human intervention, adding curated figures to the AI-generated text to improve the overall impact of the materials. In this way, clinicians can work in tandem with AI resources to produce impactful educational public health materials.