Public Health

Comprehensive Summary

In this study, Nishisako and colleagues examine the cancer information supplied by generative AIs with retrieval-augmented generation (RAG) to determine how reliable sources can reduce the number of hallucinations from AI chatbots. The research was carried out by developing 6 types of generative AI chatbots, each a combination of a large learning model (LLM) and 3 information reference sources, and were given a set of 62 cancer-related questions. The responses to the prompts were evaluated and classified by accuracy. It was found that depending on the chatbot used, the rate of hallucinations varied significantly. Hallucinations were most common in Conv. GPT-3.5 at 39% of answers, and least common in CIS GPT-4 in 2% of the answers. Additionally, it was determined that responses generated from Google search results were more likely to generate hallucinations. Overall, when chatbots were constrained to certain data sets to answer questions, hallucinations were able to be reduced. However, it is difficult to completely eliminate hallucinations, as questions regarding cancer are broad, and do not cannot always be reliably answered with evidence-based content.

Outcomes and Implications

Misinformation concerning cancer is abundant on the internet, and AI chatbots can often contribute to this cycle through incorrect, yet seemingly plausible answers, called hallucinations. When individuals are confronted with these hallucinations, they may take steps in their healthcare plans which lead to poorer outcomes. Through work aiming to improve the responses provided by chatbots, clinicians and patients will be able to remain on the same page in terms of knowledge regarding treatment and prognosis.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team