Public Health

Comprehensive Summary

Recent events in public health have highlighted how important it is for public health organizations to have access to a large body of epidemiological data that they can analyze for patterns, assess for risks, and otherwise use to guide their decision making. It's for this reason that the Global Repository for Epidemiological Parameters (grEPI) - an openly-accessible repository of data for all manners of professionals - was created. Currently, an AI pipeline is in development in order to make it easier for grEPI users and operators to identify and subsequently extract epidemiological data that is relevant to the issue at hand. Multiple different language-learning models (aka LLMs) especially are being considered as a solution. The methodology used in this study to help develop the LLMs' data screening abilities can be split into the fine-tuning and then operational phase. The fine-tuning phase requires that humans input any disagreements that they have with any of the LLMs' citations and analysis of data back into the LLMs; this feedback allows all of the LLMs to be fine-tuned and improve in their ability to recognize and extract important data. The second phase is the operational phase, which demands less human involvement: the data analysis and extraction is much more LLM-driven, human input being needed for refining the analysis only if the LLMs express low confidence levels or there is disagreement between the different LLMs being used. The screening function of the LLM provides a label of the given article in the prompt based on the question being asked in the prompt. The critical agent is another function of the LLM, which can provide an alternate answer to the question provided in the prompt despite using the same article. If the critical agent is unable to provide an alternate answer, then that means the screening and critical agents agree and the original answer from the screening agent is the final label of the article. If there's disagreement between the screening and critical agent, another step called an ensemble agent is used; the ensemble agent is only used as a last resort when the two previous steps disagree as opposed to running it on all the data to avoid unnecessary costs of running an LLM. Humans can also provide the LLM with feedback if they disagree with the label that the LLM provides to an article, which the LLM can then re-incorporate for better performance in the future. It was found that the use of LLMs has helped reduce human workflow by performing analysis on the data. It was also found that merely changing question prompts to be more specific and precise changed the accuracy and precision - changing the question prompt from "is this study reporting on measles disease?" to "is the main focus of this study about measles disease?" changed accuracy from 87.4% to 81.1% and precision from 89.6% to 81.6%, most likely as changing the focus of the question from "reporting" to "about" confused the LLM during the fine-tuning phase. The critical agent was most likely to disagree with the screening agent when the human input had disagreement with the label that came from the screening agent.

Outcomes and Implications

There has been an increased need for large-scale public health data analysis in the past few years, which often requires large amounts of time and intensive labor. The usage of LLMs to screen data can help reduce human workload and streamline the process of data extraction, only provided that the LLM has been adequately trained. This reflects in the sensitivity of LLMs to wording in the original questions inputted into the prompt when screening data. Wording that is too precise can end up confusing the LLM during the initial fine-tuning phase and end up requiring more human feedback and input in the long run to correct the LLMs' labels, which is the opposite of what the end goal (reduced human involvement) is with LLM employment. The framework in this study is designed with the goal of being applicable to all LLMs, as opposed to being restricted to only one specific LLM, but more extensive research needs to be done in this area by looking at the applicability of this framework with data from more pathogens - so far this study was restricted to public health parameters in regards to measles.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team