Comprehensive Summary
Szolnoky et al. examined the potential integrations of LLM with other tools like electronic health record systems, knowledge databases, and other AI models to automate actions and tasks. One shortcoming of LLMs is they struggle with processing tasks that require up-to-date information or interaction with the external environment. A tool complimenting an LLM would need to harness search engines, calculators, and other functionalities not native to the LLM itself. The study uses a fine-tuned version of OpenAI's ChatGPT 4 to autonomously manage an example patient using a simulated electronic health record and three distinct databases: one containing disease-specific management information, one listing available diagnostic tests, and one with various risk-scoring models. The LLM autonomously managed the patient's care, calling on databases to gather details needed to calculate the patient's risk of having prostate cancer. The LLM avoided irrelevant data in the tools but made multiple erroneous calls, the most concerning of which was ordering an MRI despite a parameter description explicitly stating a digital rectal exam should be used to estimate prostate volume.
Outcomes and Implications
Integrating LLMs into other tools has the potential to revolutionize care. The errors in the example of using an LLM with tools in patient care demonstrate the critical need for human oversight for clinical decision-making. LLMs also need not be implemented in fully autonomous solutions but can leave space for human involvement. Centrally run LLMs pose a challenge for medical uses, as patient data has higher security levels. Open LLMs will allow for applications that respect patient information privacy.