Comprehensive Summary
This study by Borgonovo et al. tests 15 large language models (LLM) to solve osteoarticular infection clinical cases. The models tested were AMBOSS GPT, ChatGPT 4o mini, Claude 3.7 Sonnet, Consensus GPT, DeepSeek, Gemini 2.0 Flash, Gemini 2.5 Pro, GPT-4o, MedGPT, Microsoft Copilot, OpenEvidence, Perplexity, Perplexity Pro, PubMed Buddy, and Scite.ai Plus. Each LLM was presented 126 questions developed by the author including topics of diabetic footy infection, fracture-related infections, prosthetic Joint Infection, septic arthritis and vertebral osteomyelitis. OpenEvidence and Microsoft Copilot had the highest overall accuracy of 94.4% correct answers. It was also found that some questions that reflected an area of limited evidence were consistently answered incorrect by most LLMs.
Outcomes and Implications
Osteoarticular infection is a growing issue in modern health care. As global life expectancy increases, more people require orthopedic implants and cases of joint infections inevitably increases. With this problem demanding more and more resources, the use of AI to counter the shortage of specialists in this field is a possible solution. LLMs in other fields such as cardiology or oncology have been tested. However, their performance in Osteoarticular infection has not yet been addressed. This study proves that current LLMs such as OpenEvidence and Microsoft Copilot show great accuracy and have the potential to assist physicians.