Assessment of Physician Preferences for Large Language Model-Generated Responses Across Geographic Regions and Clinical Experience Levels: Preliminary Survey Study

Comprehensive Summary

This research, presented by Brooks et al. examined physician preferences to responses generated by LLM of real patient questions. A cross sectional survey was administered online between March and May 2025 sourced from r/AskDocs forum on Reddit, of which each question had a human doctor’s response and LLM generated responses from GPT-4.0 and Meta AI. Physicians with medical licenses (n = 52) from various countries were recruited for this study with half having greater than five years of clinical experience. Responses were ranked by how correct and responsive they were, in a range, 1 being most preferred and 3 least preferred. Results indicated that responses derived from GPT-4.0 were most preferred (~1.63), then Meta AI (~1.83), and finally physician responses were least preferred (~2.53). Africa, Asia, and North America were geographical regions found to favor GPT-4.0 reponses while Meta AI was more preferred in some parts of Europe and the Americas. Both physicians with less than five years of clinical practice and physicians with greater than 15 years of experience favored LLM responses instead of human-written answers. Brooks et al. acknowledged that doctors may trust AI more given that the LLM responses were significantly favored by physicians across multiple locations and stages of experience.

Outcomes and Implications

Physicians widely chose LLM answers over human written responses for patient questions showing that AI may well become a trusted tool in the clinical setting and in patient-centered activities. AI integration is emerging in medicine, however, physician input and oversight remain critical to proper medical care. Not only that, but LLM could be a significant support for delivering information to patients or providing counseling help.