AI-Assisted Dialysis Decision-Making: Assessing Agreement Between ChatGPT and Nephrologist in Initial Dialysis Indication and Prescription in Emergency and ICU Settings

Comprehensive Summary

As advances in the field of artificial intelligence (AI) rapidly emerge, the applicability of this tool for decision-making in the scope of contemporary emergency and ICU settings is being explored. The current study performs a retrospective observation to assess the level of agreement between ChatGPT (version: GPT-4, OpenAI, USA) and nephrologists’ consultation notes regarding decisions on dialysis treatment plans for patients in emergency and ICU settings. The level of agreement was focused on evaluating dialysis initiation, modality, and prescription parameters using Gwet’s AC1 coefficient and Cramér’s V Statistic for eighty-four patients seen between January 2024 to July 2024 at Mengucek Gazi Training and Research Hospital. The study found a 100% agreement on the initiation of dialysis treatment for all 84 patients. With regard to dialysis modality, findings showed a 92.9% agreement (Cramér’s V = 0.87, P<0.001) between the decisions of ChatGPT and those of the attending nephrologist. Decisions on dialysis prescription parameters for patients with hemodialysis without ultrafiltration (UF) showed high levels of agreement for all criteria evaluated (P < 0.001) with similar results for hemodialysis with UF (P <0.001). Despite the high level of agreement found between the ChatGPT tool and nephrologists' decisions, the use of AI in emergency medicine and ICU settings is complicated by the dynamic nature of these critical care settings and the need for constant reevaluation of patients’ treatment plans.

Outcomes and Implications

The findings of the study highlight the high level of agreement between ChatGPT and nephrologists’ consultation notes regarding the dialysis treatment plans of patients seen emergency and ICU settings. These conclusions open the discussion of integrating AI tools into critical care settings to serve as clinical-decision support tools. The conservative approach of the authors further implies that the use of AI should serve to support nephrologists’ decisions and standardizing decision protocols rather than replacing the clinicians' decision-making and judgements.