Evaluating ChatGPT-generated psychoeducation for mood disorders: comparative insights from patients and mental health professionals

Comprehensive Summary

This study analyzed ChatGPT-generated psychoeducation for mood disorders. In the cross-sectional study, 30 patients asked ChatGPT 5 open-ended questions and its responses were rated on a five-point scale for relevance, comprehensibility, usefulness, empathy, and acceptance. These responses were then blindly evaluated by three psychiatrists and three psychiatric rehab technicians (PRTs). In the results, patients assigned higher scores (22.43±2.64) than psychiatrists (15.42±2.02) and PRTs (17.63±3.39). The biggest gaps were seen in empathy and acceptance, while relevance, usefulness, and comprehensibility saw less variability. PRT ratings were closer to the patient group ratings on relevance, comprehensibility, usefulness, but sided more with the psychiatrists’ ratings on empathy and acceptance. In the discussion, one of the points discussed by the authors is why patients might receive ChatGPT as more empathetic. In their response, the authors said that AI models are trained to provide responses that are more comforting to the users, which may appear as more empathetic but have no clinical intent.

Outcomes and Implications

From their research, the authors indicate that they are optimistic about the use of AI models in psychoeducation. Adding ChatGPT responses to into treatments between clinical appointments can help patient understanding and engagement with treatment. However, the authors also noted that these tools should be developed under strict medical guidance and that more research needs to be done with larger sample sizes. Using large language model-powered support as an adjunct to treatment as usual may help patients in need receive in-the-moment extra support in between therapy sessions.