Evaluating the Agreement Between ChatGPT-4 and Validated Mental Health Scales in Older Adults: A Cross-Sectional Study

Comprehensive Summary

This study is an overview of the agreement between ChatGPT-4 and widely used mental health scales in older adults. For this study, 127 older adults were recruited from Wuhan between December of 2023 to October of 2024. Then questions were created using ChatGPT using six scales (PHQ-9, GAD-7, K10, PSS-4, ULS-6, WHO-5). The level of agreement between the ChatGPT-adapted scales and the traditional scales were then compared using Spearman correlation, Cronbach’s alpha constant, intraclass correlation coefficient (ICC), and Bland-Altman analysis. In the results, the ChatGPT-adapted scales showed a moderate to strong correlation between already proven measurements depression, anxiety, perceived stress, and loneliness. There were positive correlations observed for total scores, including PHQ-9 (rho = 0.61), GAD-7 (rho = 0.66), K10 (rho = 0.75), PSS-4 (rho = 0.71), and ULS-6 (rho = 0.73). There was a slightly weaker correlation observed for WHO-5 (rho = 0.41). For the reliability analysis, the questionnaires yielded Cronbach values of 0.66 to 0.91 and ICC values ranging from 0.47 to 0.81, displaying the consistency of these tests. In the discussion, the authors talked about the promise these ChatGPT-aided questionnaires have.

Outcomes and Implications

This research is important because these questionnaires can be personalized and be used as a supplemental mental health assessment tool in the future. But as of right now, the researchers noted that the ChatGPT questionnaires did not show any strengths beyond those of existing measures. For future studies, the authors recommend that data should be acquired from other settings for patients to have more variability in their psychopathology levels.