Psychiatry

Comprehensive Summary

In this article, Wang et al. developed and tested a large language model (LLM) framework for extracting suicide-related Social Determinants of Health (SDoH) from unstructured narratives such as clinical notes. The framework involves two steps, context retrieval and relevance verification, after which SDoH factors are extracted. During context retrieval, input text is split into individual sentences, and a pre-trained LLM identifies which sentences contain the SDoH factor of interest. In the relevance verification step, chosen sentences are given to another LLM which verifies their relevance. These sentences are then given to an LLM that identifies and extracts specific SDoH factors occurring within two weeks before suicide. This framework was compared to a fine-tuned BioBERT model, a GPT-3.5-turbo End-to-End model, a GPT-3.5-turbo Chain-of-Thought model, and DeepSeek-R1, and data from the 2020 version of the National Violent Death Reporting System was used. A pilot user study was also performed where participants were tasked to annotate selected SDoH factors across incidents with and without the relevant context highlighted by the framework. The framework outperformed all baseline methods in 9 out of 10 infrequent suicide-related SDoH factors and improved F-1 scores for 5 out of 8 frequent SDoH factors. DeepSeek-R1 outperformed other models in 7 out of 16 SDoH factors, but the proposed framework outperformed it on recall for 11 factors. The pilot user study showed that the AI-assisted method was faster and slightly more accurate and that it was preferred among participants. The results suggest that the proposed framework may be suited to high-coverage extraction tasks while reasoning models like DeepSeek-R1 are preferable for minimizing false positives.

Outcomes and Implications

Suicide is a major global public health concern and can be difficult to identify and prevent. Using AI-assisted tools such as the proposed framework to identify suicide-related SDoH factors in data such as clinical notes can allow for faster and more accurate identification of suicide risk and prevent suicide. It may also reduce the stress of clinicians and allow them to perform their jobs more efficiently. However, this study has multiple limitations: the study only tested a small subset of existing SDoH factors, the dataset used may not equally represent all cultural contexts and may be biased, explainability of LLMs remains limited, and task familiarity may have introduced confounding variables to the pilot user study. Further research is needed to refine the framework and confirm the findings in this study.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

Our mission is to

Connect medicine with AI innovation.

No spam. Only the latest AI breakthroughs, simplified and relevant to your field.

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team

AIIM Research

Articles

© 2025 AIIM. Created by AIIM IT Team