A new study from the Boston-based Massachusetts General Hospital’s Center for Precision Psychiatry examined how accurately clinicians can predict suicide attempt risk using traditional structured assessments, and whether machine learning models can enhance those predictions.
The analysis included 89,957 patients ages 5 and older who received suicide risk assessments across 12 hospitals in the Boston-based Mass General Brigham system between July 2019 and February 2023. Researchers Kate Bentley, PhD, Chris Kennedy, PhD, and Taylor Burke, PhD, used a prognostic study design based on data from the electronic health record and identified emergency department visits with ICU-10 codes for suicide attempts within 90 and 180 days of the initial assessment.
Here are four findings:
- Clinicians used the SAFE-T framework to assign suicide risk levels — minimal, low, moderate or high — during outpatient, inpatient and emergency department visits. The framework includes structured questions about suicidal thoughts, behaviors and intent.
- Clinician assessments moderately predicted 90-day suicide attempts in outpatient settings (AUC=0.77), but showed lower accuracy in inpatient (AUC=0.64) and emergency department (AUC=0.60) settings. AUC, or area under the curve, represents the “probability that a randomly chosen patient who attempts suicide will have been judged at higher risk than a randomly chosen patient who does not,” with 1.0 indicating perfect accuracy and 0.5 meaning no better than chance.
- Models incorporating all 87 clinician-documented assessment variables improved 90-day predictive accuracy to AUC values of 0.87 for outpatient, 0.79 for inpatient and 0.76 for the emergency department, with similar results for the 180-day suicide risk predictions.
- Integrating these models into the EHR could enable real-time suicide risk estimation and support targeted intervention strategies immediately following clinician assessments, the study’s authors said.
Read the full study here.
