Accuracy is not enough: explainable boosting machine model and identification of candidate biomarkers for real-time sepsis risk assessment in the emergency department

Comprehensive Summary

Yagin et al. conducted a retrospective analysis to evaluate the ability of an explainable boosting machine (EBM) model to identify biomarkers and provide both high accuracy and clinical interpretability regarding sepsis risk assessment in the emergency department. The researchers analyzed open-access data from a prospective observational study of emergency department adult patients with and without sepsis. Among 1,572 patients, 560 had sepsis. To address the class imbalance in the sepsis distribution of the sample, SMOTE-NC was applied to the training dataset to create a balanced dataset. The authors assessed the diagnostic performance of sepsis biomarkers alone and in conjunction with Sepsis-3 criteria, which defines organ failure as a sudden change of at least two points in the Sequential Organ Failure Assessment (SOFA) score due to infection. Bacterial infection was confirmed by a positive blood culture for bacteremia and identification of associated bacteria by culture. The data were partitioned into 80% training and 20% testing sets. The process was repeated 100 times and the average performance metrics were reported to maintain reliability. Five tree-based models (EBM, CatBoost, AdaBoost, Gradient Boosting, LightGBM) were trained and evaluated. The performances of all five models were compared on the original data and the SMOTE-NC balanced data. This resampling method increased the performance for all the models, particularly the F1-score, AUC value, and sensitivity. The EBM model achieved the highest performance after resampling, reaching 79.1% F1-score, 84.8% AUC, and 80.9% sensitivity. The global explanation analysis of the optimal EBM model identified the variables with the greatest weight in decision-making as positive blood culture, oxygen saturation, and procalcitonin, respectively. Clinical inflammatory markers such as age, neutrophil-lymphocyte ratio, C-reactive protein, and vital signs (temperature, blood pressure, respiratory rate) were also significant predictors. Laboratory parameters such as leukocyte count, hemoglobin, and clinical outcome factors (28-day survival and intensive care unit admission) also played significant roles. The transparent structure of the EBM model graphically displays the contribution of each variable in determining sepsis risk along with estimated accuracy, facilitating clinician interpretability.

Outcomes and Implications

This study presents an integrated assessment of clinical and laboratory parameters using explainable machine learning models for early prediction of sepsis risk. The findings highlight the EBM model for its high performance and transparency, supporting its potential use in clinical decision support systems. Further external validation is needed through prospective multicenter trials across diverse patient populations and health care settings. Important limitations include reduced generalizability due to the single-center data source, lack of rare clinical scenarios in the testing data, and dependence on laboratory parameters that may be unavailable in low-resource settings.