Machine learning for endoscopic third ventriculostomy success prediction—a systematic review and meta-analysis

Comprehensive Summary

The aim of this study was to evaluate the ability of machine learning models (MLMs) to predict rates of success for an endoscopic third ventriculostomy (ETV), a method used to relieve pressure from hydrocephalus. Studies utilizing MLMs were compared with those using Endoscopic Third Ventriculostomy Success Score (ETVSS) and logistic regression (LR) models, standard methods of success rate prediction. Multiple databases were used to search for studies that included those with ETV, used advanced MLMs (not including those using ETVSS and/or LR), and those that reported effectiveness of their respective models. 2 authors independently assessed the quality of included studies, and studies focused on the wrong objectives were filtered out in the selection process, leaving 4 final articles. All articles were flagged with having high or moderate bias according to funnel plot asymmetry and Egger’s test. Azim et al. 's model (ANN) had the highest AUC (0.87), while Malhotra et al. 's (SVM) had the lowest (0.52), with values below 0.6 indicating poor model performance and those between 0.8 to 0.9 as high performance. ANN was found to be an outlier for heterogeneity, meaning that its performance data was highly varied and not consistent with other studies, and increased overall heterogeneity from 46% to 96% when included. AUCs and heterogeneity between models with imaging access and those without were similar, respectively (AUC = 0.74 vs. 0.72 ; I2 = 96% vs. 97%). When MLMs were compared with the ETVSS method and LR model, there was no statistically significant difference found (p = 0.65 and 0.45 respectively). Average AUC for all models was calculated to be 0.63 (range being 0.52 to 0.87), indicating a moderate prediction ability. Heterogeneity above 90% between studies could be explained by differences in data type included as well as patient demographics and overall study design. Researchers lastly noted that more model training, study variables, trials standardization between studies, diverse training sets, different model types, and less biased study approaches would benefit future cohorts.

Outcomes and Implications

Though methods like the ETVSS are utilized for success rate prediction in ETV, some studies have highlighted that it often underestimates success rate values. Consequently, there needs to be more accurate prediction tools for this highly successful procedure. Seeing the potential for MLMs to improve these predictions, the study’s authors condensed data and used statistical analysis on 4 studies ultimately showing heterogenous results that currently do not outperform more traditional models. With further validation, optimization in larger, multicenter studies, improved data reporting, and external validation, researchers note that there is still potential to develop MLMs that will better predict ETV outcomes in pediatric patients with obstructive hydrocephalus in the future.