Explainability in AI-enabled medical neurotechnology: a scoping review

Comprehensive Summary

There has been an increase in artificial intelligence neurotechnologies, including brain-computer interfaces, AI-assisted neuromodulation, neurorehabilitation devices, and neuroimaging decision-support systems. These neurotechnologies can improve diagnosis and treatment decisions in a clinical context. Although artificial intelligence models are becoming increasingly important in medicine, there are concerns about how their outputs are generated. This can limit trust not only in the model but also in a physician's decision regarding treatment plans for a patient. The authors proposed a scoping review on research surrounding artificial intelligence neurotechnologies, as well as evaluation methods and ethical considerations. They searched databases such as PubMed and Scopus to conduct their review and focused on sources that specifically explain the implementation and results of artificial intelligence in medicine. They focused on different types of neurotechnologies, the artificial intelligence models used, and clinical domains. The authors found that the majority of the studies published on major biomedical databases surround neuroimaging-based diagnosis, for instance, Alzheimer's, brain tumors, stroke, and epilepsy. The most common methods in these models were feature importance and visualization-based explanations. However, most studies focused only on technical performance and did not necessarily assess decision impact or clinical usefulness. Ethical analysis was another topic that was very limited in discussion regarding physician trust, underscoring the need for further analysis and studies that more fully explain the models.

Outcomes and Implications

The use of artificial intelligence with a thorough explanation of the models could increase clinician trust and adoption. It can also help to support clinical reasoning and biases; however, current systems contain a lack of evidence with oversimplified explanations. The explanation of these models and how they are used is also important in life-threatening decisions for patients when it comes to accountability and ethical use of neurotechnology. In order to fully incorporate artificial intelligence in a clinical setting, there needs to be expanded research and in-depth explanations of models.