Unblackboxing decision making behind artificial intelligence algorithms in intraoperative neurophysiologic monitoring

Background: We elucidate the decision making that lies behind artificial intelligence (AI) algorithms in the example of muscle classification in intraoperative neurophysiological monitoring (IONM). The goal is to uncover decisive parameters in motor evoked potentials (MEP) to understand intraoperative changes and optimize AI decision making.
Methods: We classified MEP in supratentorial surgery in a bi-centric setup, training on 160 patients from one center and validating on 50 patients from an independent center. We trained random forests (RF), and 1D and 2D convolutional neural nets (CNN) on a total of 37’000 MEPs and uncovered the decision making by looking into the feature importance and gradient class activation maps (Grad-CAM).
Results: The RF achieved 89% test accuracy and 80% accuracy on the validation dataset from the independent center, whereas the 1D CNN achieved 85% test and 76% validation accuracy. Finally, the 2D CNN achieved 86% test and 81% valication accuracy. Inspecting the RF feature importance reveals that the algorithm focuses on the time interval where the potential has highest amplitude. On the other hand, the grad-CAM reveals that the CNNs might be focusing on the biggest slope of the potential.
Conclusions: Analyzing the decision making of artificial intelligence algorithms is an essential part of ensuring the quality and evidence behind the good performances of these methods. Understanding this rationale will be crucial when improving intraoperative MEP alarm criteria. We showed the key features during identification of MEPs and validated the results in a bicentric setup. To our knowledge, it is the first time an IONM machine learning classification task has been implemented in a multicenter set-up.