Decoding Micro-Electrocorticographic Signals by Using Explainable 3D Convolutional Neural Network to Predict Finger Movements

Introduction: High gamma (HG) activity in the somatosensory cortex has been correlated with individual and combined finger movements and has been used for decoding electrocorticography (ECoG). For predicting finger movements, deep learning via a neural network approach was used. To extract the explanation of the algorithm, the concept of explainable artificial intelligence was also introduced.

Methods: ECoG signals from a micro-ECoG array were captured concurrently with finger movements via a dataglove. For predictions of thumb-only, index-only, and pinch movements, a three-dimension convolutional neural network (3D-CNN) model using cross-frequency bandpass power as input was used. The Gradient-weighted Class Activation Mapping (Grad-CAM) was utilized to compute and describe the physiological implications of each bandpass power in the neural network

Results: The 3D-CNN model functioned well in predicting individual and combined finger movements. The highest weight was found in the high gamma (HG, 90–120 Hz) bandpass power when calculated using Grad-CAM. The spatial characteristics in the saliency maps between actual micro-array and Grad-CAM computed for HG activation indicated comparable presentation in thumb-only, index-only, and pinch movements. Both spatial features of the pinch movement showed a linear combination with those of the thumb-only and index-only movements.

Conclusion : The Grad-CAM in the 3D-CNN model revealed that HG activation had a major influence on real finger movement as well as a prediction in the 3D-CNN model.