Region-based Phoneme Decoding during Production from Stereo-electroencephalography

Introduction: Recent research has shown that brain-computer interfaces (BCIs) can be used to decode speech from neural activity, with the goal of aiding patients with speech impairments, such as dysarthria or aphasia. Stereo-electroencephalography (sEEG) has emerged as a means of achieving widespread coverage across multiple regions of the language network with minimal surgical risks.

Methods: 25 patients, with sEEG electrodes implanted for seizure onset localization, participated in a sequential production task where four words with a constrained CVC or CVCC sequence were presented to patients with intractable epilepsy.

Results: Using high gamma trials of neural data, Seq2Seq models were built to either decode a set length of phoneme sequences (Encoder) or variable length sequences (Encoder-Decoder). Encoder models performed significantly better than linear models for decoding during production, with an average phoneme error rate (PER) of 59.38 +/- 4.61 SE, with the best subject achieving 40% PER during production. The encoder-decoder model, however, gave a crucial advantage achieving high decoding accuracies during pre-articulatory activity with an average PER of 65.17 +/- 5.31 S.E., and the best subject achieving 48% PER. Additionally, we parcellated the electrodes into regions involved in speech production and evaluated their influence on phoneme decoding by themselves as well as when part of a network using a lesioning-based analysis. We found significant influence from regions such as IFG, SCG, pSTG, and pMTG on speech reconstruction with the various linear and non-linear models.

Conclusion : In this study, we successfully demonstrated the feasibility of predicting sequentially articulated phonemes by leveraging neural representations across the language network using sEEG electrodes. Future work will involve replicating similar decoding frameworks with more naturalistic speech production tasks as well as investigating their role in reconstructing imagined speech.