Native Language and Stimuli Signal Prediction from EEG

Madhumitha Sakthi, Ahmed Tewfik, Bharath Chandrasekaran

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Understanding the neural processing of natural speech processing is an important first step for designing Brain-Computer Interface (BCI) based speech enhancement and speech recognition systems. Complex neural signals like electroencephalography (EEG) are time-varying and has a non-linear relationship with continuous speech. Linear models can decode stimulus features reliably, but the correlation between the reconstructed signal and continuous EEG remain low despite attempts at optimization. In the current application, we demonstrate the utility of a Recurrent Neural Networks (RNN) model to relate various stimuli features such as the envelope, spectrogram to the continuous EEG in a cocktail party scenario. We use a Long Short-Term Memory (LSTM) neural network architecture that has self-connecting loops which help in preserving past information to predict future value. Given that predictability plays a critical role in speech comprehension, we posit that such a neural network architecture yield better results. In attended condition, for native participants, the LSTM models yield 30% and 22% mean correlation improvement and for non-native participants, 43% and 37% improvement over linear models for envelope and spectrogram respectively with EEG. Finally, we have trained a single model to predict the native language of a participant using EEG and it yielded 95% accuracy.

Original languageEnglish (US)
Title of host publication2019 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Proceedings
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3902-3906
Number of pages5
ISBN (Electronic)9781479981311
DOIs
StatePublished - May 2019
Event44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019 - Brighton, United Kingdom
Duration: May 12 2019May 17 2019

Publication series

NameICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume2019-May
ISSN (Print)1520-6149

Conference

Conference44th IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2019
Country/TerritoryUnited Kingdom
CityBrighton
Period5/12/195/17/19

Funding

Research reported here was supported by the National Institute On Deafness and Communication Disorders of the National Institute of Health under Award Numbers:R01DC015504,R01DC015504, R01DC013315.

Keywords

  • EEG
  • Neural Signal Processing
  • RNN
  • Speech enhancement

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Native Language and Stimuli Signal Prediction from EEG'. Together they form a unique fingerprint.

Cite this