Deep Learning prediction of BOLD-fMRI signals from simultaneous EEG during motor imagery

Simultaneous EEG-fMRI acquisitions leverage the complementary strengths of the two functional neuroimaging modalities, with promising applications in the development of neurofeedback (NF) brain-computer interfaces (BCIs). While fMRI provides superior mapping of brain activity, EEG is more accessible for NF-BCI interventions. Previous work has attempted to identify the EEG features that best predict fMRI activity patterns, but the performance achieved is still poor. In this work, we leverage a well-established deep learning network for the classification of EEG signals, EEGNet, and propose an extension to the regression task of predicting the fMRI signal at a specific time point from concurrent EEG data (R-EEGNet). We target the activity of the somatomotor network (SMN) during the execution of two motor imagery (MI) tasks used in NF-BCIs for motor rehabilitation in stroke patients. For this purpose, we use a simultaneous EEG-fMRI dataset collected from 15 healthy subjects while executing the tasks in two separate sessions. The fMRI data are analyzed to extract a time series of MI activity for each subject and task, and the R-EEGNet model is trained to predict each fMRI time sample from a 15-seconds segment of multi-channel EEG data. We evaluated the proposed R-EEGNet model performance in comparison with a conventional machine learning model (Group Lasso) trained on EEG spectral features, as well as with a Naïve model based on the EEG somatomotor rhythm. We found that R-EEGNet achieved a similar performance to Group Lasso, both being significantly superior to the Naïve model. Our results provide the first demonstration of the ability of a subject-specific deep learning model to predict fMRI motor signals based directly on the EEG signal, without the need to extract spectral features. Future work should improve model performance through further hyperparameter optimization and the exploitation of data augmentation to cope with the typically small size of EEG-fMRI datasets. © 2025 European Signal Processing Conference, EUSIPCO. All rights reserved.