NeuroRead: EEG-Based Speech Decoding

The Problem

Current educational tools have no way of measuring whether a student understood what they just read in real time. Comprehension is only assessed after the fact through quizzes and tests. By then, the moment of confusion has already passed.

The brain produces measurable signals at the exact moment of confusion. The N400, a neural signature peaking around 400ms after a word is seen, directly reflects how hard the brain is working to integrate the meaning of each word. We built a system to capture and use this signal.

The Pipeline

Data Collection

Recording brain activity while reading

Hardware: BCI Core-8 consumer EEG headset, 8 channels, 250Hz
Paradigm: Rapid Serial Visual Presentation (RSVP)
Each word flashed on screen triggers a precise timestamp in the EEG recording
30 sentences presented, including semantically incongruent words to elicit N400 responses

Preprocessing

Cleaning the signal

Select EEG channels only (drop EMG/EOG)
Bandpass filter 0.1–40 Hz, downsample to 50 Hz
Robust scaling per channel (median subtraction, IQR normalization)
Cut 3-second epochs at each word onset; baseline correction over the first 500ms
Hard clamp values to ±5 to remove extreme artifacts
Final input shape: 8 channels × 150 time points + 2D electrode positions

Our Contribution

N400 Feature Extraction

N400 amplitude: mean voltage at centro-parietal channels, 300–500ms window, reflecting semantic processing difficulty per word
Theta band power (4–8 Hz): measures working memory load per word
Both z-score normalized and concatenated into the EEG feature vector

Decoding

EEG-to-Text Decoding

Built on model architecture publicly released by Meta researchers
CNN: learns spatial patterns across electrodes and temporal patterns within each epoch
Transformer: looks across the entire sentence, using surrounding word context to refine each prediction
Maps brain signal to T5 word embedding space; cosine similarity retrieval returns top-K word candidates per position

Our Contribution

LLM Semantic Correction

Language model receives the top-K candidate grid plus the per-word N400 signal
Uses linguistic context and the brain's difficulty signal together
Reconstructs the most semantically coherent sentence
The N400 signal tells the LLM where in the sentence to focus correction effort

Results

EEG-to-Text Decoding

66.7% Top-10 retrieval accuracy

5.5–6.7% Top-1 retrieval accuracy

31.8% BERT semantic similarity

0.627 Contrastive cosine similarity

1.1% Chance baseline (top-10)

Top-10 of 66.7% means the correct word appeared in the model's top 10 guesses two-thirds of the time, versus 1.1% by chance.

N400 Detection on Real EEG

t = 1.67 t-statistic

p = 0.097 p-value

0.22 Cohen's d

P7, P8 Strongest channels (posterior parietal)

A trend toward the expected N400 effect was observed with a consumer 8-channel headset: incongruent words produced more negative voltage in the 300–500ms window at posterior parietal sites, consistent with neuroscience literature. At p = 0.097, though, this does not reach statistical significance at conventional thresholds (p < 0.05).

Our Contributions

N400 feature integration

Neural comprehension markers extracted per epoch and fed as additional model inputs.

LLM correction guided by N400

The brain's difficulty signal directs where the language model focuses sentence correction.

Consumer hardware validation

N400 trend detectable with 8-channel consumer EEG, supporting real-world educational deployment.

Honest Limitations

Trained on a small dataset; ZuCo-scale data would improve accuracy
8 channels provides limited spatial resolution vs. 64-channel setups
N400 effect size is small on single trials, and did not reach conventional statistical significance (p = 0.097)
Current pipeline is offline, not real-time

Tech Stack

EEG Hardware

BCI Core-8 (8ch, 250Hz)

Signal Processing

MNE-Python SciPy

Deep Learning

PyTorch HuggingFace Transformers

Architecture

CNN encoder + Transformer (based on Meta BrainMagick)

Word Embeddings

T5-small

Contrastive Loss

SigLIP D-SigLIP

LLM Correction

Claude API

Experiment Design

RSVP paradigm, 30 sentences

Reference Dataset

ZuCo

Key Papers

Défossez et al. (2023). Decoding speech perception from non-invasive brain recordings. Nature Machine Intelligence.
d'Ascoli et al. (2025). Towards decoding individual words from non-invasive brain recordings. Nature Communications.
Duan et al. (2023). DeWave: Discrete EEG Waves Encoding for Brain Dynamics to Text Translation. NeurIPS.

As learners ourselves, we understand that learning can be a difficult and frustrating process, and that there is no single remedy. Every learner thinks and processes information differently. With this project, we hoped to contribute toward a system that meets learners where they are, making learning a more rewarding and accessible experience for everyone.

If parts of this explanation were hard to follow, that is exactly the problem we are trying to solve. Imagine a future where a lightweight wearable device could detect which concepts did not land, and automatically generate a more tailored explanation just for you.