eeg-to-text-real-world-feasibility

name	eeg-to-text-real-world-feasibility
description	Neuropsychology-inspired EEG-to-Text benchmark (COFETT) that addresses EEG instability and enables teacher-forcing-free evaluation, providing evidence for real-world EEG2Text feasibility. Use when evaluating or building non-invasive brain-to-text decoders, designing EEG benchmarks, or studying EEG instability in neural decoding.

EEG-to-Text Real-World Feasibility (COFETT)

Description

This skill summarizes the methodology and findings of the ACL 2026 paper Is EEG-to-Text Feasible in Real-World Scenarios? (arXiv:2607.18749). The authors argue that existing EEG2Text benchmarks have been evaluated with teacher-forcing, which masks exposure bias and inflates performance. They identify EEG instability—trial-to-trial and session-to-session shifts in signal statistics and spatial topography—as a neglected confound, and introduce COFETT, a 128-channel high-density EEG dataset collected with a neuropsychology-informed inner-speech imagery paradigm. COFETT supports teacher-forcing-free evaluation and shows stronger discriminative power among model architectures than prior benchmarks.

When to Use

• Designing or evaluating EEG-to-text / brain-to-language decoders
• Building benchmarks for neural decoding, especially with EEG
• Studying EEG instability, non-stationarity, and cross-session generalization
• Comparing teacher-forcing vs. autoregressive evaluation for sequence decoders in BCI
• Collecting high-density EEG datasets for inner speech or imagined speech

Core Contributions

1. COFETT dataset — 128-channel high-density EEG recordings collected with a multi-round inner-speech imagery paradigm; repeated readings across sessions explicitly account for EEG instability.
2. Teacher-forcing-free evaluation — Prohibits ground-truth token feeding during inference, measuring genuine linguistic decoding from EEG rather than exposure-bias-inflated scores.
3. Feasibility evidence — Shows that EEG contains linguistically decodable information and that robust evaluation protocols can distinguish model architectures, opening a path toward practical non-invasive BCI communication.

Key Concepts

• EEG instability: Temporal drift in EEG signal statistics and topography within the same participant, driven by attention, fatigue, electrode impedance, and physiological fluctuations. This makes cross-session decoding difficult.
• Teacher-forcing evaluation: Sequence-model evaluation that feeds the ground-truth previous token to predict the next token. Inappropriate for real-world use because ground-truth tokens are unavailable; it masks exposure bias and can make models appear meaningful even when fed random noise.
• Neuropsychology-informed paradigm: Data collection design that maximizes recoverable linguistic signal through carefully spaced repetitions and multi-round inner-speech imagery.
• COFETT: Corpus OF Eeg-To-Text, a dedicated benchmark for EEG2Text evaluation. Open-sourced at https://github.com/baoyudu/COFETT.

Workflow

1. Diagnose Benchmark Limitations

• Check whether reported EEG2Text results use teacher-forcing during evaluation.
• If so, re-evaluate models autoregressively (no ground-truth tokens at inference).
• Check whether the dataset accounts for cross-session / cross-day EEG instability.

2. Collect or Augment Data for Stability

• Use high-density EEG caps (e.g., 128 channels) to capture spatial topography.
• Include repeated readings of the same linguistic items across multiple sessions/days.
• Record participants in a consistent state and track electrode impedance/quality.

3. Build a Teacher-Forcing-Free Evaluation Protocol

• Train sequence decoders with any teacher-forcing schedule during training (standard practice).
• At inference, generate text autoregressively from the EEG-conditioned initial token.
• Report metrics such as BLEU, ROUGE, CIDEr, BERTScore, and semantic similarity on the generated text, not on teacher-forced outputs.
• Compare model performance against random-EEG and shuffled-label baselines to detect inflated scores.

4. Apply Stability-Aware Preprocessing

• Consider common spatial patterns (CSP) or stationary CSP variants biased toward invariant subspaces.
• Apply domain adaptation or adversarial normalization to reduce session shifts.
• Use correlated components analysis (CCA) or similar methods to align session-specific EEG distributions.

5. Interpret Results with Neuroscience

• Compare learned representations to known language-related EEG components (e.g., N400, P600-like effects).
• Analyze whether the model relies on genuine neural signals rather than artifacts (eye movement, muscle noise).
• Report cross-session generalization as the primary robustness criterion.

Pitfalls

• Teacher-forcing inflation: High BLEU/ROUGE under teacher-forcing does NOT imply real-world usability. Always evaluate autoregressively.
• Ignoring EEG instability: Models trained on a single session may fail dramatically on data collected a day later from the same subject.
• Small datasets: COFETT emphasizes repeated within-participant data over large cross-participant data; design studies accordingly.
• Artifact leakage: Ensure models are not exploiting eye-tracking or muscle correlates that happen to align with linguistic labels.

Activation Keywords

• EEG-to-text
• EEG2Text
• brain-to-text
• teacher-forcing-free evaluation
• EEG instability
• cross-session EEG
• COFETT
• inner speech decoding
• non-invasive BCI
• 128-channel EEG
• neural decoding benchmark
• neuropsychology-informed paradigm

Resources

• Paper: https://arxiv.org/abs/2607.18749
• Dataset & code: https://github.com/baoyudu/COFETT
• DOI: https://doi.org/10.18653/v1/2026.acl-long.61

References

• Zhang, Z., Bao, Y., Ding, X., Jiang, T., & Xiong, K. (2026). Is EEG-to-Text Feasible in Real-World Scenarios? An In-Depth Analysis Using a Neuropsychology-Inspired Benchmark. Proceedings of ACL 2026, 1378–1393.