| name | zero-shot-imagined-speech-meg |
| description | Zero-shot imagined speech decoding from MEG via imagined-to-listened cross-condition mapping. Trains models to map imagined MEG responses to listened responses, then decodes using listened-only decoder. Three-stage pipeline: (1) mapping imagined→listened MEG, (2) train contrastive word decoder on listened MEG with multi-embedding evaluation, (3) decode imagined speech via mapping pipeline on held-out subjects. Use when: imagined speech decoding, MEG BCI, cross-condition neural mapping, zero-shot brain decoding, non-invasive speech BCI. Activation: imagined speech, MEG decoding, brain-computer interface, speech neuroprosthetics
|
Zero-Shot Imagined Speech Decoding via Imagined-to-Listened MEG Mapping
Three-stage pipeline that maps imagined MEG to listened MEG representations, enabling zero-shot imagined speech decoding using listened-only training data.
Metadata
- Source: arXiv:2605.08075
- Authors: Maryam Maghsoudi, Shihab Shamma
- Published: 2026-05-08
- Subjects: cs.LG; eess.AS
Core Methodology
Key Innovation
Imagined speech datasets are scarce and hard to align temporally. This approach circumvents the data scarcity problem by:
- Collecting paired listened + imagined MEG from trained musicians (improved temporal alignment)
- Training models to map imagined → listened MEG representations
- Using the mapping to enable imagined speech decoding via a listened-only decoder
Three-Stage Pipeline
Stage 1: Imagined-to-Listened Mapping
- Train 6 linear and neural models mapping imagined MEG → listened MEG
- Validate against null baseline from unseen subjects
- Verify predicted-listening responses preserve stimulus-specific information
Stage 2: Listened Word Decoder
- Train contrastive word decoder exclusively on listened MEG
- Evaluate with 4 embedding strategies: semantic, acoustic, phonetic, + 1 unspecified
- Rank-based analysis for word identification
Stage 3: Zero-Shot Imagined Decoding
- Process held-out subjects' imagined MEG through mapping pipeline
- Decode via listened decoder using predicted listening responses
- Show imagined words decodable significantly above chance
- Demonstrate scalability with training data size
Implementation Guide
Key Design Choices
- Use trained musicians — rhythmic perception improves temporal alignment across conditions
- Paired recordings — same subjects provide both imagined and listened data for same stimuli
- Cross-subject validation — all evaluations on held-out subjects
- Multiple embeddings — semantic, acoustic, phonetic representations for decoder evaluation
- Null baseline — unseen subjects' data provides chance-level baseline
Model Types
- Linear mappings (baseline)
- Neural network mappings (non-linear)
- Contrastive word decoder (stage 2)
Evaluation
- Rank-based analysis: imagined words ranked above chance
- Training data scaling: performance improves with more data
- Cross-subject generalization: held-out subject evaluation
Applications
- Non-invasive speech BCI for locked-in patients
- Imagined communication interfaces
- Cross-condition neural representation learning
- MEG-based neuroprosthetics
Pitfalls
- Requires paired imagined/listened recordings for training mapping
- Temporal alignment critical — trained musicians help but may limit population
- Proof-of-concept stage — scalability needs validation on larger datasets
- Non-invasive MEG limits spatial resolution vs. iEEG
Related Skills
- brain-to-speech-prosody-feature-engineering
- brain-to-text-unified-decoding
- iphoneme-brain-to-text-als-conformerxl
- meta-learning-in-context-brain-decoding
