zero-shot-imagined-speech-meg

name

zero-shot-imagined-speech-meg

description

Zero-shot imagined speech decoding from MEG via imagined-to-listened cross-condition mapping. Trains models to map imagined MEG responses to listened responses, then decodes using listened-only decoder. Three-stage pipeline: (1) mapping imagined→listened MEG, (2) train contrastive word decoder on listened MEG with multi-embedding evaluation, (3) decode imagined speech via mapping pipeline on held-out subjects. Use when: imagined speech decoding, MEG BCI, cross-condition neural mapping, zero-shot brain decoding, non-invasive speech BCI. Activation: imagined speech, MEG decoding, brain-computer interface, speech neuroprosthetics

Zero-Shot Imagined Speech Decoding via Imagined-to-Listened MEG Mapping

Three-stage pipeline that maps imagined MEG to listened MEG representations, enabling zero-shot imagined speech decoding using listened-only training data.

Metadata

• Source: arXiv:2605.08075
• Authors: Maryam Maghsoudi, Shihab Shamma
• Published: 2026-05-08
• Subjects: cs.LG; eess.AS

Core Methodology

Key Innovation

Imagined speech datasets are scarce and hard to align temporally. This approach circumvents the data scarcity problem by:

1. Collecting paired listened + imagined MEG from trained musicians (improved temporal alignment)
2. Training models to map imagined → listened MEG representations
3. Using the mapping to enable imagined speech decoding via a listened-only decoder

Three-Stage Pipeline

Stage 1: Imagined-to-Listened Mapping

• Train 6 linear and neural models mapping imagined MEG → listened MEG
• Validate against null baseline from unseen subjects
• Verify predicted-listening responses preserve stimulus-specific information

Stage 2: Listened Word Decoder

• Train contrastive word decoder exclusively on listened MEG
• Evaluate with 4 embedding strategies: semantic, acoustic, phonetic, + 1 unspecified
• Rank-based analysis for word identification

Stage 3: Zero-Shot Imagined Decoding

• Process held-out subjects' imagined MEG through mapping pipeline
• Decode via listened decoder using predicted listening responses
• Show imagined words decodable significantly above chance
• Demonstrate scalability with training data size

Implementation Guide

Key Design Choices

1. Use trained musicians — rhythmic perception improves temporal alignment across conditions
2. Paired recordings — same subjects provide both imagined and listened data for same stimuli
3. Cross-subject validation — all evaluations on held-out subjects
4. Multiple embeddings — semantic, acoustic, phonetic representations for decoder evaluation
5. Null baseline — unseen subjects' data provides chance-level baseline

Model Types

• Linear mappings (baseline)
• Neural network mappings (non-linear)
• Contrastive word decoder (stage 2)

Evaluation

• Rank-based analysis: imagined words ranked above chance
• Training data scaling: performance improves with more data
• Cross-subject generalization: held-out subject evaluation

Applications

• Non-invasive speech BCI for locked-in patients
• Imagined communication interfaces
• Cross-condition neural representation learning
• MEG-based neuroprosthetics

Pitfalls

• Requires paired imagined/listened recordings for training mapping
• Temporal alignment critical — trained musicians help but may limit population
• Proof-of-concept stage — scalability needs validation on larger datasets
• Non-invasive MEG limits spatial resolution vs. iEEG

Related Skills

• brain-to-speech-prosody-feature-engineering
• brain-to-text-unified-decoding
• iphoneme-brain-to-text-als-conformerxl
• meta-learning-in-context-brain-decoding