cocot-eeg-contrastive-foundation

name	cocot-eeg-contrastive-foundation
description	Contrastive pretraining methodology for EEG foundation models using multiscale convolutional Transformer architecture. Demonstrates contrastive learning as a superior alternative to masked reconstruction pretraining for EEG, which has high noise and narrow-band information. Achieves SOTA on heterogeneous electrode configurations. Activation: EEG contrastive learning, CoCoT, EEG foundation model, masked reconstruction, multiscale temporal convolution, self-supervised EEG, electrode heterogeneous, EEGFM, EEG decoding, LaBraM alternative, REVE alternative
tags	["eeg","foundation-model","contrastive-learning","self-supervised","neuroscience","deep-learning"]
metadata	{"arxiv_id":"2607.09543","published":"2026-07-10","authors":"Gabriel Mahuas, Victoria Shevchenko, Ugo Tanielian, Yassir Bendou, Richard Gao","conference":"arXiv"}

CoCoT-EEG: Contrastive-Pretrained Multiscale Convolutional Transformer for EEG Decoding

Core Innovation

CoCoT-EEG demonstrates that contrastive learning is a superior pretraining strategy for EEG foundation models compared to the dominant masked reconstruction (MAE) approach. The key insight: EEG has high noise amplitude and information confined to limited dimensions (narrow frequency bands), making reconstruction-based objectives suboptimal.

Why Reconstruction Fails for EEG

1. High noise amplitude — Masked autoencoders waste capacity reconstructing noise
2. Narrow-band information — Only specific frequency bands carry task-relevant signal
3. Heterogeneous electrode configurations — Different datasets have different montages, making token-level reconstruction hard to generalize

Architecture

Multiscale Temporal Convolution Input Layers

• Parallel convolutions at multiple temporal scales capture different frequency bands
• Acts as a learnable filter bank replacing hand-crafted spectral features
• Each scale focuses on a different temporal resolution

Transformer Encoder Blocks

• Standard Transformer architecture applied after multiscale convolution
• Processes the filtered representations for downstream task learning
• Benefits from both local (convolution) and global (attention) processing

Pretraining Strategy

Contrastive Learning for EEG

• Positive pairs: augmented views of same EEG segment
• Negative pairs: different EEG segments
• Forces model to learn invariant, task-relevant representations
• Avoids the noise-reconstruction trap of MAE

EEG-Specific Augmentations

• Time warping, channel dropout, frequency masking
• Preserves task-relevant structure while creating diverse views

Key Results

• CoCoT pretrained matches or beats reconstruction-pretrained SOTA models (LaBraM, CBraMod, CSBrain, LUNA, REVE)
• CoCoT from scratch outperforms previous single-task decoding models and rivals pretrained models
• Works well with heterogeneous electrode configurations — no montage-specific tuning needed
• Demonstrates the viability of contrastive learning as a general EEG FM strategy

Comparison: Contrastive vs Reconstruction Pretraining

Aspect	Reconstruction (MAE)	Contrastive (CoCoT)
Noise sensitivity	High (reconstructs noise)	Low (learns invariance)
Band efficiency	Poor (all bands equal)	Good (task-relevant only)
Electrode heterogeneity	Poor	Good
Data efficiency	Requires massive data	Works from scratch
Architecture flexibility	Fixed tokenization	Multiscale convolution

Applications

• EEG foundation model pretraining
• Brain-computer interface (BCI) decoding
• Cross-dataset EEG generalization
• Clinical EEG analysis with limited labeled data
• Heterogeneous electrode montage handling

Pitfalls

1. Augmentation design is critical: Poor augmentations destroy task-relevant EEG structure. Must preserve frequency-band information that downstream tasks need.
2. Negative sample selection: Random negatives may not be hard enough for EEG where segments from same subject/session are similar.
3. Multiscale filter design: Number and size of convolutional scales should match the frequency bands of interest for the target domain.
4. Not a silver bullet: While contrastive outperforms MAE on average, the gap varies by task — motor imagery may still benefit from reconstruction.
5. From-scratch performance: While CoCoT from scratch rivals pretrained models on some tasks, large-scale pretraining still provides the best cross-domain generalization.

Related Skills

• reve-eeg-foundation — REVE: EEG FM with 4D positional encoding (reconstruction-based)
• eeg-channel-adaptation-benchmark — Channel adaptation for EEG FMs
• eeg-test-time-adaptation-benchmark — TTA benchmark for EEG FMs
• cross-subject-eeg-decoding — Cross-subject EEG decoding survey
• kast-brain-autoregressive — KAST-BAR: EEG foundation with brain topology

References

• arXiv: 2607.09543 (2026-07-10)
• Authors: Mahuas, Shevchenko, Tanielian, Bendou, Gao
• Affiliations: Sigma Nova (Paris), Goethe University Frankfurt