boosting-brain-to-image-tribe-v2

name	boosting-brain-to-image-tribe-v2
description	TRIBE v2 data augmentation methodology for brain-to-image decoding. Uses pretrained encoding model on 1000+ hours of video/audio/language fMRI to generate synthetic data, achieving 68% improvement in Top-10 retrieval accuracy. Supports zero-shot decoding when trained exclusively on synthetic fMRI.
metadata	{"arxiv_id":"2606.06345","published":"2026-06-05","authors":"Yohann Benchetrit, Marlène Careil, Simon Dahan, Hubert Banville, Stéphane d'Ascoli, Jean-Rémi King","subjects":"cs.AI, cs.LG, q-bio.NC","key_results":"68% improvement in Top-10 image retrieval accuracy via synthetic data augmentation"}
license	Complete terms in LICENSE.txt

Boosting Brain-to-Image Decoding with TRIBE v2 Data Augmentation

arXiv ID: 2606.06345
Authors: Yohann Benchetrit, Marlène Careil, Simon Dahan, Hubert Banville, Stéphane d'Ascoli, Jean-Rémi King
Subjects: cs.AI, cs.LG, q-bio.NC

Abstract

Brain decoding is limited by the availability of labeled neural data, and remains challenging in low-data regimes. To address this issue, we investigate whether and when brain decoding can be boosted by augmenting small fMRI datasets with synthetic data generated by a pretrained model of fMRI responses to stimuli. We use TRIBE v2, a large encoding model pretrained on more than 1000 hours of fMRI responses to video, audio and language. For each dataset, we evaluate systematic grids that show how the performance of image decoders varies with the amount of synthetic data used for training. Our results, based on two datasets (the 7T fMRI Natural Scenes Dataset and 3T fMRI BOLD5000), show up to 68% improvement in Top-10 image-retrieval accuracy compared to decoders trained only on real data. Importantly, the proportion of augmented data required to reach a given image decoding performance needs to be adjusted depending on the data source. Surprisingly, image decoders trained exclusively on synthetic fMRI can perform above chance in some settings, suggesting that TRIBE v2 can support zero-shot brain-to-image decoding. Together, these results show how large-scale models of the fMRI responses to sight, sound and language may provide a foundation to improve the data efficiency for image decoding.

Core Methodology

Key Results

1. 68% improvement in Top-10 image retrieval accuracy via synthetic data augmentation
2. TRIBE v2 pretrained on 1000+ hours of video/audio/language fMRI
3. Zero-shot brain-to-image decoding possible with synthetic-only training
4. Augmentation ratio varies by data source (NSD vs BOLD5000)
5. Foundation model approach for data-efficient image decoding

Implementation Steps

1. Paper Analysis: Extract methodology from full paper (PDF/HTML)
2. Method Validation: Verify key claims against experimental data
3. Reproducibility Check: Identify required datasets/tools
4. Integration: Connect to existing neuroscience skills/workflows

Experimental Setup

• Datasets: NSD (7T fMRI), BOLD5000 (3T fMRI) for image decoding
• Metrics: Top-10 retrieval accuracy, zero-shot performance
• Baselines: Raw FC, pretrained BFMs (BrainLM, etc.)

Pitfalls

• Data Source Dependency: Augmentation ratio must be tuned per dataset
• Variance Allocation: BFMs may destroy third-order statistics critical for cognition
• Scale Paradox: Larger models may perform worse (BrainLM 650M < 111M)

Verification

• Check Top-10 retrieval accuracy improvement (>50%)
• Verify zero-shot decoding above chance
• Compare BFM vs raw FC performance gap

Related Skills

• brain-foundation-model-batch-effects - BFM batch effect analysis
• brain-dit-fmri-foundation-model - Brain-DiT foundation model
• cross-scale-spatial-generative-neurodegenerative - Generative modeling

Activation Keywords

TRIBE, brain-to-image, image decoding, fMRI augmentation, synthetic data, zero-shot decoding, foundation model, encoding model