boosting-brain-to-image-tribe-v2

name	boosting-brain-to-image-tribe-v2
description	TRIBE v2 数据增强提升脑到图像解码性能方法论。使用大规模预训练编码模型生成合成fMRI数据，在小数据集上实现显著性能提升。
version	1.0.0
author	Yohann Benchetrit et al.
arxiv_id	2606.06345
date	2026-06-04T00:00:00.000Z
categories	["neuroscience","brain-decoding","data-augmentation","foundation-models"]
tags	["fMRI","brain-to-image","TRIBE","data-augmentation","zero-shot","neural-decoding"]
activation_keywords	["TRIBE","brain-to-image","fMRI decoding","data augmentation","zero-shot decoding","synthetic data","neural decoding"]

TRIBE v2: Boosting Brain-to-Image Decoding

Overview

TRIBE v2 methodology for boosting brain-to-image decoding through data augmentation using large-scale pretrained encoding models. Addresses the fundamental challenge of limited labeled neural data in brain decoding tasks.

Paper: arXiv:2606.06345 (2026-06-04) Authors: Yohann Benchetrit, Marlène Careil, Simon Dahan, Hubert Banville, Stéphane d'Ascoli, Jean-Rémi King

Core Innovation

Problem

• Brain decoding is limited by availability of labeled neural data
• Low-data regimes remain challenging for decoders
• Traditional methods require extensive training data

Solution

• TRIBE v2: Large encoding model pretrained on >1000 hours of fMRI responses to video, audio, and language
• Generate synthetic fMRI data to augment small datasets
• Systematic evaluation of augmentation ratios across different data sources

Key Results

Performance Improvement

• Up to 68% improvement in Top-10 image-retrieval accuracy
• Tested on:
  • 7T fMRI Natural Scenes Dataset (NSD)
  • 3T fMRI BOLD5000

Zero-Shot Capability

• Surprising finding: Image decoders trained exclusively on synthetic fMRI perform above chance
• TRIBE v2 supports zero-shot brain-to-image decoding

Data-Efficiency

• Proportion of augmented data must be adjusted per data source
• Foundation models provide principled data augmentation strategy

Methodology

Step 1: Pretrained Encoding Model

TRIBE v2 Architecture:
- Input: Video, audio, language stimuli
- Output: fMRI response predictions
- Training data: >1000 hours of multimodal fMRI
- Foundation: Large-scale encoding model

Step 2: Synthetic Data Generation

For each stimulus:
1. Extract visual features
2. Generate synthetic fMRI responses using TRIBE v2
3. Match spatial/temporal characteristics to real data
4. Create augmented training set

Step 3: Decoder Training

Training strategy:
- Mix real + synthetic fMRI data
- Systematic grid search for augmentation ratio
- Evaluate on held-out real data
- Optimize per dataset characteristics

Step 4: Zero-Shot Evaluation

Zero-shot protocol:
- Train decoder on ONLY synthetic data
- Evaluate on real fMRI without real training samples
- Measure retrieval accuracy
- Compare to chance-level performance