TRIBE v2 tri-modal foundation model for in-situ fMRI brain-to-image decoding with synthetic data augmentation. Addresses low-data regime challenges in brain decoding.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
TRIBE v2 tri-modal foundation model for in-situ fMRI brain-to-image decoding with synthetic data augmentation. Addresses low-data regime challenges in brain decoding.
TRIBE v2: Tri-modal Foundation Model for Brain Decoding
Overview
TRIBE v2 is a large encoding model pretrained on more than 1000 hours of fMRI responses to video, audio, and language stimuli. It enables synthetic data augmentation for boosting brain-to-image decoding in low-data regimes.
Key Contribution: Addresses the fundamental limitation in brain decoding — availability of labeled neural data — by using pretrained foundation models to generate synthetic fMRI data for augmentation.
Core Methodology
1. Tri-modal Foundation Model Architecture
Pretraining: 1000+ hours of fMRI responses to:
Video stimuli
Audio stimuli
Language stimuli
Encoding model: Maps stimuli → fMRI responses
Synthetic generation: Given new stimuli, generate predicted fMRI responses
2. Data Augmentation Strategy
Grid-based evaluation:
Systematic grids showing how augmentation effectiveness varies with:
Amount of synthetic data added
Quality threshold for synthetic samples
Domain mismatch between pretrained model and target dataset
Training strategy (augment vs. pretrain)
3. Low-Data Regime Applications
When augmentation helps:
Small fMRI datasets (< 50 subjects)
Limited labeled stimuli
Novel stimulus types not in pretraining
Cross-subject generalization
When augmentation is less effective:
Large datasets (> 500 subjects)
Domain mismatch without adaptation
Very different stimulus modalities
Technical Details
Synthetic Data Generation Pipeline
Input Stimuli → TRIBE v2 Encoder → Predicted fMRI → Quality Filter → Augmented Training Set
Quality filtering criteria:
Prediction confidence threshold
Activation pattern similarity to real data
Region-of-interest consistency
Training Strategies
Option A: Direct Augmentation
# Mix real and synthetic fMRI data
augmented_dataset = real_fMRI + synthetic_fMRI[quality > threshold]
model.train(augmented_dataset)
Option B: Pretrain-then-Finetune
# Use TRIBE v2 as pretrained backbone
pretrained_encoder = TRIBE_v2.load()
finetuned_decoder = pretrained_encoder.adapt(target_dataset)
Brain-to-Image Decoding
Traditional limitation:
Requires large fMRI-to-image paired datasets
Subject-specific training expensive
Limited stimulus diversity
TRIBE v2 solution:
Generate synthetic fMRI for any image/video stimulus
Expand training set without additional scanning
Enable zero-shot or few-shot decoding for novel stimuli
Use Cases
1. Visual Reconstruction from fMRI
Problem: Reconstruct viewed images from brain activity
TRIBE v2 approach: Augment with synthetic fMRI generated from image dataset
2. Cross-subject Transfer
Problem: Train decoder on few subjects, generalize to new subjects
Solution: Use TRIBE v2 pretrained representations as shared basis
3. Novel Stimulus Modalities
Problem: Decode brain responses to stimuli not in training set
Approach: Generate synthetic fMRI for new stimuli using TRIBE v2 encoder
Related: Brain-DiT, NeuroSTORM, fMRI foundation models
Activation: Use this skill when working on brain decoding, fMRI data augmentation, foundation models for neuroscience, or low-data regime brain-to-image reconstruction.
