This skill encodes expert methodological knowledge for multivariate neural decoding analyses in systems neuroscience. It covers cross-validated classification (MVPA), representational similarity analysis (RSA), temporal generalization, and encoding models. The skill provides domain-specific decision logic, parameter recommendations, and pitfall warnings that a machine-learning engineer without neuroscience training would not know.
When to Use This Skill
Determining whether stimulus or task information is represented in neural population activity
Comparing the representational geometry of brain regions to computational models
Characterizing the temporal dynamics of neural representations from EEG/MEG
Building encoding models to predict neural responses from stimulus features
Designing a decoding analysis pipeline and choosing appropriate methods and parameters
Research Planning Protocol
Before executing the domain-specific steps below, you MUST:
State the research question — What specific representational or informational question is this decoding analysis addressing?
Justify the method choice — Why decoding/RSA (not univariate analysis, connectivity, etc.)? What alternatives were considered?
Declare expected outcomes — What decoding accuracy or representational structure would support vs. refute the hypothesis?
Note assumptions and limitations — What does this method assume? Where could it mislead (e.g., confounds, leakage)?
Present the plan to the user and WAIT for confirmation before proceeding.
For detailed methodology guidance, see the research-literacy skill.
⚠️ Verification Notice
This skill was generated by AI from academic literature. All parameters, thresholds, and citations require independent verification before use in research. If you find errors, please open an issue.
When to Use Decoding vs. Univariate Analysis
Univariate analysis tests whether the mean activity level differs across conditions in a region. Decoding tests whether spatial patterns of activity carry information, even when mean activity is identical across conditions (Haynes, 2015). Use decoding when:
You expect information to be encoded in distributed patterns, not mean amplitude
The signal-to-noise ratio per voxel/channel is low but the population carries information
You want to compare neural representations to computational model predictions (use RSA)
You want to track when information emerges and transforms over time (use temporal generalization)
Domain judgment: High decoding accuracy does NOT mean the decoded region is the source of the representation. It means the information is accessible from that region's patterns. A downstream region receiving a copy of the signal will also decode well (Haynes, 2015).
Method Selection Decision Tree
What is your research question?
|
+-- "Is stimulus/task information present in this brain region's patterns?"
| --> Cross-validated classification (MVPA)
| Output: classification accuracy or d-prime
|
+-- "How are representations organized? Does the geometry match a model?"
| --> Representational Similarity Analysis (RSA)
| Output: model-RDM correlation, noise ceiling
|
+-- "When does information emerge and how does it transform over time?"
| --> Temporal Generalization (time x time decoding)
| Output: temporal generalization matrix
| Best for: EEG, MEG, intracranial recordings
|
+-- "What stimulus features drive neural responses across the feature space?"
--> Encoding Models (voxelwise/channel-wise prediction)
Output: prediction accuracy (R^2), feature tuning maps
Cross-Validated Classification (MVPA)
Classifier Selection
Classifier
When to Use
When to Avoid
Source
Linear SVM
Default choice; robust to high dimensionality; works well with small samples
When you need probabilistic outputs (use logistic regression)
Misaki et al., 2010; Varoquaux et al., 2017
LDA
Fast; good when n_features << n_samples after reduction
Raw high-dimensional data (covariance estimate unstable)
Misaki et al., 2010
Logistic Regression
When you need class probabilities; with L1 for sparse solutions
Rarely a bad choice; comparable to linear SVM
Varoquaux et al., 2017
Linear kernel (general)
Almost always for fMRI/EEG
Nonlinear kernels rarely improve and risk overfitting
Misaki et al., 2010
Domain judgment: Linear classifiers are strongly preferred in neuroimaging because (1) fMRI/EEG patterns are high-dimensional relative to sample size, making nonlinear methods prone to overfitting, and (2) linear weights are more interpretable neurally, though see Haufe et al. (2014) on the distinction between classifier weights and activation patterns.
Cross-Validation Strategy
Strategy
When to Use
Rationale
Leave-one-run-out
fMRI (standard)
Respects temporal autocorrelation within runs; prevents leakage from slow hemodynamic signals (Varoquaux et al., 2017)
Stratified k-fold (k=5-10)
EEG/MEG with many trials
Balances class proportions in each fold; k=5 recommended for bias-variance tradeoff (Varoquaux, 2018)
Leave-one-trial-out
When few trials available
Maximum training data but high variance; avoid for fMRI due to temporal autocorrelation (Varoquaux et al., 2017)
Leave-one-subject-out
Between-subject generalization
Tests whether patterns generalize across individuals
CRITICAL -- Information leakage: Feature selection, normalization, and dimensionality reduction MUST be performed WITHIN each cross-validation fold, using ONLY training data. Fitting a PCA or z-scoring across all data before splitting inflates accuracy by leaking test-set statistics into training (Kriegeskorte et al., 2009; Varoquaux et al., 2017).
Chance Level and Statistical Testing
Theoretical chance: 1/n_classes for balanced designs (e.g., 50% for 2-class)
CRITICAL: Theoretical chance is only valid with infinite samples. With small samples, empirical accuracy on random data can substantially exceed 1/n_classes (Combrisson & Jerbi, 2015)
Use permutation testing: Shuffle labels 1000+ times, decode each permutation, compute p-value as the proportion of permuted accuracies >= observed accuracy (Combrisson & Jerbi, 2015)
Binomial test: Acceptable quick alternative for large trial counts, but permutation testing is preferred (Combrisson & Jerbi, 2015)
For group-level inference: test accuracy against chance across subjects using a one-sample t-test or Wilcoxon signed-rank test on subject-level accuracies
Minimum Data Requirements
Trials per class: Minimum 20-30 trials per class for reliable within-subject decoding (Varoquaux, 2018; Grootswagers et al., 2017)
Cross-validation error bars: With ~100 samples, expect confidence intervals of approximately +/-10% on accuracy estimates (Varoquaux, 2018)
Trial averaging: Averaging 5-10 trials before classification improves SNR but reduces effective sample size; balance based on total trial count (Grootswagers et al., 2017)
Representational Similarity Analysis (RSA)
RSA abstracts from activity patterns to a condition-by-condition dissimilarity matrix (RDM), enabling comparison across brain regions, species, and computational models (Kriegeskorte et al., 2008).
RDM Construction
Distance Metric
Properties
When to Use
Source
Correlation distance (1 - Pearson r)
Invariant to mean and scale
Default for comparing pattern shape; standard in early RSA
Kriegeskorte et al., 2008
Euclidean distance
Sensitive to amplitude
When amplitude differences are meaningful
Kriegeskorte et al., 2008
Crossnobis distance
Cross-validated Mahalanobis; unbiased estimator with interpretable zero
Preferred for inferential statistics; requires multi-run data
Walther et al., 2016; Kriegeskorte & Diedrichsen, 2019
Domain judgment: The crossnobis estimator is unbiased -- its expected value is zero when two conditions have identical representations, unlike correlation distance or Euclidean distance which are positively biased by noise. This means crossnobis values can be negative (not a true distance), but this property makes it valid for statistical inference without bias correction (Walther et al., 2016).
Model Comparison
Pearson/Spearman correlation: Correlate model RDM with brain RDM (use Spearman for robustness to outliers; Nili et al., 2014)
Partial correlation: Control for one model while testing another (essential when models are correlated)
Regression on RDMs: Fit multiple model RDMs simultaneously; use weighted least squares or component models (Kriegeskorte & Diedrichsen, 2019)
Statistical Inference
Noise ceiling: Upper and lower bounds on the best achievable model fit given between-subject variability. Upper bound: average correlation of each subject's RDM with the group mean RDM. Lower bound: same but computed with leave-one-subject-out (Nili et al., 2014)
Stimulus-label randomization: Permute condition labels to construct a null distribution for RDM correlation (Nili et al., 2014)
Bootstrap confidence intervals: Resample subjects with replacement to estimate confidence intervals on model-RDM correlations
Domain judgment: If a model falls within the noise ceiling, it explains as much variance as is explainable given the noise in the data. A model below the lower bound leaves systematic variance unexplained. This is NOT the same as a significance test -- a model can be significantly correlated with brain RDMs yet still fall below the noise ceiling (Nili et al., 2014).
See references/rsa-guide.md for a complete step-by-step RSA workflow.
Temporal Generalization (EEG/MEG)
Train a classifier at each time point t, test it at every time point t'. The resulting time x time matrix reveals the dynamics of neural representations (King & Dehaene, 2014).
Interpreting the Temporal Generalization Matrix
Pattern
Matrix Shape
Interpretation
Example
Diagonal only
Thin diagonal stripe
Information is present but the neural code changes over time (chain of transient states)
Relationship to RSA: When using a zero-mean isotropic Gaussian weight prior (ridge regression), encoding models, RSA, and pattern component models test equivalent hypotheses captured by the second moment matrix G (Kriegeskorte & Diedrichsen, 2019)
Common Pitfalls
1. Information Leakage (The Most Common Error)
Feature selection, z-scoring, PCA, or any data-driven preprocessing on the full dataset before cross-validation splitting will leak information from test folds into training, inflating accuracy. ALL such steps must occur WITHIN each fold (Kriegeskorte et al., 2009; Varoquaux et al., 2017).
2. Confounds Driving Decoding
Decoding "success" may reflect confounds rather than neural representations:
Eye movements: Systematic gaze differences between conditions produce decodable EOG artifacts in EEG and BOLD signal differences near the eyes in fMRI
Reaction time differences: Conditions with different RTs produce different motor preparation signals
Head motion: Differential motion across conditions (e.g., speech vs. rest) introduces confounded spatial patterns
Run order: If conditions are blocked or ordered within runs, slow signal drifts can drive classification
3. Interpreting Accuracy Magnitude
Decoding accuracy reflects information accessible to the classifier, not the amount of neural information (Haynes, 2015)
Low accuracy does not mean low information (features may be nonlinearly coded)
High accuracy does not imply the region is the origin of the representation
Never compare accuracy magnitudes across regions with different voxel counts, SNR, or dimensionality without controlling for these factors
4. Double-Dipping in Searchlight Analysis
Selecting an ROI based on significant searchlight clusters and then performing additional analyses on those clusters is circular (Kriegeskorte et al., 2009; Etzel et al., 2013). Use independent data or pre-registered ROIs for follow-up analyses.
5. Classifier Weights Are Not Activation Patterns
Raw SVM or regression weights do NOT indicate which voxels/channels are most activated by a condition. They indicate which features are most useful for discrimination, which can include suppressing noise. To obtain neurophysiologically interpretable maps, transform weights into activation patterns using the method of Haufe et al. (2014).
6. Imbalanced Classes
Unequal trial counts across classes bias accuracy toward the majority class. Solutions:
Undersample the majority class
Use balanced accuracy (average of per-class recall)
Stratify cross-validation folds
Minimum Reporting Checklist
Based on Haynes (2015), Varoquaux et al. (2017), and Grootswagers et al. (2017):
Classifier type and hyperparameters (e.g., linear SVM, C=1)
Feature space (number of voxels/channels, ROI definition or searchlight radius)
Preprocessing steps performed WITHIN vs. OUTSIDE cross-validation folds
Number of trials/samples per class per subject
Statistical test for significance (permutation test with N permutations, or parametric test)
Effect size or confidence intervals on accuracy
For RSA: distance metric, model RDMs tested, noise ceiling
For temporal generalization: window width, step size, cluster-correction parameters
Software and version used
Key References
Combrisson, E., & Jerbi, K. (2015). Exceeding chance level by chance: The caveat of theoretical chance levels in brain signal classification and statistical assessment of decoding accuracy. Journal of Neuroscience Methods, 250, 126-136.
Etzel, J. A., Zacks, J. M., & Braver, T. S. (2013). Searchlight analysis: Promise, pitfalls, and potential. NeuroImage, 78, 261-269.
Grootswagers, T., Wardle, S. G., & Carlson, T. A. (2017). Decoding dynamic brain patterns from evoked responses: A tutorial on multivariate pattern analysis applied to time series neuroimaging data. Journal of Cognitive Neuroscience, 29(4), 677-697.
Haufe, S., et al. (2014). On the interpretation of weight vectors of linear models in multivariate neuroimaging. NeuroImage, 87, 96-110.
Haynes, J.-D. (2015). A primer on pattern-based approaches to fMRI: Principles, pitfalls, and perspectives. Neuron, 87(2), 257-270.
King, J.-R., & Dehaene, S. (2014). Characterizing the dynamics of mental representations: The temporal generalization method. Trends in Cognitive Sciences, 18(4), 203-210.
Kriegeskorte, N., Goebel, R., & Bandettini, P. (2006). Information-based functional brain mapping. Proceedings of the National Academy of Sciences, 103(10), 3863-3868.
Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis -- Connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 4.
Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S., & Baker, C. I. (2009). Circular analysis in systems neuroscience: The dangers of double dipping. Nature Neuroscience, 12(5), 535-540.
Kriegeskorte, N., & Diedrichsen, J. (2019). Peeling the onion of brain representations. Annual Review of Neuroscience, 42, 407-432.
Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG- and MEG-data. Journal of Neuroscience Methods, 164(1), 177-190.
Misaki, M., Kim, Y., Bandettini, P. A., & Kriegeskorte, N. (2010). Comparison of multivariate classifiers and response normalizations for pattern-information fMRI. NeuroImage, 53(1), 103-118.
Nili, H., et al. (2014). A toolbox for representational similarity analysis. PLoS Computational Biology, 10(4), e1003553.
Varoquaux, G., et al. (2017). Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines. NeuroImage, 145, 166-179.
Varoquaux, G. (2018). Cross-validation failure: Small sample sizes lead to large error bars. NeuroImage, 180, 68-77.
Walther, A., et al. (2016). Reliability of dissimilarity measures for multi-voxel pattern analysis. NeuroImage, 137, 188-200.
See references/decoding-methods.md for detailed classifier comparisons, searchlight parameters, and software tools.
See references/rsa-guide.md for a complete step-by-step RSA analysis workflow.
