| name | snn-fmri-visual-decoding |
| version | 1.0.0 |
| description | Spiking Neural Networks for fMRI-Based Visual Semantic Decoding - methodology for using SNN-derived visual features as alternative targets for fMRI-based visual decoding, demonstrating stronger alignment with fMRI responses and improved visual semantic decoding performance compared to ANN-derived features. |
| author | Jiahong Zhang, Jinning Zhao, Sijun Shen, Siyuan Xu, Bo Xu, Guoqi Li |
| license | arXiv.org perpetual non-exclusive license |
| arxiv_id | 2607.1917 |
| date_added | 2026-07-23T00:00:00.000Z |
| categories | ["neuroscience","computational-neuroscience","brain-computer-interface","spiking-neural-networks","fmri-decoding"] |
Spiking Neural Networks for fMRI-Based Visual Semantic Decoding
Overview
This skill implements the methodology from the paper "Spiking Neural Networks for fMRI-Based Visual Semantic Decoding" (arXiv:2607.19170) which demonstrates that SNN-derived visual features provide superior targets for fMRI-based visual decoding compared to conventional ANN-derived features. The key insight is treating the target visual representation as a scientific variable rather than just an engineering choice.
Key Findings
- Stronger fMRI alignment: SNN representations better correspond to measured brain responses than ANN representations
- Improved feature prediction: On GoD dataset, SNN-derived features reduce feature-prediction error from 0.7707 to 0.0282
- Enhanced semantic decoding: Top-1 semantic decoding accuracy improves from 0.1800 to 0.4400 on GoD dataset
- Multiple SNN variants tested: LIF, PSN, MPSN, and BuSNN all show advantages over ANN baseline
- Temporal dynamics matter: Both spiking neural dynamics and temporal simulation steps contribute to observed advantages
Methodology
Problem Formulation
The framework treats fMRI-based visual semantic decoding as mapping brain activity into visual features:
-
Stimulus processing: Same images processed by either ANN or SNN feature extractors
- ANN features: dense and static (ResNet-18 backbone)
- SNN features: spike-based and temporally dynamic (SEW-ResNet-18 variants)
-
Controlled comparison: Same L2-regularized linear fMRI-to-feature decoder used for all models
- Only feature vectors used as regression targets are varied
- Decoder form, input preprocessing, and training protocol kept fixed
-
Evaluation protocol: Multi-level assessment including:
- Voxel-level alignment (PCC analysis)
- Semantic-level alignment
- Semantic classification
- Image retrieval (fMRI-to-image)
- Semantic-guided reconstruction
SNN Variants
Four spiking neuron variants were evaluated under the same SEW-ResNet-18 backbone:
- LIF (Leaky Integrate-and-Fire): Classical membrane integration
- PSN (Parallel Spiking Neuron): Parallel temporal computation
- MPSN (Memory-based Parallel Spiking Neuron): Explicit memory propagation
