| name | mt-direction-maps-spatiotemporal |
| description | Spatiotemporal TDANN for modeling self-organized MT direction selectivity maps in the dorsal stream. Uses 3D ResNet with Momentum Contrast (MoCo) self-supervised learning and biological spatial loss to produce direction-selective pinwheel structures matching macaque MT physiology. Use when modeling cortical topographic self-organization, dorsal stream computation, direction selectivity, or spatiotemporal contrastive learning for visual neuroscience. arXiv: 2605.11718 (q-bio.NC, cs.AI, cs.NE). Gu, Li, Su, Liu, Qian, Wang.
|
Spatiotemporal TDANN for MT Direction Maps
3D ResNet with MoCo self-supervised learning + spatial loss produces brain-like direction maps
and topological pinwheel structures in MT area, matching in vivo macaque physiology.
Metadata
- Source: arXiv:2605.11718
- Authors: Zhaotian Gu, Molan Li, Jie Su, Chang Liu, Tianyi Qian, Dahui Wang
- Published: 2026-05-12
- Subjects: q-bio.NC, cs.AI, cs.NE
Core Problem
While TDANN has successfully modeled ventral stream topography (e.g., IT cortex), the computational
origins of dorsal stream topographies — particularly direction-selective maps in MT (middle temporal)
area — remained unresolved. This work unifies ventral and dorsal stream origins under one mechanism.
Key Innovation
Spatiotemporal TDANN: Extends Topographic Deep Artificial Neural Network to 3D (spatiotemporal)
domain with two training objectives:
- Task-driven discriminative pressure: MoCo (Momentum Contrast) self-supervised learning on
naturalistic videos produces motion-direction-selective representations
- Spatial regularization: Biological spatial loss enforces local connectivity patterns
The strict optimization trade-off between these two objectives produces:
- Strong direction selectivity with residual axial component
- Spontaneous emergence of brain-like direction maps
- Topological pinwheel structures matching biology
Technical Framework
Architecture
- 3D ResNet backbone for spatiotemporal feature extraction
- Trained on naturalistic video stimuli
- MoCo self-supervised paradigm (contrastive learning)
- Biologically inspired spatial loss function
Emergent Properties
- Direction-selective maps in MT-like units
- Pinwheel structures with biologically realistic density
- Tuning properties matching in vivo macaque MT recordings:
- Direction selectivity index (DSI)
- Circular variance
- Pinwheel density
Core Mechanism
MT tuning emerges from the balance:
- Discriminative pressure → direction selectivity
- Spatial regularization → topographic organization
- The trade-off produces the characteristic residual axial component of MT neurons
Applications
- Modeling dorsal stream visual processing
- Cortical topographic self-organization research
- Understanding computational origins of direction selectivity
- Neuro-inspired computer vision with biological inductive biases
Related Skills
- self-organized-criticality-brain-body-resonance
- neuroscience-of-transformers
- primary-visual-cortex-v1-functions
- untrained-cnns-match-backprop-v1
