brain-dnn-transformation-alignment

name

brain-dnn-transformation-alignment

description

Naturality Violation Score (NVS) for brain-DNN alignment beyond object-level comparison. Uses category theory to test whether brains and DNNs preserve the same transformations among stimuli. Covers approximate naturality, axis-resolved alignment analysis, and hierarchy crossover detection. Use when: (1) evaluating brain-DNN alignment at the transformation level, (2) comparing neural representations across model architectures, (3) analyzing semantic vs low-level visual alignment, or (4) designing neuroscience-inspired model evaluation. Activation: naturality violation score, NVS, brain-DNN alignment, transformation alignment, category theory neuroscience, axis-resolved alignment, hierarchy crossover, approximate naturality, brain model comparison.

Brain-DNN Transformation Alignment (NVS)

Based on Kamitani (2026) — arXiv:2605.06420.

Core Concept

Traditional brain-DNN alignment measures per-stimulus correspondence (encoding accuracy, RSA, CKA). This paper introduces a transformation-level alignment test: do the brain and model preserve the same transformations among stimuli?

Mathematical Framework

Approximate Naturality

Inspired by category theory, alignment is formalized as naturality:

Brain side:    x ──f──► x'
                  │         │
              φ_brain    φ_model
                  ↓         ↓
Model side:   φ(x)─f'──► φ(x')

If the brain and model preserve the same transformation f, the naturality square approximately commutes:

φ_model(f(x)) ≈ f'(φ_brain(x))

Naturality Violation Score (NVS)

NVS quantifies deviation from commutativity, normalized to a permutation null:

NVS(f) = ||φ_model(f(x)) - f'(φ_brain(x))|| / E_perm[||...||]

• NVS ≈ 0: perfect transformation preservation
• NVS ≈ 1: no better than random permutation
• Lower NVS = better alignment for that transformation axis

Key Findings

Hierarchy Crossover

Applied to fMRI (GOD dataset, 5 subjects) + 3 vision DNNs + 3 world-model proxies:

Alignment Axis	Best Aligned To	NVS
Semantic (animacy)	High-level visual cortex + deep DNN layers	0.39
Mid-level visual	Mid visual cortex + mid DNN layers	< next-best
Low-level visual	Early visual cortex + shallow layers	< next-best

This reveals that alignment is axis-selective rather than uniform — different transformation families align at different depths.

Dissociation from Traditional Metrics

• NVS captures alignment failures that RSA/CKA cannot resolve
• Encoding/decoding accuracy does not predict NVS
• Alignment is selective over candidate morphism families

Implementation Workflow

Step 1: Define Proxy Space

Choose a proxy space W and comparison maps:

• Proxy: semantic embeddings (e.g., CLIP, Word2Vec)
• Maps: morphisms between stimuli (e.g., animacy gradient, pose rotation)

Step 2: Compute NVS Per Axis

For each transformation axis f:

1. Apply f to stimuli on brain side → get brain responses
2. Apply f to stimuli on model side → get model activations
3. Measure commutativity deviation
4. Normalize against permutation null distribution

Step 3: Axis-Resolved Analysis

• Compare NVS across multiple axes (semantic, spatial, temporal)
• Identify which brain regions align with which model layers per axis
• Detect hierarchy crossovers (different axes peak at different depths)

Practical Applications

Model Selection

Use NVS to choose DNN architectures that best preserve brain-relevant transformations for a specific task domain.

Layer-wise Alignment

Map which DNN layers correspond to which brain areas for different transformation types — useful for neuroscientific interpretability.

Proxy Space Design

The method generalizes to any proxy space (world models, symbolic representations, etc.), enabling richer brain-model comparisons.

Comparison with Existing Methods

Method	Granularity	Transformation-aware?	Null model
Encoding accuracy	Per-stimulus	No	None
RSA	Geometry	Partial	None
CKA	Global similarity	No	None
NVS (this work)	Per-axis	Yes	Permutation null

Common Pitfalls

• Proxy space quality: NVS results depend on proxy choice; poor proxies yield misleading NVS values
• Transformation definition: candidate morphisms must be well-defined and meaningful for both brain and model domains
• Permutation null: ensure sufficient permutations (≥1000) for stable normalization
• W-less control: always run anchor-ablation to verify alignment isn't driven by trivial correlations

Activation Keywords

• naturality violation score, NVS, brain-DNN transformation alignment, approximate naturality, axis-resolved alignment, hierarchy crossover, category theory neuroscience, brain model comparison, transformation preservation, semantic alignment