visual-cortex-diffusion-inference

name	visual-cortex-diffusion-inference
description	Skill for understanding and applying the mechanistic model of perceptual inference in visual cortex equivalent to a minimal diffusion model (arXiv:2607.15693). Enables extraction of principles linking sparse coding, recurrent dynamics, and diffusion model training for neuroscience-inspired machine learning.

Visual Cortex Diffusion Inference Mechanism

Core Methodology

1. Sparse Coding Base: Start with standard sparse coding model where latent variables represent sparse features of natural images.
2. Non-Factorial Prior: Introduce an unconstrained, pairwise interaction matrix between latent variables, replacing the typical factorial prior. This couples latent variables and enables recurrent dynamics.
3. Recurrent Dynamical System: The inference becomes a recurrent dynamical system where latent states evolve via gradient descent on an energy function defined by the sparse coding term plus the interaction term.
4. Training Objective: Use denoising score-matching on corrupted images, coupled with implicit differentiation to efficiently learn the interaction matrix gradients.
5. Learned Interaction Structure: After training on natural images, the symmetric interaction matrix learned mirrors the anatomical connectivity of horizontal connections in superficial layers of V1, linking neurons with similar orientation tuning.
6. Denoising Performance: The resulting model achieves denoising performance comparable to standard diffusion models, especially in restoring extended contours under high visual ambiguity.
7. Jacobian Decomposition: Because the model is simple and energy-based, the Jacobian of the inference dynamics can be analytically decomposed into contributions from the interaction matrix, revealing how recurrent dynamics assign probability to continuous structural deformations (e.g., contour continuations).
8. Latent Disconnection & Hierarchy: A subset of learned latent units decouple from direct visual input, forming a latent hierarchy that enforces global consistency across image features (e.g., contour closure, texture consistency).
9. Dual‑Domain Insight:
   • Neuroscience: Generates testable hypotheses about functional connectivity in recurrent V1 circuits during perceptual inference.
   • Machine Learning: Provides an interpretable mechanistic explanation for how diffusion models learn rich priors enabling infinite sample generation from finite data.

Implementation Steps

1. Define Latent Variables: Let (z \in \mathbb{R}^N) be sparse code coefficients for an image patch.
2. Energy Function: [ E(x, z) = \frac{1}{2}|x - D z|^2 + \frac{1}{2} z^\top W z - b^\top z ] where (D) is dictionary, (W) symmetric interaction matrix (zero diag), (b) bias.
3. Inference Dynamics: [ \tau \dot{z} = -\nabla_z E = D^\top (x - Dz) - W z + b ] (gradient descent on energy).
4. Learning:
   • Corrupt input (x \rightarrow \tilde{x} = x + \epsilon).
   • Train (W, b, D) to minimize denoising score matching: (\mathbb{E}{\tilde{x}}[|\nabla{\tilde{x}} \log p_\theta(\tilde{x}) - \nabla_{\tilde{x}} \log q(\tilde{x}|x)|^2]) via implicit differentiation through the fixed‑point of the dynamics.
5. Symmetry Constraint: Enforce (W = W^\top) during training to ensure energy‑based interpretation.
6. Analysis:
   • Compare learned (W) with anatomical connectivity matrices from V1 histology.
   • Compute Jacobian (J = \partial \dot{z}/\partial z = -D^\top D - W) and inspect eigenvectors for modes corresponding to contour integration.
   • Measure fraction of units with low input‑weight norm ((|D_{i,:}|)) to quantify disentangled latent subset.

Pitfalls

• Symmetry Enforcement: Forgetting to symmetrize (W) breaks the energy‑function interpretation and leads to unstable dynamics.
• Learning Rate Coupling: The interaction matrix learns slowly; use separate learning rates or preconditioning.
• Diagonal Entries: Keep diagonal of (W) zero (or absorb into biases) to avoid double‑counting self‑interaction.
• Initialization: Initialize (W) small and symmetric (e.g., zero) to start from standard sparse coding.
• Convergence Criteria: The recurrent dynamics must reach a fixed point for each stimulus; use sufficient integration steps or Anderson acceleration.

Verification

• Verify that the symmetric learned (W) reproduces known V1 horizontal connection patterns (e.g., orientation‑specific lateral connections).
• Confirm denoising PSNR/SSIM on natural image benchmarks matches or approaches that of standard DDPM‑style diffusion models.
• Check that Jacobian eigenvectors correspond to intuitively meaningful image transformations (e.g., curve extensions, texture filling).
• Ablate the interaction term (set (W=0)) and observe degradation to baseline sparse coding performance.

Activation Keywords

visual cortex inference, diffusion model, sparse coding, recurrent dynamics, denoising score matching, interaction matrix, V1 horizontal connections, Jacobian decomposition, latent hierarchy, mechanistic interpretation, neuroscience‑ML bridge