efficient-coding-criticality-sloppiness

name	efficient-coding-criticality-sloppiness
description	Efficient coding under resource constraints drives neural systems towards criticality and sloppiness. Links Fisher information maximization to power-law distributions and critical brain hypothesis.
triggers	["efficient coding","neural criticality","brain criticality","sloppiness","Fisher information neural","power-law neural","critical brain","neural avalanche","Gaussian population coding","soft modes neural"]
category	neuroscience
tags	["criticality","efficient-coding","Fisher-information","sloppiness","neural-population-coding","brain-criticality","power-law"]
source	arXiv:2605.22598
authors	["He Xiao","Xinyue Zhao","Weikang Wang"]

Efficient Coding Under Constraint → Neural Criticality & Sloppiness

Overview

This skill encapsulates the theoretical framework from arXiv:2605.22598 showing that maximizing Fisher information under resource constraints naturally leads neural systems to operate near criticality — unifying efficient coding theory with the critical brain hypothesis.

Key insight: The brain's critical state is not accidental but is a functional consequence of information-theoretic optimization under metabolic/resource constraints.

Core Methodology

1. Gaussian Population Coding Model

• Represent neural population responses as Gaussian distributions
• Define Fisher information as the metric for coding efficiency
• Introduce resource constraints (energy, metabolic cost, channel capacity)
• Maximize Fisher information subject to these constraints

2. Emergence of Criticality

Under constrained Fisher information maximization:

• Soft modes emerge: response directions with near-zero eigenvalues
• Diverging correlation lengths: long-range correlations characteristic of criticality
• Power-law distributions: neural avalanche statistics naturally follow power laws
• This recapitulates both statistical criticality and dynamical criticality

3. Unifying Two Criticality Perspectives

Perspective	Mechanism	Observable
Statistical criticality	Diverging correlation length	Power-law spatial correlations
Dynamical criticality	Critical slowing down + bifurcation	Slow timescales, bifurcation point

The spatial structure in the model bridges these two views.

4. Sloppiness as Emergent Property

• Sloppiness: neural systems have highly variable parameter sensitivities — some parameters matter enormously, others barely at all
• This framework shows sloppiness is a natural consequence of efficient coding: optimized systems develop stiff and sloppy parameter combinations automatically
• Sloppy directions ≈ soft modes ≈ directions that don't cost Fisher information

Implementation Steps

Step 1: Build the Population Coding Model

import numpy as np
from scipy.linalg import eigh

# N neurons, d stimulus dimensions
N, d = 100, 2

# Tuning curves: f_i(s) = exp(-||s - mu_i||^2 / (2*sigma^2))
mu = np.random.randn(N, d)  # preferred stimuli
sigma = 1.0

def tuning_curve(s, mu, sigma):
    """Gaussian tuning curves for N neurons"""
    diffs = s - mu  # (N, d)
    return np.exp(-np.sum(diffs**2, axis=1) / (2 * sigma**2))

Step 2: Compute Fisher Information Matrix

def fisher_information(s, mu, sigma, noise_cov):
    """
    Compute Fisher Information Matrix at stimulus s.
    J(s) = (df/ds)^T * Sigma^{-1} * (df/ds)
    """
    f = tuning_curve(s, mu, sigma)
    # Gradient of tuning curves
    df_ds = -f[:, None] * (s - mu) / sigma**2  # (N, d)
    
    noise_inv = np.linalg.inv(noise_cov)
    J = df_ds.T @ noise_inv @ df_ds
    return J

# Total Fisher info (scalar): trace or determinant
def total_fisher(J):
    return np.trace(J)  # or np.linalg.det(J)

Step 3: Optimize Under Resource Constraint

from scipy.optimize import minimize

def constrained_fisher_maximization(N, d, resource_budget):
    """
    Maximize sum of Fisher information subject to:
    - Metabolic cost: sum(f_i) <= budget
    - Normalization constraints
    """
    def objective(params):
        mu = params[:N*d].reshape(N, d)
        sigma = params[N*d]
        # Compute expected Fisher info over stimulus space
        J_total = compute_expected_fisher(mu, sigma)
        return -J_total  # minimize negative = maximize
    
    def constraint_cost(params):
        mu = params[:N*d].reshape(N, d)
        sigma = params[N*d]
        avg_rate = compute_average_rate(mu, sigma)
        return resource_budget - avg_rate  # >= 0
    
    result = minimize(objective, x0, 
                      constraints={'type': 'ineq', 'fun': constraint_cost},
                      method='SLSQP')
    return result

Step 4: Analyze Eigenspectrum for Criticality

def analyze_criticality(J_fisher):
    """
    Check for criticality signatures in Fisher Information Matrix.
    Criticality: eigenvalue spectrum follows power law.
    """
    eigenvalues, eigenvectors = eigh(J_fisher)
    eigenvalues = np.sort(eigenvalues)[::-1]
    
    # Check power-law fit
    log_rank = np.log(np.arange(1, len(eigenvalues)+1))
    log_eig = np.log(eigenvalues + 1e-10)
    
    # Fit power law: log(lambda) ~ -alpha * log(rank)
    from numpy.polynomial import polynomial as P
    coeffs = np.polyfit(log_rank, log_eig, 1)
    alpha = -coeffs[0]
    
    return {
        'eigenvalues': eigenvalues,
        'power_law_exponent': alpha,
        'is_critical': abs(alpha - 1.0) < 0.3,  # ~power law with exponent ~1
        'num_soft_modes': np.sum(eigenvalues < 0.01 * eigenvalues[0])
    }

Step 5: Measure Sloppiness

def sloppiness_index(eigenvalues):
    """
    Sloppiness: large ratio between largest and smallest eigenvalues.
    Sloppy if lambda_max / lambda_min >> 1.
    """
    sorted_eig = np.sort(np.abs(eigenvalues))[::-1]
    ratio = sorted_eig[0] / (sorted_eig[-1] + 1e-10)
    
    # Sloppiness score: number of decades spanned
    import math
    decades = math.log10(ratio)
    return {
        'eigenvalue_ratio': ratio,
        'decades': decades,
        'is_sloppy': decades > 4  # sloppy if > 4 orders of magnitude
    }

Key Results

1. Fisher info maximization → Power-law eigenspectrum (criticality signature)
2. Resource constraints (metabolic budget) are necessary — unconstrained optimization does not produce criticality
3. Spatial structure bridges statistical and dynamical criticality perspectives
4. Sloppiness emerges automatically: stiff directions encode task-relevant information; sloppy directions are near-null modes

Practical Applications

Neural Data Analysis

def analyze_neural_population(spike_rates, stimulus_conditions):
    """
    Test if a recorded neural population shows critical signatures.
    1. Estimate Fisher information from data
    2. Check eigenspectrum for power-law
    3. Quantify sloppiness
    """
    # Estimate noise covariance
    noise_cov = np.cov(spike_rates.T)
    
    # Compute Fisher info at each stimulus
    J_list = []
    for s in stimulus_conditions:
        J = estimate_fisher_from_data(spike_rates, s)
        J_list.append(J)
    
    J_avg = np.mean(J_list, axis=0)
    criticality = analyze_criticality(J_avg)
    sloppiness = sloppiness_index(np.linalg.eigvalsh(J_avg))
    
    return criticality, sloppiness

Neural Network Design

• Use as regularization: penalize deviation from critical eigenspectrum
• Initialize network weights to produce soft modes
• Use sloppiness as a training diagnostic

Connections to Existing Theory

Concept	Connection
Critical Brain Hypothesis	Derived from first principles here
Maximum Entropy Principle	Fisher information maximization is dual
Edge of Chaos	Critical point = edge of bifurcation
Free Energy Principle	Resource-constrained inference
Neural Manifold Hypothesis	Soft modes = low-dimensional manifold

Pitfalls

• Resource constraint form matters: different cost functions lead to different critical regimes
• Finite-size effects: criticality is approximate in finite populations
• Noise model sensitivity: Gaussian noise assumption may not hold for all neural systems
• Power-law fitting requires sufficient dynamic range (at least 2-3 decades)

Citation

@article{xiao2026efficient,
  title={Efficient coding under constraint drives neural systems towards criticality and sloppiness},
  author={Xiao, He and Zhao, Xinyue and Wang, Weikang},
  journal={arXiv:2605.22598},
  year={2026}
}