Free-probability framework for analyzing stationary covariance spectra in non-normal random recurrent neural networks. Derives closed functional equations for moment generating functions and analyzes tail eigenvalue behavior in critical regimes. arXiv:2606.31944
Standardmäßig ist der Prompt ausgewählt, der zuerst die Quelle prüft. Sie können zu einem direkten Befehl wechseln oder eine lokale Kopie herunterladen.
Quelldateien prüfen
Lesen Sie SKILL.md und alle von SkillsMP angezeigten Begleitdateien, bevor Sie sich für eine Installation entscheiden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
Free-probability framework for analyzing stationary covariance spectra in non-normal random recurrent neural networks. Derives closed functional equations for moment generating functions and analyzes tail eigenvalue behavior in critical regimes. arXiv:2606.31944
Stationary Covariance Spectra of Discrete-Time Non-Normal Random Recurrent Dynamics
Overview
This skill implements theoretical and computational frameworks for analyzing stationary covariance spectra in random recurrent neural networks using free-probability theory. The methodology provides closed-form functional equations for moment generating functions and characterizes tail eigenvalue behavior in critical regimes.
Key Contributions:
Closed functional equation for stationary covariance spectrum moment generating function
Analysis of tail eigenvalues in critical regime
Comparison between discrete-time and continuous-time dynamics
Free-probability approach vs infinite Schwinger-Dyson hierarchy
Core Methodology
1. Problem Setup
Consider discrete-time random recurrent dynamics:
x[t+1] = W · φ(x[t]) + ε[t]
where:
W = random non-normal Gaussian weight matrix (variance g²/N)
φ = element-wise nonlinear activation
ε = stationary noise process
N = network size
Stationary covariance matrix:
C = lim_{T→∞} (1/T) Σ_{t=1}^T (x[t] - μ)(x[t] - μ)^T
2. Free-Probability Framework
Moment generating function:
M(z) = Σ_{k=0}^∞ m_k z^{-k}
where m_k = (1/N) Tr(C^k) are normalized moments of covariance spectrum.
Closed functional equation (discrete-time):
M(z) = F(M(z), z, g, σ)
where:
g = spectral radius parameter (g < 1 for stability)
σ = noise variance
F = function derived via free-probability tools
Key insight: Discrete-time dynamics yield a closed scalar equation, unlike continuous-time which produces infinite Schwinger-Dyson hierarchy.
3. Critical Regime Analysis
At critical point g → 1⁻:
Tail eigenvalues exhibit power-law scaling
Covariance spectrum becomes heavy-tailed
Network operates at edge of chaos
Critical scaling:
λ_max ~ N^{1/2} · (1-g)^{-α}
where α depends on activation function statistics.
4. Implementation
4.1 Numerical Verification
import numpy as np
from scipy.linalg import sqrtm
defsimulate_covariance_spectrum(N=1000, g=0.95, T=10000, phi=np.tanh):
"""
Simulate discrete-time random recurrent network and compute covariance spectrum.
Args:
N: network size
g: spectral radius parameter
T: simulation timesteps
phi: activation function
Returns:
eigenvalues: sorted eigenvalues of covariance matrix
"""# Random non-normal weight matrix
W = np.random.randn(N, N) * g / np.sqrt(N)
# Simulate dynamics
x = np.random.randn(N)
trajectory = []
for t inrange(T):
x = phi(W @ x) + 0.01 * np.random.randn(N)
trajectory.append(x)
# Compute covariance matrix
X = np.array(trajectory)
C = np.cov(X.T)
# Eigenvalue spectrum
eigenvalues = np.linalg.eigvalsh(C)
eigenvalues = np.sort(eigenvalues)[::-1]
return eigenvalues
defcompute_moment_generating_function(eigenvalues, z_values):
"""
Compute empirical moment generating function M(z) = Σ m_k z^{-k}
Args:
eigenvalues: covariance spectrum eigenvalues
z_values: complex z values
Returns:
M_z: moment generating function values
"""
N = len(eigenvalues)
moments = np.array([np.sum(eigenvalues**k) / N for k inrange(1, 20)])
M_z = np.zeros(len(z_values), dtype=complex)
for i, z inenumerate(z_values):
M_z[i] = np.sum([m * z**(-k-1) for k, m inenumerate(moments)])
return M_z
4.2 Critical Scaling Analysis
defanalyze_critical_scaling(g_values, N_values, num_trials=10):
"""
Analyze how maximum eigenvalue scales near critical point g → 1.
Args:
g_values: array of g values (e.g., np.linspace(0.5, 0.99, 20))
N_values: array of network sizes
num_trials: number of trials per parameter
Returns:
scaling_exponent: fitted critical exponent α
"""
lambda_max = np.zeros((len(g_values), len(N_values), num_trials))
for i, g inenumerate(g_values):
for j, N inenumerate(N_values):
for trial inrange(num_trials):
eigs = simulate_covariance_spectrum(N=N, g=g)
lambda_max[i, j, trial] = eigs[0]
# Average over trials
lambda_max_mean = lambda_max.mean(axis=2)
# Fit power law: λ_max ~ (1-g)^{-α}# Use log-linear regression
log_lambda = np.log(lambda_max_mean[:, 0])
log_delta = np.log(1 - g_values)
# Linear fit
coeffs = np.polyfit(log_delta, log_lambda, 1)
scaling_exponent = -coeffs[0]
return scaling_exponent
4.3 Free-Probability Theoretical Prediction
deffree_probability_prediction(g, sigma, phi_deriv_squared):
"""
Compute theoretical moment generating function via free-probability.
Args:
g: spectral radius parameter
sigma: noise variance
phi_deriv_squared: average squared derivative of activation
Returns:
M_theory: theoretical moment generating function
"""# This requires solving the closed functional equation# M = F(M, z, g, σ) iteratively# Simplified version for Marchenko-Pastur regime# Full implementation requires numerical root findingdeffixed_point_iteration(M_guess, z, g, sigma, max_iter=100):
"""Iterate to find self-consistent solution."""
M = M_guess
for _ inrange(max_iter):
# Self-consistency equation (simplified)
M_new = sigma * z**(-1) * (1 + g**2 * phi_deriv_squared * M)
if np.abs(M_new - M) < 1e-8:
break
M = M_new
return M
return fixed_point_iteration
Experimental Validation
Test 1: Marchenko-Pastur Limit (g=0)
When g=0, network reduces to noise:
x[t] = ε[t]
Covariance spectrum should follow Marchenko-Pastur law:
defmarchenko_pastur_density(sigma, N, T):
"""
Marchenko-Pastur density for white noise covariance.
λ± = σ²(1 ± √(N/T))²
"""
lambda_plus = sigma**2 * (1 + np.sqrt(N/T))**2
lambda_minus = sigma**2 * (1 - np.sqrt(N/T))**2return lambda_minus, lambda_plus
Test 2: Critical Scaling
As g → 1, verify:
Tail eigenvalues grow as power law
Covariance spectrum becomes heavy-tailed
Network approaches edge of chaos
Test 3: Discrete vs Continuous Comparison
Compare discrete-time (closed equation) with continuous-time (infinite hierarchy):
