| name | non-normal-covariance-spectra-rnn |
| description | Free-probability approach to stationary covariance spectra of discrete-time non-normal random recurrent dynamics - derives closed functional equation for moment generating function of limiting stationary covariance spectrum, analyzes tail eigenvalue behavior in critical regime, and shows continuous-time analog leads to infinite Schwinger-Dyson hierarchy instead of closed scalar equation |
| tags | ["recurrent-neural-networks","random-matrix-theory","free-probability","covariance-spectrum","non-normal-dynamics","stationary-covariance","neural-data-analysis","critical-regime","schwinger-dyson-equations","pca-analysis"] |
Stationary Covariance Spectra of Non-Normal Random Recurrent Dynamics
arXiv: 2606.31944
Authors: Jacob A. Zavatone-Veth
Published: 2026-06-30
Categories: q-bio.NC, cond-mat.dis-nn
Core Contribution
This paper uses free probability theory to formally derive a closed functional equation for the moment generating function of the limiting stationary covariance spectrum of discrete-time random recurrent neural networks with non-normal Gaussian weights. This allows analysis of tail eigenvalue behavior in the critical regime. Crucially, the analogous continuous-time dynamics leads to an infinite hierarchy of Schwinger-Dyson equations rather than a closed scalar equation.
Theoretical Framework
Problem Statement
- PCA is widely used to characterize RNN dynamics structure
- For stationary noise-driven dynamics, variance distribution among principal components is determined by the stationary covariance matrix spectrum
- Spectral properties well-understood for normal synaptic weight matrices
- Non-normal dynamics (biologically realistic) remain poorly understood
Key Mathematical Results
Discrete-Time Dynamics (Closed Form)
For discrete-time dynamics with random non-normal Gaussian weights:
x_{t+1} = W * x_t + ξ_t
where W is non-normal Gaussian and ξ_t is noise.
Result: Closed functional equation for moment generating function M(z) of limiting stationary covariance spectrum:
M(z) = f(M(z), z, spectral_parameters)
This allows analysis of tail eigenvalues in the critical regime.
Continuous-Time Dynamics (Infinite Hierarchy)
For continuous-time analog:
dx/dt = -x + W * x + ξ(t)
Result: Same free-probability approach yields an infinite hierarchy of Schwinger-Dyson equations, not a closed scalar equation. This represents a fundamental asymmetry between discrete and continuous formulations.
Critical Regime Analysis
The closed form enables analysis of tail eigenvalue behavior when the network operates near criticality:
- Tail eigenvalues determine slow modes and long-timescale dynamics
- Non-normality amplifies certain directions in state space
- Critical regime shows power-law-like eigenvalue distributions
Methodology
Free Probability Approach
- Random matrix limit: Consider N → ∞ limit of weight matrices
- Free probability tools: Use R-transform and S-transform for non-normal ensembles
- Moment generating function: Derive functional equation for covariance spectrum
- Tail analysis: Extract asymptotic behavior of extreme eigenvalues
Comparison: Discrete vs Continuous
| Property | Discrete-Time | Continuous-Time |
|---|
| Closed form | ✓ Yes | ✗ No |
| Equation type | Scalar functional | Infinite hierarchy (Schwinger-Dyson) |
| Tail analysis | Direct | Requires truncation/approximation |
| Critical regime | Analytically tractable | More complex |
Applications
- Neural data analysis: Comparing non-normal RNN models to recorded neural data
- PCA interpretation: Understanding variance distribution in neural population recordings
- Critical dynamics: Analyzing networks operating near critical points
- Model selection: Choosing between discrete and continuous formulations based on analytical tractability
- Dimensionality reduction: Understanding which directions in state space carry most variance
Relevance to Neural Data
The paper concludes with comments on comparing non-normal dynamics models to neural data:
- Non-normal dynamics produce asymmetric variance distributions
- Tail eigenvalues correspond to slow, behaviorally-relevant modes
- Free-probability predictions can be tested against population recordings
- Discrete-time models may be more appropriate for certain analytical questions
Comparison to Related Work
| Approach | Non-Normal Handling | Closed Form | Critical Regime | Neural Data Comparison |
|---|
| Normal matrix theory | ✗ Assumes normal | Yes | Limited | Poor fit |
| Numerical simulation | ✓ But no theory | No | Empirical only | Direct |
| This work (free probability) | ✓ Rigorous | Yes (discrete) | Analytical | Testable predictions |
Pitfalls
- Infinite-dimensional limit: Results assume N → ∞; finite-N corrections may be significant
- Gaussian assumption: Weight matrices assumed Gaussian; structured weights may behave differently
- Stationarity requirement: Analysis assumes stationary noise-driven dynamics
- Discrete vs continuous gap: The fundamental difference between discrete and continuous formulations means results don't directly transfer
- Schwinger-Dyson truncation: For continuous-time, any practical analysis requires truncating the infinite hierarchy
Activation Keywords
non-normal dynamics, stationary covariance spectrum, free probability, random recurrent networks, moment generating function, critical regime, Schwinger-Dyson equations, PCA analysis, neural population dynamics, tail eigenvalues, discrete-time RNN, continuous-time RNN, variance distribution
