kinetic-energy-random-rnn-chaos

## Core Discovery Kinetic energy of neural activity provides quantitative link between: - **Chaotic dynamics**: High-dimensional neural fluctuations - **Unstable fixed points**: Equilibria in phase space Key finding: Average kinetic energy shifts from zero to positive at critical coupling variance with **cubic scaling behavior** near critical point. ## Technical Framework ### Dynamical Mean-Field Theory (DMFT) Analytical framework for large random RNNs: - Self-consistent equations for neural activity statistics - Tracks kinetic energy evolution $E_k = \langle \dot{x}_i^2 \rangle$ - Connects microscopic dynamics to macroscopic observables ### Critical Behavior Characterization At critical synaptic gain $\sigma_c$: - Kinetic energy: $E_k \sim (\sigma - \sigma_c)^3$ for $\sigma > \sigma_c$ - Cubic scaling verified by simulations - Continuous transition from ordered to chaotic regime ### Geometric Analysis Phase space structure: - Chaotic dynamics arranged in **shell-like structure** - Gradient dynamics of kinetic energy form concentric shells - Original dynamics and gradient dynamics **separated in polar direction** ## Physical Interpretation ### Kinetic Energy as Chaos Quantifier Two roles: 1. **Distance from equilibrium**: Measures how far chaotic orbit is from unstable fixed points 2. **Change rate**: Characterizes how fast dynamics evolve during chaos onset ### Shell Structure Geometry - Chaotic manifold: Thin shell in phase space - Gradient flow: Concentric shells around shell center - Polar separation: Chaotic dynamics and gradient dynamics orthogonal components ### Trajectory Length Analysis Trajectory length on chaotic manifold: $$L = \int_0^T \|\dot{x}(t)\| dt = T \sqrt{E_k}$$ Derived from stationary kinetic energy, provides geometric characterization. ## Numerical Validation ### Scaling Behavior Verification Simulations on finite-size systems confirm: - Cubic scaling near critical point - Kinetic energy matches DMFT predictions - Finite-size corrections negligible for $N > 1000$ ### Activity Distribution Analysis Steady-state distribution from theory: - Gaussian-like profile from kinetic energy optimization - Shell-like structure confirmed by geometric analysis - Polar coordinates reveal chaotic-gradient separation ## Connection to Neural Computation ### Chaos in Recurrent Networks Random RNNs model biological circuits: - Chaos emergence at critical connectivity - Kinetic energy quantifies computational regime - Fixed points as potential computation states ### Distance from Equilibria Kinetic energy measures: - Stability of fixed points (unstable in chaotic regime) - Exploration capacity (high $E_k$ = large exploration) - Information processing (chaos encodes variability) ### Scaling Law Implications Cubic scaling predicts: - Sharp transition to chaos (sensitive to synaptic gain) - Robustness margin around critical point - Transition rate affects computational dynamics ## Mathematical Details ### Random RNN Model $$\dot{x}_i = -x_i + \sum_j J_{ij} \phi(x_j) + \eta_i$$ where $J_{ij} \sim \mathcal{N}(0, \sigma^2/N)$, $\phi$ activation, $\eta_i$ input. ### Kinetic Energy Definition $$E_k = \frac{1}{N} \sum_{i=1}^N \dot{x}_i^2$$ Average squared velocity of neural activity. ### DMFT Equations Self-consistent equations: - Activity statistics: $\langle x^2 \rangle$, $\langle \dot{x}^2 \rangle$ - Correlation functions: $C(t) = \langle x(t)x(0) \rangle$ - Response functions: $R(t) = \langle x(t)\eta(0) \rangle$ Solved numerically to extract kinetic energy scaling. ## Broader Implications ### Chaos-Computation Relationship - **Chaos onset**: Transition to exploratory dynamics - **Kinetic energy**: Quantifies exploration rate - **Fixed points**: Computation anchors in phase space ### Scaling Universality Cubic scaling may apply to: - Other random network architectures - Different activation functions - Networks with structured connectivity ### Geometric Interpretation Shell structure suggests: - Chaotic orbits confined to thin manifold - Gradient dynamics explores manifold interior - Polar separation enables dual perspective ## Cross-Domain Applications ### Neural Network Training Dynamics Kinetic energy concept generalizes to: - Training dynamics in deep networks - Gradient flow geometry - Loss landscape exploration ### Biological Neural Circuits Random connectivity in: - Cortical microcircuits (chaos computation hypothesis) - Hippocampal networks (memory encoding via variability) - Sensory processing (noise-driven exploration) ### Complex Systems Physics Kinetic energy framework applies to: - Spin glass dynamics - Disordered systems - Critical phenomena in high dimensions ## Methodological Patterns ### DMFT for Large Networks Pattern applicable when: - Network size $N \to \infty$ - Connectivity random with known statistics - Want self-consistent statistics without simulation ### Kinetic Energy as Dynamical Quantifier Use kinetic energy when: - Analyzing chaos onset in dynamical systems - Quantifying distance from equilibria - Characterizing phase space geometry ### Geometric Phase Space Analysis Shell structure analysis for: - High-dimensional dynamical systems - Chaotic attractor characterization - Gradient flow vs. dynamics separation ## Pitfalls ### Finite-Size Effects - Theory valid for $N \to \infty$ - Finite networks show corrections - Critical behavior smeared for small $N$ ### Activation Function Dependence - Cubic scaling verified for specific $\phi$ - Different activations may change scaling exponent - Need numerical verification per activation type ### Input Noise Effects - External noise $\eta_i$ affects kinetic energy - Input-driven vs. internally-generated chaos distinction - Noise can suppress or enhance kinetic energy ## Related Work Connections to: - **Sompolinsky et al. (1988)**: Original chaos in random RNNs - **Rajan et al. (2010)**: Fixed point structure in chaos - **Kinzel (1983)**: DMFT for neural networks This work quantifies chaos-structure relationship via kinetic energy. ## Key Takeaways 1. **Kinetic energy quantifies chaos**: Links dynamics to fixed point distance 2. **Cubic scaling**: Sharp critical transition with universal exponent 3. **Shell geometry**: Chaotic manifold has thin shell structure 4. **Polar separation**: Dynamics and gradient flow orthogonal 5. **Trajectory length**: Derives from kinetic energy, geometric characterization 6. **DMFT validation**: Theory matches simulations for large networks ## Metadata - **arXiv ID**: 2508.04983 - **Authors**: Li-Ru Zhang, Haiping Huang - **Categories**: cond-mat.stat-mech, nlin.CD, q-bio.NC - **Published**: 2025-08-07 (v1), 2026-06-02 (v3) - **Journal**: Physical Review E (revised)