krylov-mean-field-chaos-predictability-2026-06-10

name	krylov-mean-field-chaos-predictability-2026-06-10
description	Theoretical framework demonstrating that mean-field chaos in random recurrent networks is predictable from continuous past history
version	1.0.0
tags	["neuroscience","dynamical-systems","chaos","recurrent-networks","mean-field-theory","predictability"]
arxiv_id	2606.08805
date	2026-06-07T00:00:00.000Z
authors	["Alkesh Yadav","Vladimir Shaidurov","Jonathan Kadmon"]

Predictable Mean-Field Chaos in Random Recurrent Networks

Overview

可预测的平均场混沌（Predictable Mean-Field Chaos） 理论框架证明：在具有足够快傅里叶衰减的解析非线性函数的随机循环网络中，平均场理论描述的混沌并非真正的随机过程，而是从连续历史可唯一确定未来的确定性动力学。

Key Contributions

1. Determinism of Mean-Field Chaos

• 关键发现：平均场混沌只是表面的随机性
• 数学证明：连续过去的轨迹唯一确定未来
• 意义：平均场理论不仅是对集合的描述，而是真实的动力学轨迹

2. Fourier Decay Condition

• 提出可预测性的数学条件：快速傅里叶衰减
• 解析非线性函数满足此条件
• 建立可预测性与数学性质的桥梁

3. Predictive Framework

• 开发从历史预测混沌的方法
• 证明预测的唯一性
• 为混沌控制提供理论基础

Mathematical Framework

Random Recurrent Network Model

import numpy as np

def random_recurrent_network(N, g, activation='tanh'):
    """
    Random recurrent network with Gaussian weights
    
    Parameters:
    - N: Number of neurons
    - g: Coupling strength (chaos when g > 1)
    - activation: Nonlinearity (must be analytic for predictability)
    
    Dynamics:
    dx/dt = -x + g * J * f(x)
    
    where:
    - J: Random Gaussian matrix (mean 0, variance 1/N)
    - f: Activation function (analytic)
    """
    # Gaussian random matrix
    J = np.random.randn(N, N) / np.sqrt(N)
    
    # Analytic activation function
    if activation == 'tanh':
        f = lambda x: np.tanh(x)  # Analytic, fast Fourier decay
    elif activation == 'sigmoid':
        f = lambda x: 1 / (1 + np.exp(-x))  # Also analytic
    else:
        raise ValueError("Non-analytic functions not predictable")
    
    return J, f, g

def simulate_chaos(J, f, g, x0, T, dt=0.01):
    """
    Simulate network dynamics
    
    Returns: trajectory x(t)
    """
    N = J.shape[0]
    trajectory = []
    x = x0.copy()
    
    for t in np.arange(0, T, dt):
        dx = -x + g * J.dot(f(x))
        x = x + dt * dx
        trajectory.append(x.copy())
    
    return np.array(trajectory)

Mean-Field Theory

def mean_field_theory(g, f, T_history):
    """
    Mean-field description of chaotic dynamics
    
    Key insight: Mean-field is deterministic, not stochastic!
    
    Equations:
    - C(t, t') = ⟨x(t)x(t')⟩  // Correlation function
    - Evolution: dC/dt = -2C + g²⟨f(x)f(x')⟩
    
    Predictability: Given continuous C(0, T_history), 
                    future C(t > T_history) is uniquely determined
    """
    # Correlation function evolution
    def correlation_evolution(C, t):
        """
        C evolves deterministically
        
        Not a stochastic process!
        """
        dC = -2 * C + g**2 * mean_field_interaction(C, f)
        return dC
    
    return correlation_evolution

Fourier Decay Condition

def check_predictability_condition(activation_func):
    """
    Check if activation function satisfies Fourier decay condition
    
    Condition: |f̂(k)| decays faster than exp(-α|k|) for some α > 0
    
    Examples:
    - tanh: Predictable (analytic)
    - sigmoid: Predictable (analytic)
    - ReLU: NOT predictable (not analytic)
    """
    # Compute Fourier transform
    from scipy.fft import fft
    
    x = np.linspace(-10, 10, 1000)
    f_values = activation_func(x)
    f_hat = fft(f_values)
    
    # Check decay rate
    k = np.arange(len(f_hat))
    decay = np.abs(f_hat)
    
    # Estimate exponential decay rate
    log_decay = np.log(decay[decay > 1e-10])
    slope = np.polyfit(k[:len(log_decay)], log_decay, 1)[0]
    
    alpha = -slope  # Decay rate
    
    is_predictable = alpha > 0.5  # Threshold for fast decay
    
    return is_predictable, alpha

Proof of Predictability

Main Theorem

Theorem: For analytic nonlinearities with fast Fourier decay,
        the mean-field trajectory is uniquely determined by 
        its continuous history.

Proof outline:
1. Mean-field dynamics: dC/dt = F(C)  // Deterministic ODE
2. Analyticity ensures F is smooth
3. Uniqueness theorem for ODEs
4. Given C(0, T), future C(t > T) is unique

Mathematical Derivation

def prove_uniqueness(C_history, f, g):
    """
    Prove uniqueness of future trajectory
    
    Given:
    - C_history: Correlation function from t=0 to T
    - f: Analytic activation
    - g: Coupling strength
    
    Result:
    - C_future: Unique correlation function for t > T
    """
    # Mean-field equation is deterministic ODE
    # dC/dt = -2C + g² * ⟨f(x) f(x')⟩
    
    # Because f is analytic, ⟨f(x)f(x')⟩ can be computed 
    # from C using Wick's theorem and moments
    
    # ODE uniqueness theorem applies
    # Solution is unique given initial condition
    
    # Therefore: C(t > T) is uniquely determined by C(0, T)
    
    return "Future trajectory is unique!"

Implications

1. For Neuroscience

def neuroscience_implications():
    """
    Predictable chaos implications for brain dynamics
    
    Key points:
    1. Brain chaos is not pure randomness
    2. Past activity determines future
    3. Chaos can be controlled
    """
    implications = {
        'predictability': 'Neural chaos is deterministic',
        'control': 'Chaos can be steered by external inputs',
        'memory': 'Past activity shapes future dynamics',
        'stability': 'Boundaries of chaotic regime are predictable'
    }
    return implications

2. For Machine Learning

• RNN设计：理解循环网络的混沌边界
• 初始化策略：避免不可预测的混沌区域
• 训练稳定性：基于可预测性优化学习

3. For Dynamical Systems Theory

Revolution in understanding:
- Chaos ≠ Randomness
- Mean-field ≠ Ensemble average only
- Deterministic chaos is predictable
- Statistical physics connects to individual trajectories

Implementation Guidelines

Step 1: Network Simulation

import numpy as np
from scipy.integrate import odeint

class PredictableChaoticRNN:
    """
    RNN with predictable mean-field chaos
    
    Requirements:
    - Analytic activation (tanh, sigmoid)
    - g > 1 for chaos
    """
    def __init__(self, N, g, activation='tanh'):
        self.N = N
        self.g = g
        self.J = np.random.randn(N, N) / np.sqrt(N)
        
        if activation == 'tanh':
            self.f = np.tanh
        elif activation == 'sigmoid':
            self.f = lambda x: 1/(1 + np.exp(-x))
        
        # Check predictability
        self.is_predictable = True  # Analytic functions
    
    def dynamics(self, x, t):
        """Network dynamics"""
        return -x + self.g * self.J.dot(self.f(x))
    
    def simulate(self, x0, T):
        """Simulate with history"""
        t = np.linspace(0, T, 1000)
        trajectory = odeint(self.dynamics, x0, t)
        return trajectory

Step 2: Mean-Field Prediction

class MeanFieldPredictor:
    """
    Predict future chaos from history
    
    Method: Use correlation function evolution
    """
    def __init__(self, g, f):
        self.g = g
        self.f = f
    
    def compute_correlation(self, trajectory):
        """
        Compute correlation function C(t, t')
        """
        T = trajectory.shape[0]
        C = np.zeros((T, T))
        
        for i in range(T):
            for j in range(T):
                C[i, j] = np.mean(trajectory[i] * trajectory[j])
        
        return C
    
    def predict_future(self, C_history, T_future):
        """
        Predict future correlation from history
        
        Key: C_history uniquely determines C_future
        """
        # Mean-field evolution equation
        def correlation_ode(C, t):
            # Use Wick's theorem for ⟨f(x)f(x')⟩
            moments = self.compute_moments(C)
            interaction = self.mean_field_interaction(moments)
            return -2*C + self.g**2 * interaction
        
        # Solve ODE forward
        C_future = odeint(correlation_ode, C_history[-1], 
                          np.arange(T_future))
        
        return C_future
    
    def ():
        
        
        
        variance = C
         {: variance}

Step 3: Predictability Testing

def test_predictability(network, T_history=50, T_test=20):
    """
    Test if chaos is predictable
    
    Method:
    1. Simulate network twice with same initial condition
    2. Compute correlation functions
    3. Check if predictions match
    """
    # Simulate two trajectories
    x0 = np.random.randn(network.N)
    
    traj1 = network.simulate(x0, T_history + T_test)
    traj2 = network.simulate(x0, T_history + T_test)
    
    # Compare correlations
    C1 = compute_correlation(traj1)
    C2 = compute_correlation(traj2)
    
    # For predictable chaos, correlations should match
    error = np.mean(np.abs(C1 - C2))
    
    is_predictable = error < 0.1
    
    return is_predictable, error

Validation Methods

1. Mathematical Proof Verification

• 检验傅里叶衰减条件
• 验证解析性要求
• 确认ODE唯一性定理适用

2. Numerical Simulation

def validate_theorem_numerically(N=1000, g=1.5, T=100):
    """
    Numerically validate predictability theorem
    
    Steps:
    1. Simulate random RNN with tanh
    2. Compute correlation function
    3. Predict from history
    4. Compare with actual simulation
    """
    network = PredictableChaoticRNN(N, g)
    
    # Simulate full trajectory
    x0 = np.random.randn(N)
    full_traj = network.simulate(x0, T)
    
    # Compute full correlation
    C_full = compute_correlation(full_traj)
    
    # Use only history (0 to T_history)
    T_history = 50
    C_history = C_full[:T_history]
    
    # Predict future correlation
    predictor = MeanFieldPredictor(g, np.tanh)
    C_predicted = predictor.predict_future(C_history, T - T_history)
    
    # Compare
    C_actual = C_full[T_history:]
    error = np.mean(np.abs(C_predicted - C_actual))
    
    print(f"Prediction error: {error}")
    print(f"Predictability confirmed: {error < 0.05}")

3. Analytic vs Non-Analytic Comparison

def compare_analytic_nonanalytic():
    """
    Show analytic functions are predictable,
    non-analytic are not
    
    Example:
    - tanh (analytic): Predictable
    - ReLU (non-analytic): Not predictable
    """
    # Analytic (tanh)
    network_tanh = PredictableChaoticRNN(500, 1.5, 'tanh')
    predictable_tanh, error_tanh = test_predictability(network_tanh)
    
    print(f"tanh: predictable={predictable_tanh}, error={error_tanh}")
    
    # Non-analytic (ReLU) - would fail predictability test
    # network_relu = PredictableChaoticRNN(500, 1.5, 'relu')
    # predictable_relu, error_relu = test_predictability(network_relu)
    # print(f"ReLU: predictable={predictable_relu}, error={error_relu}")

Applications

1. RNN Training Stability

def stabilize_rnn_training(network):
    """
    Use predictability to stabilize training
    
    Strategy:
    - Avoid crossing chaotic boundary unpredictably
    - Monitor correlation function evolution
    - Adjust g to stay in predictable regime
    """
    # Monitor correlation
    trajectory = network.simulate(x0, 100)
    C = compute_correlation(trajectory)
    
    # Check if correlation evolution is smooth
    if not is_smooth_evolution(C):
        # Reduce coupling to avoid unpredictable chaos
        network.g *= 0.95
    
    return network

2. Chaos Control

def control_chaos(network, target_state):
    """
    Control chaotic dynamics to target state
    
    Method: Use external input to steer trajectory
    """
    # Because chaos is predictable, we can compute
    # the input needed to reach target
    
    def dynamics_controlled(x, t, u):
        # Add control input u
        return -x + g * J.dot(f(x)) + u
    
    # Compute control signal
    u = compute_control_for_target(target_state, history)
    
    return u

3. Neural Dynamics Analysis

• 分析大脑中的混沌是否可预测
• 验证神经网络的混沌边界
• 开发基于预测的干预策略

Related Concepts

• Dynamical Mean-Field Theory (DMFT)：平均场动力学理论
• Chaotic Dynamics：混沌动力学基础
• RNN Theory：循环神经网络理论
• Gaussian Process：高斯过程与混沌
• Sompolinsky-Crisanti-Sommers Model：SCS混沌模型

Future Directions

1. 扩展非线性函数：研究更多解析函数的可预测性
2. 有限尺寸效应：有限神经元数量的修正
3. 非高斯权重：扩展到其他权重分布
4. 应用开发：基于可预测性的混沌控制算法

Key Equations

Mean-Field Dynamics

dC(t,t')/dt = -C(t,t') + g²⟨f(x_i(t))f(x_j(t'))⟩

where:
- C(t,t'): Correlation function
- g: Coupling strength
- f: Analytic activation

Fourier Decay Condition

|f̂(k)| ~ exp(-α|k|)  for some α > 0

Analytic functions satisfy this condition

Predictability Theorem

Given: C(t,t') for t,t' ∈ [0, T_history]
Result: C(t,t') for t,t' > T_history is unique

Proof: Mean-field equation is deterministic ODE,
       uniqueness theorem applies

Practical Tips

1. 选择激活函数：

   • 使用 tanh、sigmoid 等解析函数
   • 避免 ReLU、leaky ReLU 等非解析函数

2. 监控混沌边界：

   • g ≈ 1 是混沌起始点
   • g > 1 进入混沌，但可预测（如果函数解析）

3. 利用预测能力：

   • 从历史轨迹预测未来
   • 用于控制和优化
   • 设计稳定的训练策略

References

• Yadav et al. (2026) - Original Paper
• Sompolinsky et al. (1988) - Chaotic RNN Theory
• Kadmon & Sompolinsky (2016) - Mean-Field Dynamics
• ODE Uniqueness Theory (Standard Math)

Activation

Trigger Keywords:

• mean-field chaos
• recurrent network
• chaos predictability
• dynamical systems
• RNN theory
• chaotic dynamics
• neural chaos
• dynamical mean-field theory

Use Cases:

• 分析循环网络的混沌行为
• 设计稳定的 RNN 架构
• 研究神经系统的混沌边界
• 开发混沌控制方法
• 验证动力学理论预测