cortico-cerebellar-modularity-rnn

name

cortico-cerebellar-modularity-rnn

description

Cortico-cerebellar modular RNN architecture methodology. Augments RNNs with cerebellar-inspired feedforward modules for efficient temporal learning. The cortical RNN acts as a fixed reservoir while the cerebellar module drives learning efficiency. Applicable to temporal sequence learning, neural network architecture design, and brain-inspired AI systems. Activation: cortico-cerebellar, cerebellar RNN, CB-RNN, cortical-cerebellar, modular RNN, temporal learning architecture, brain-inspired RNN, fixed reservoir, heterogeneous modularity, cerebellar module, architectural inductive bias

Cortico-Cerebellar Modular RNN Architecture

Based on: Voce, Giannakakis & Clopath (2026) arXiv:2605.10356

Core Finding

Augmenting an RNN with a cerebellar-inspired feedforward module (CB-RNN) enables faster learning and higher performance than fully recurrent baselines. After minimal training of the recurrent core, freezing it and delegating subsequent learning to the cerebellar module preserves efficiency.

Architecture

Input → [Cortical RNN (frozen reservoir)] → [Cerebellar Feedforward Module] → Output

• Cortical RNN: Recurrent core that processes temporal context, trained briefly then frozen as a fixed reservoir
• Cerebellar Module: Feedforward module that receives cortical representations and performs the primary adaptive learning

Key Principles

1. Heterogeneous Modularity: Different module types serve distinct computational roles
2. Fixed Reservoir: Cortical RNN need not be fully trained; frozen weights still provide rich temporal representations
3. Delegated Learning: Cerebellar module absorbs subsequent learning, enabling rapid adaptation without destabilizing core representations
4. Structural Inductive Bias: Architecture itself encodes priors that accelerate learning

Implementation Pattern

import torch
import torch.nn as nn

class CerebellarModule(nn.Module):
    """Feedforward module mimicking cerebellar learning."""
    def __init__(self, input_dim, hidden_dim, output_dim):
        super().__init__()
        self.fc1 = nn.Linear(input_dim, hidden_dim)
        self.fc2 = nn.Linear(hidden_dim, output_dim)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        return self.fc2(self.relu(self.fc1(x)))

class CorticalRNN(nn.Module):
    """Recurrent cortical core (frozen after warmup)."""
    def __init__(self, input_dim, hidden_dim):
        super().__init__()
        self.rnn = nn.RNN(input_dim, hidden_dim, batch_first=True)
    
    def forward(self, x, h0=None):
        return self.rnn(x, h0)

class CBRNN(nn.Module):
    """Cortico-cerebellar RNN architecture."""
    def __init__(self, input_dim, cortical_dim, cerebellar_dim, output_dim):
        super().__init__()
        self.cortex = CorticalRNN(input_dim, cortical_dim)
        self.cerebellum = CerebellarModule(cortical_dim, cerebellar_dim, output_dim)
    
    def forward(self, x, warmup=False):
        # Cortical processing
        cortex_out, h_n = self.cortex(x)
        # Cerebellar readout
        output = self.cerebellum(cortex_out)
        return output, h_n
    
    def freeze_cortex(self):
        """Freeze cortical weights after warmup phase."""
        for param in self.cortex.parameters():
            param.requires_grad = False

# Usage:
# 1. Warmup: train both cortex and cerebellum for N epochs
# 2. Freeze: model.freeze_cortex()
# 3. Continue: train only cerebellar module

Training Protocol

1. Warmup Phase: Train full CB-RNN on target task (few epochs)
2. Freeze Cortex: Set requires_grad=False on cortical RNN parameters
3. Cerebellar Learning: Continue training with only cerebellar module gradients

Advantages Over Baselines

• Faster convergence: Cerebellar module adapts more rapidly than full RNN retraining
• Higher performance: Surpasses parameter-matched fully recurrent networks
• Stability: Freezing core prevents catastrophic forgetting during adaptation
• Energy efficiency: Fewer trainable parameters during deployment phase

Applications

• Temporal sequence prediction
• Continuous learning scenarios
• Brain-inspired neural architectures
• Robotics control with temporal dependencies
• Speech and language processing

Related Skills

• spiking-bandpass-wavelet-encoding - Spiking temporal encoding
• working-memory-heterogeneous-delays - Working memory in SNNs
• brain-inspired-snn-pattern-analysis - Brain-inspired computing patterns

ArXiv Reference

• Paper: arXiv:2605.10356v1
• Title: Cortico-cerebellar modularity as an architectural inductive bias for efficient temporal learning
• Authors: Alexandra Voce, Emmanouil Giannakakis, Claudia Clopath
• Date: 2026-05-11
• Categories: q-bio.NC