mpcs-neuroplastic-continual-learning

name

mpcs-neuroplastic-continual-learning

description

Multi-Plasticity Continual System (MPCS) integrating 11 neuroplastic mechanisms for continual learning. Key finding: EWC regularization degrades performance at high task similarity. Pareto frontier analysis for model compression. Activates: continual learning, neuroplastic architecture, EWC regularization, plasticity-stability tradeoff, MEP-BENCH, multi-component learning, task-driven neurogenesis, topology-aware EWC.

MPCS: Multi-Plasticity Continual Learning System

Integrates eleven complementary neuroplastic mechanisms for continual learning, evaluated on MEP-BENCH with three-dimensional Pareto criterion (performance, representation diversity, gradient conflict).

Metadata

• Source: arXiv:2605.02509
• Authors: Joern Hentsch
• Published: 2026-05-04
• Categories: cs.LG, cs.NE

Core Methodology

Key Innovation

Comprehensive neuroplastic continual learning architecture integrating 11 mechanisms with systematic ablation study using three-dimensional Pareto frontier analysis (Perf, RD, GCR). Critical finding: EWC regularization is counterproductive at high task similarity (s_bar ≈ 0.95), establishing monotone relationship: global EWC < topology EWC < no EWC.

The 11 Components

#	Component	Role
1	Task-Driven Neurogenesis	Add new neurons for novel tasks
2	Fourier-Encoded Inputs	Frequency-based input representation (most critical — removal drops Perf by 30.7 pp)
3	EWC Regularization	Elastic weight consolidation (found counterproductive)
4	Meta-Replay	Replay-based consolidation
5	Mixed Consolidation	Combined consolidation strategies
6	Hybrid Gating	Dynamic routing between components
7	Synapse Pruning/Regeneration	Dynamic connectivity adaptation
8	Hebbian Updates	Local plasticity rules
9	Task Similarity Routing	Route based on task similarity
10	Adaptive Growth Control	Regulate network expansion
11	Continuous Neuron Importance Tracking	Monitor and rank neuron contributions

Benchmark: MEP-BENCH

• 31 tasks across regression, classification, logic, and mixed domains
• Three-dimensional Pareto criterion: Performance (Perf), Representation Diversity (RD), Gradient Conflict Rate (GCR)
• 15 ablation configurations: 3 seeds × 4 tracks × 2000 epochs
• Normalized Efficiency Score (NES): Composite metric for ranking

Critical Findings

1. Fourier encoding is essential: Single most critical component; removal drops performance by 30.7 percentage points and fails MEP gate on 14% of tasks

2. EWC is counterproductive at high task similarity:

   • Global EWC: NES = -4.2 (fails)
   • Topology-local EWC: NES = 91.8 (better but not best)
   • No EWC: NES = 90.5 (MPCS_EFFICIENT achieves highest Perf)
   • Monotone relationship: global EWC < topology EWC < no EWC (at s_bar ≈ 0.95)

3. Pareto frontier predicts model compression:

   • Removing two Pareto-dominated components (EWC + Hebbian) jointly yields MPCS_EFFICIENT
   • Improves Perf by 0.6 pp at 4.7× lower compute cost (127 vs. 602 min)
   • Pareto status assessment is actionable for model compression

Implementation Guide

Architecture Design

class MPCS(nn.Module):
    def __init__(self, input_dim, hidden_dim, task_dim):
        super().__init__()
        # Fourier encoding (critical)
        self.fourier_encoder = FourierEncoding(input_dim, num_frequencies=64)
        # Hybrid gating
        self.gate = HybridGating(task_dim, hidden_dim)
        # Neurogenesis module
        self.neurogenesis = TaskDrivenNeurogenesis(hidden_dim)
        # Topology-local EWC (optional)
        self.ewc = TopologyLocalEWC()
        # Meta-replay buffer
        self.replay = MetaReplayBuffer()
        # Hebbian update module
        self.hebbian = HebbianUpdater()
        # Pruning/regeneration
        self.pruning = SynapsePruningRegeneration()
        # Importance tracking
        self.importance = NeuronImportanceTracker()
    
    def forward(self, x, task_id, replay=False):
        x = self.fourier_encoder(x)  # Critical step
        x = self.gate(x, task_id)
        # ... routing through components based on task similarity
        return output

EWC Decision Rule

def should_use_ewc(task_similarity, ewc_type="topology"):
    """
    Decision rule based on MPCS findings:
    At high task similarity (s_bar >= 0.95):
    - Skip EWC entirely for best performance
    - At moderate similarity, topology-local EWC may help
    """
    if task_similarity >= 0.95:
        return False  # No EWC
    elif task_similarity >= 0.8:
        return ewc_type == "topology"  # Topology-local only
    else:
        return True  # Global EWC may help for very different tasks

Ablation Study Pattern

# Systematic ablation for Pareto analysis
components = ["fourier", "ewc", "replay", "hebbian", "pruning", "neurogenesis"]
results = {}
for mask in product([True, False], repeat=len(components)):
    config = {c: m for c, m in zip(components, mask)}
    score = evaluate(config)  # Perf, RD, GCR
    results[config] = compute_nes(score)
    
# Find Pareto frontier
pareto_set = find_pareto_frontier(results)
# Identify dominated components for compression

Applications

• Continual learning systems for autonomous agents
• Multi-task learning with task similarity estimation
• Model compression via Pareto analysis
• Neuroplastic architecture design for edge AI
• Benchmarking continual learning approaches (MEP-BENCH)

Pitfalls

• EWC is not universally beneficial: At high task similarity, it actively degrades performance — test before applying
• Fourier encoding is non-negotiable: Removing it causes catastrophic performance drop; don't skip it for efficiency
• 11 components is overkill: MPCS_EFFICIENT (removing EWC + Hebbian) outperforms full MPCS with 4.7× less compute
• MEP-BENCH is domain-specific: Results validated on 31 tasks across 4 domains; generalization to other domains needs empirical validation
• Topology-local EWC: Requires defining task topology/neighborhood structure — not always available
• Pareto analysis requires multi-objective tracking: Need to measure Perf, RD, and GCR simultaneously, not just accuracy

Related Skills

• mistake-gated-continual-learning
• feedback-hebbian-continual-learning
• multi-plasticity-snn-training
• noracl-neurogenesis-continual-learning
• sleep-like-plasticity