dendricl-icl-single-layer-snn

name	dendricl-icl-single-layer-snn
description	DendriCL methodology for dendritic in-context learning in single-layer spiking neural networks. The apical compartment's subthreshold dynamics implement online Widrow-Hoff LMS, enabling general-purpose ICL without attention, depth, or inference-time plasticity. First SNN to solve Garg-2022 ICL benchmark at d≥30 where Transformers fail. Trigger: dendritic computation, in-context learning SNN, compartmental neuron, online LMS, biological ICL, neuromorphic ICL, apical dendrite, single-layer learning
arxiv_id	2607.02283v1
date	2026-07-02
authors	Juwei Shen, Yujie Wu, Changwen Chen (HK PolyU)
categories	["cs.NE","cs.LG"]
tags	["spiking neural network","in-context learning","dendritic computation","compartmental neuron","online LMS","neuromorphic"]

DendriCL: Dendritic In-Context Learning in a Single-Layer Spiking Neural Network

Core Insight

In-context learning (ICL) does NOT require attention, depth, or inference-time plasticity. A single dendritic compartment with online-LMS dynamics is sufficient.

The subthreshold dynamics of a single apical dendritic compartment implement a complete online learning algorithm (leaky Widrow-Hoff LMS). By treating the compartment as the computational substrate rather than a passive conduit, ICL emerges from dynamics alone with frozen synaptic weights at inference.

Architecture

Single-layer compartmental spiking network with d_model = 384 parallel pyramidal-like units:

Per-unit recurrence at context position t:
  uB(t) = WB * xt                    (basal projection)
  ŷt = uA(t)^T * WA * xt            (scalar prediction)
  et = (1 - flagt) * (yt - ŷt)      (gated error)
  uA(t+1) = α * uA(t) + γ * et * WA * xt   (apical online-LMS)
  vsoma(t) = gB * uB(t) + gA * WA,out * uA(t)  (somatic integration)
  s(t) = ⊮[vsoma(t) > θ]            (LIF spike, soft reset)

Key design choices:

• Apical state uA is NOT reset by somatic spikes — evolves continuously across full context
• Error et is gated OFF at query position
• All synaptic weights frozen at inference time
• ~0.75M total parameters, single layer

Structural Equivalence: Apical ≡ Leaky Online LMS

When WA = I, the apical update reduces to:

ŵ_{t+1} = α * ŵ_t + γ * (yt - ŵ_t^T * xt) * xt

This is classical leaky Widrow-Hoff LMS. Under i.i.d. inputs and linear targets, E||uA(k) - w||² = O(d/k) — the classical LMS convergence theorem.

Linear probe recovers reference online-LMS trajectory from apical membrane at R² = 0.93, confirming the algorithm is structurally embedded in the dynamics.

Key Results

Garg-2022 ICL Benchmark

• d=10: DendriCL R² = 0.807 (competitive with all architectures)
• d=20: DendriCL R² = 0.820 (best spike-based model, within 3pp of non-spiking ablation)
• d=30: DendriCL R² = 0.807 (ONLY architecture maintaining >0.5)
• d=40: DendriCL R² = 0.787 (Transformers collapse to chance)
• d=50: DendriCL R² = 0.649 (all others at chance floor)

Seed Stability

• DendriCL: σ ≤ 0.036 across entire super-d range (uniquely stable)
• Transformer: bimodal at d=30, collapses at d≥40 (architectural failure, not budget)
• Spikformer: degrades gracefully but trails by 0.17 at d=30

Efficiency

• ~4× spike reduction over Pure LIF
• Projected ~10× Loihi-class energy advantage
• Architectural simplicity and inference-time efficiency co-vary (not trade off)

Biological Grounding

• Three-compartment layout (apical-basal-soma) matches cortical layer-5 pyramidal neurons
• Apical compartment carries persistent multi-dimensional subthreshold voltage with calcium plateaus on 100+ ms timescales
• Consistent with predictive coding framework (apical = error-driven dynamics)
• Falsifiable hypothesis: in vivo recordings should reveal LMS-like dynamics in apical tufts

Contrast with Prior Work

Model	Apical Role	Adaptation Mechanism
Urbanczik-Senn 2014	Teacher signal	Plasticity-driven
Sacramento 2018	Backprop error	Plasticity-driven
Iyer 2022 (Active Dendrites)	External context gate	Plasticity-driven
Miconi 2018	Differentiable plasticity	Hebbian fast weights
DendriCL (Ours)	Online LMS estimator	Dynamics-driven, frozen weights

Practical Applications

1. Neuromorphic ICL: Deploy in-context learning on Loihi/SpiNNaker without inference-time weight updates
2. Energy-efficient adaptation: Single-layer architecture enables ~10× energy savings
3. Brain-inspired AI: Demonstrates that biological dendritic computation is computationally sufficient for ICL
4. Robust high-dimensional learning: Only architecture seed-stable at d≥30
5. Hardware-software co-design: Compartmental dynamics map naturally to analog neuromorphic circuits

Training Details

• BPTT from scratch, AdamW, lr=1e-3, weight decay 1e-4, cosine schedule
• Batch size 64, 10k steps (baseline) or 50k steps (compute-matched)
• LIF surrogate gradients: arctan approximation
• 3 seeds per config (5 seeds for d∈{20,30})
• No pre-training, no plasticity at inference

Implications

1. ICL is a generic property of trained sequence models — now extended to spiking substrate
2. The "depth hypothesis" for ICL is wrong: a single compartment suffices
3. Dendritic computation is not just biologically plausible — it's computationally optimal for ICL
4. Opens path to fully neuromorphic in-context learning systems