dendritic-in-context-learning-snn

name	dendritic-in-context-learning-snn
description	DendriCL methodology for dendritic in-context learning in single-layer spiking neural networks. Demonstrates that a single dendritic compartment with online-LMS dynamics implements complete in-context learning, eliminating the need for attention, depth, or inference-time plasticity.
created	2026-07-12T00:00:00.000Z
source	arXiv:2607.02283
tags	["spiking neural networks","in-context learning","dendritic computation","neuromorphic computing","online learning","compartmental models"]

Dendritic In-Context Learning in Single-Layer Spiking Neural Networks

Overview

Paper: Dendritic In-Context Learning in a Single-Layer Spiking Neural Network
arXiv: 2607.02283 (July 2026)
Authors: Juwei Shen, Yujie Wu, Changwen Chen

Problem Statement

In-context learning (ICL) is a hallmark capability of modern AI architectures (Transformers, Mamba, state-space models, MLPs), operating via implicit gradient descent embedded in the forward pass. Capturing ICL in biologically plausible Spiking Neural Networks (SNNs) has been an open challenge — existing SNNs fail the Garg-2022 benchmark at non-trivial task dimensions.

Key Insight

Prior SNN designs route adaptation through inference-time synaptic plasticity, treating the dendritic compartment as a passive conduit for error or teacher signals. DendriCL challenges this: the subthreshold dynamics of a single dendritic compartment already implement a complete online learning algorithm.

Core Methodology

DendriCL Architecture

1. Single-layer compartmental spiking architecture with apical recurrence
2. Apical compartment treated as the computational substrate (not a passive conduit)
3. Structural equivalence between apical recurrence and leaky online Widrow-Hoff LMS (Least Mean Squares)
4. Dynamics-only update collapses the architectural depth required for general-purpose ICL to a single layer

Mathematical Foundation

• The apical dendritic compartment dynamics implement online LMS through subthreshold membrane potential evolution
• Apical recurrence pattern: V_apical(t) = α·V_apical(t-1) + W·input(t) + bias
• This is structurally identical to leaky online Widrow-Hoff LMS: w(t+1) = (1-λ)w(t) + η·error·input
• A linear probe recovers the reference online-LMS trajectory directly from the apical membrane at R² = 0.93

Key Results

• Uniquely seed-stable at super-dimensional Garg-2022 ICL
• Dense Transformers exhibit grokking-style instability and fail past moderate task dimension; DendriCL does not
• ICL requires neither attention, depth, nor inference-time plasticity
• A single compartment with online-LMS dynamics is sufficient for general-purpose ICL

Implications

For Neuromorphic Computing

• Eliminates the need for multi-layer SNN architectures for ICL tasks
• Enables energy-efficient, single-layer spiking processors capable of in-context learning
• Reduces hardware complexity while maintaining ICL capability

For Biological Plausibility

• Aligns with biological evidence that dendritic compartments perform local computation
• Suggests biological neurons may implement ICL-like capabilities through dendritic dynamics alone
• Provides a bridge between theoretical ICL mechanisms and biological neural computation

For SNN Design

• Shifts design paradigm from synaptic plasticity-based adaptation to dendritic dynamics-based computation
• Enables simpler, more efficient SNN architectures for tasks requiring online adaptation
• Opens new directions for compartmental spiking models

Implementation Guide

Architecture Components

1. Basal dendrite: Receives standard sensory input
2. Apical dendrite: Recurrent compartment implementing online LMS dynamics
3. Somatic layer: Spike generation based on combined dendritic inputs
4. No inference-time synaptic weight updates: All adaptation occurs through compartment dynamics

Training Protocol

1. Train the feedforward weights (basal→soma) using standard surrogate gradient methods
2. Configure apical compartment parameters (leak rate, integration time constant) to match online LMS
3. During inference, apical dynamics automatically adapt to new contexts

Hyperparameter Guidelines

• Leak rate: Controls adaptation speed (higher = faster but less stable)
• Integration time constant: Determines context window length
• Apical recurrence strength: Must be tuned to match the learning rate of online LMS

Activation Triggers

Use this skill when working with:

• In-context learning in spiking neural networks
• Biologically plausible online learning mechanisms
• Dendritic computation and compartmental neuron models
• Single-layer SNN architectures for adaptation tasks
• Neuromorphic implementations of transformer-like capabilities
• Widrow-Hoff LMS in neural dynamics
• Garg-2022 benchmark for SNN in-context learning

Related Concepts

• Online Widrow-Hoff LMS algorithm
• Compartmental neuron models
• Surrogate gradient learning in SNNs
• Grokking instability in neural networks
• Garg-2022 ICL benchmark
• Dendritic computation theory
• Biological plausibility of in-context learning

References

• Shen, J., Wu, Y., Chen, C. (2026). "Dendritic In-Context Learning in a Single-Layer Spiking Neural Network." arXiv:2607.02283
• Garg, S., et al. (2022). "What Can Transformers Learn In-Context? A Case Study of Simple Function Classes." NeurIPS 2022