predictive-coding-light

name

predictive-coding-light

description

Predictive Coding Light+ (PCL+) methodology for spiking neural network sequence prediction. A spiking neural network architecture for unsupervised sequence processing that learns recurrent excitatory connections with delays to enable short-term retention of information. Combines spike timing-dependent plasticity (STDP) with synaptic delays to learn predictive representations. Successfully reproduces classic visual cortex sequence learning findings and learns to fill in missing inputs in gesture recognition tasks. Activation: Predictive Coding Light+, PCL+, STDP sequence learning, spiking neural network prediction, synaptic delay learning, unsupervised sequence processing, visual cortex sequence learning, short-term memory SNN, predictive coding spiking, spike timing-dependent plasticity sequence, gesture recognition SNN.

Predictive Coding Light+ Methodology

Overview

Predictive Coding Light+ (PCL+) (arXiv:2605.12732v1) addresses a fundamental question: how can biological or artificial spiking neural networks learn to maintain past sensory information to predict the future? It proposes a novel SNN architecture combining spike timing-dependent plasticity (STDP) with synaptic delays to learn recurrent excitatory connections for short-term information retention.

Core Problem

Successfully predicting future sensory input requires maintaining a memory of the recent past. In spiking neural networks, it remains unclear how to learn this short-term retention without complex recurrent architectures or external memory modules.

Key Contributions

1. PCL+ Architecture

PCL+ learns recurrent excitatory connections with delays to maintain a record of the recent past:

• STDP-driven learning: Synaptic weights adapt based on spike timing correlations
• Synaptic delays: Different connection delays create a distributed temporal buffer
• Recurrent excitatory connections: Enable short-term retention without external memory
• Unsupervised sequence processing: No labeled data required

2. Sequence Learning in Visual Cortex

PCL+ reproduces classic findings on sequence learning in visual cortex, demonstrating:

• Temporal prediction of visual sequences
• Learning of stimulus order statistics
• Emergent selectivity for temporal patterns

3. Missing Input "Fill-in"

PCL+ learns to reconstruct missing input in a challenging gesture recognition task:

• Given partial temporal sequences, predict missing components
• Demonstrates robust temporal pattern completion
• Shows how short-term retention supports inference

Architecture Details

Core Components

Input Layer → [Recurrent Layer with Delays] → Prediction Output
                  ↑ STDP learning rule
                  ← Synaptic delay distribution

STDP with Delays

The key insight is that synaptic delays create a temporal buffer:

• Short delays → recent input retention
• Long delays → extended history access
• STDP learns which delays are useful for prediction
• Emergent delay distribution matches task temporal structure

Learning Rule

# Conceptual STDP update with delays
def stdp_update(pre_spike_time, post_spike_time, delay, weight):
    """STDP rule incorporating synaptic delay"""
    effective_time = post_spike_time - (pre_spike_time + delay)
    
    if effective_time > 0:
        # Pre before post → potentiation
        delta_w = A_plus * exp(-effective_time / tau_plus)
    else:
        # Post before pre → depression
        delta_w = -A_minus * exp(effective_time / tau_minus)
    
    return weight + delta_w

Application to Skill Development

When to Apply PCL+

• Building energy-efficient sequence prediction models
• Implementing biological short-term memory in SNNs
• Unsupervised learning of temporal patterns
• Gesture recognition with temporal context
• Visual sequence prediction tasks
• Any task requiring "filling in" missing temporal data

Comparison with Alternative Approaches

Approach	Memory Mechanism	Supervision	Biological Plausibility
PCL+	Synaptic delays + STDP	Unsupervised	High
RNN/LSTM	Hidden state	Supervised	Low
Transformer	Self-attention	Supervised	Low
Liquid State	Reservoir dynamics	Supervised (readout)	Medium

Implementation Considerations

1. Delay Distribution: Use a range of delays (e.g., 1-20ms) to capture different timescales
2. STDP Parameters: Tune A+, A-, τ+, τ- for the specific temporal structure
3. Network Size: Larger networks capture more complex temporal dependencies
4. Input Encoding: Use latency coding or rate coding depending on the task

Research Implications

1. Biological plausibility: Synaptic delays are a known biological mechanism for temporal processing
2. Energy efficiency: SNNs with STDP are significantly more energy-efficient than traditional RNNs
3. Unsupervised learning: No labeled data required — learns from raw temporal structure
4. Short-term memory: Emerges naturally from recurrent connections with delays, no external memory needed

Related Skills

• stdp-synaptic-delay-learning - STDP learning rule extension
• spiking-neural-network-analysis - SNN paper analysis methodology
• spiking-computational-neuroscience-survey - SNN applications survey
• spikingjelly-framework - SNN deep learning framework

References

• Predictive Coding Light+: arXiv:2605.12732
• Classic visual cortex sequence learning studies
• STDP learning rule literature