feedback-hebbian-continual-learning

# Backpropagation-Free Feedback-Hebbian Continual Learning ## Overview Training framework that eliminates backpropagation by combining random feedback alignment with local Hebbian plasticity rules. Error signals are transmitted through fixed random feedback connections, while forward weights are updated using biologically plausible local learning rules. ## Core Architecture ### Forward Path - Standard feedforward or recurrent network - Forward weights updated via local Hebbian rules - No gradient computation or backpropagation ### Feedback Path - Fixed random feedback weights (initialized once, never updated) - Transmit error signals from output to hidden layers - Replace the need for weight transport (symmetric backprop) ### Local Learning Rule - Hebbian plasticity modulated by feedback error signals - Each synapse updates based only on local pre/post activity and global error - No need to store gradients or compute Jacobians ## Implementation Pattern ```python class FeedbackHebbianNetwork: def __init__(self, layers, feedback_scale=0.1, hebbian_lr=0.001): self.layers = layers self.forward_weights = [torch.randn(l, n) * 0.1 for l, n in zip(layers[:-1], layers[1:])] self.feedback_weights = [torch.randn(n, l) * feedback_scale for l, n in zip(layers[:-1], layers[1:])] self.lr = hebbian_lr def forward(self, x): activations = [x] for w in self.forward_weights: h = torch.relu(activations[-1] @ w) activations.append(h) return activations def hebbian_update(self, activations, target): output = activations[-1] error = output - target for i in range(len(self.forward_weights) - 1, -1, -1): local_error = error if i == len(self.forward_weights) - 1 else local_error local_error = local_error @ self.feedback_weights[i] pre = activations[i] delta_w = self.lr * (pre.T @ local_error) self.forward_weights[i] += delta_w ``` ## Continual Learning Properties - **No catastrophic forgetting**: Local updates don't overwrite all weights simultaneously - **No replay buffer needed**: Natural resilience to sequential task learning - **Biological plausibility**: No weight transport, no global error gradient - **Scalability**: No backpropagation memory overhead ## References - Josh Li, Fow-sen Choa, "A Backpropagation-Free Feedback-Hebbian Network for Continual Learning Dynamics", arXiv:2601.06758v3, 2026-01-11