meta-representational-predictive-coding

name	meta-representational-predictive-coding
description	Meta-Representational Predictive Coding (MPC) — encoder-only neuroscience-informed self-supervised learning within the free energy principle, using cross-stream latent prediction and active inference saccade planning instead of backpropagation (arXiv: 2503.21796v2)
license	Complete terms in LICENSE.txt
metadata	{"arxiv_id":"2503.21796","arxiv_version":"v2","published":"2025-03-22","updated":"2026-07-02","authors":"Alexander G. Ororbia, Karl Friston, Rajesh P. N. Rao","tags":["predictive-coding","self-supervised-learning","free-energy-principle","active-inference","NeuroSSL","brain-inspired","encoder-only","biological-plausibility"]}

Meta-Representational Predictive Coding (MPC)

arXiv: 2503.21796v2 | Updated: 2026-07-02 | Authors: Ororbia (RIT), Friston (VERSES AI), Rao (UW)

Core Concept

Meta-Representational Predictive Coding (MPC) inverts standard predictive coding: instead of learning a generative decoder that predicts raw sensory input (pixels), MPC learns an encoder-only model where latent representations predict each other across parallel neural streams. This creates biologically plausible self-supervised learning (NeuroSSL) without backpropagation.

Key Insight: Standard predictive coding (GPC) requires predicting high-dimensional sensory data. MPC sidesteps this by predicting representations across streams — foveal predicts parafoveal, parafoveal predicts peripheral, and vice versa — using local Hebbian plasticity.

Architecture

Multi-Stream Visual Structure

Inspired by central/peripheral vision functional anatomy:

• Foveal streams: High-resolution fine-grained features (stroke/arc components)
• Parafoveal streams: Medium-resolution object/part chunks
• Peripheral streams: Low-resolution global context

Each stream V processes sensory glimpses through L layers with:

mu^{l,v} = W^{l,v} · phi(z^{l-1,v}(t))    (intra-stream prediction)

Inter-Stream Cross-Prediction

Stream v predicts stream q's latent activity:

mu_C^{l,v,q} = R^{l,v,q} · phi(z^{l,q}(t)) + A^{l,v,q} · a^{l,v}    (inter-stream + action-conditional)

Where:

• R^{l,v,q}: inter-stream predictive synapses
• A^{l,v,q}: action-conditional afferent synapses (glimpse position)
• a^{l,v}: action/coordinate vector

Variational Free Energy Objective

F = sum_{v=1}^{V} [ sum_{l=1}^{L} gamma_{v,q} · N(z^{l,q}(t); mu_C^{l,v,q}, Sigma_C) + lambda_w · ||W^{l,v}||^2 ]

Inter-stream topology weighting: gamma_{v,q} = alpha^{(q-v) mod V} (power decay kernel)

Learning Rules (Backprop-Free)

Synaptic Plasticity (Local Hebbian)

Intra-stream synapses (Eq. 11):

tau_w · dW^{l,v}/dt = -lambda_w · W^{l,v} + e^{l,v} · (phi(z^{l-1,v}))^T

Inter-stream synapses (Eq. 12):

tau_w · dR^{l,v,q}/dt = -lambda_w · R^{l,v,q} + e_C^{l,v,q} · (phi(z^{l,v}))^T

Action-conditional synapses (Eq. 13):

tau_w · dA^{l,v,q}/dt = -lambda_w · A^{l,v,q} + e_C^{l,v,q} · (a^{l,v})^T

Where e^{l,v} and e_C^{l,v,q} are prediction errors (intra- and inter-stream).

Inference Dynamics (E-step)

Neuronal state update via Euler integration (Eq. 10):

tau · dz^{l,v}/dt = -z^{l,v} + phi(z^{l,v}) + e^{l,v} + sum_q e_C^{l,v,q}

Scheduled as Expectation-Maximization: E-step (inference) then M-step (synaptic update).

Active Inference: Epistemic Saccade Planning

MPC uses active perception to decide where to look next, driven by free energy minimization:

1. Bottom-up focus landscape: S_BU(u,v) = G_BU(o_lgn) — local feature energy from LGN-filtered image
2. Top-down epistemic map: accumulated prediction errors guide information foraging
3. Focus map: A = S_BU + S_TD balances reflexive and epistemic signals
4. Next saccade: c_{k+1} = argmax(A) (or stochastic Gumbel-Max sampling)

Stochastic Foraging Temperature Schedule

Dual-phase sigmoidal decay (Eq. 38):

T(rho) = tau_min + (tau_max - tau_min) / (1 + exp(lambda * (rho - rho_0)))

• Phase 1 (exploration): High temperature for k < 0.35K
• Phase 2 (exploitation): Low temperature for k >= 0.35K
• Config: tau_max=0.012, tau_min=0.0012, rho_0=0.35, lambda=10.0

Latent Glimpse Path Integration

Global code z_K built iteratively across K saccades, inspired by grid-cell spatial mapping:

Content-Location Binding (what-where):

• Content: stream activity magnitudes (z_k)
• Location: coordinate vector (c_k)

Running moments (Eq. 25-26):

Sigma_{f,k} = Sigma_{f,k-1} + f_k       (first-order)
Sigma_{f^2,k} = Sigma_{f^2,k-1} + f_k^2  (second-order)

Where f_k = [z_k; c_k] (concatenation of activity and coordinates).

Key Results

Downstream Classification (KNN probe)

Dataset	MPC (proposed)	GPC-fov	I-JEPA	BP-CNN (supervised)
MNIST	>98% (100 glimpses)	~95%	~97%	~99%
K-MNIST	Competitive	Lower	Lower	~95%
NORB	>88% (80 glimpses)	Lower	N/A	~92%

Key Findings

• MPC outperforms I-JEPA and GPC models in KNN discriminative performance
• Generalization improves with glimpse budget (more saccades = better representation)
• Epistemic saccade planner maintains 80.5% accuracy under maximal boundary, vs 61% for random saccades
• Zero-shot generalization demonstrated in v2 update

Comparison: MPC vs Standard Predictive Coding (GPC)

Feature	MPC (Encoder-only)	GPC (Generative decoder)
Target	Latent representations	Raw sensory input
Architecture	Multi-stream encoder	Decoder-centric
Learning	Hebbian (local)	Hebbian (local)
Self-supervised	Yes (cross-stream)	No (unsupervised reconstruction)
Active sensing	Yes (saccade planning)	Optional
Biological plausibility	High (no backprop)	High (no backprop)

Implementation Notes

Model Configuration

• Number of layers: L=3
• Stream types: foveal, parafoveal, peripheral (V total streams)
• Activation: element-wise nonlinearity phi()
• Synaptic decay: lambda_w (regularization)
• Time constant: tau_w (plasticity rate)

Training Protocol

1. Extract multi-resolution glimpses from image
2. Run E-step: iterate neuronal dynamics to convergence
3. Run M-step: update synapses via Hebbian rules
4. Plan next saccade via epistemic focus map
5. Integrate latent code across saccade trajectory
6. Repeat for K glimpses per image

Datasets Validated

• MNIST (28x28, 10 classes)
• Kuzushiji-MNIST (28x28, 10 classes)
• NYU NORB (96x96, 5 classes, stereo)
• ETH-80 (256x256, 8 classes)

Limitations

1. Biological fidelity: MPC is loose inspiration, not faithful neuroanatomy
2. Scale: Validated on small datasets (MNIST, NORB), not large-scale vision
3. Rate codes: Uses rate-based neurons, not spikes
4. Single modality: Visual only, not extended to audio/language
5. Temporal dynamics: Static images, not video sequences

Activation Keywords

predictive coding, meta-representational, MPC, NeuroSSL, self-supervised learning, free energy principle, active inference, saccade planning, encoder-only, biological plausibility, Hebbian learning, backprop-free, cross-stream prediction, foveal, parafoveal, peripheral vision, free energy, Ororbia, Friston, Rao, joint-embedding, JEPA alternative

Related Skills

• predictive-coding-light — Simpler predictive coding framework for SNNs
• predictive-coding-exponential-family — Extended PC with exponential family
• neocortex-error-driven-predictive-learning — Neocortical learning via error-driven PC
• online-generalised-predictive-coding — Online generalized PC via dynamic expectations
• closed-form-predictive-coding-hgf — Closed-form PC via hierarchical Gaussian filtering

References

1. Ororbia, A., Friston, K., & Rao, R.P.N. (2025/2026). Meta-Representational Predictive Coding: Neuroscience-Informed Self-Supervised Learning. arXiv:2503.21796v2.
2. Rao, R.P.N. & Ballard, D.H. (1999). Predictive coding in the visual cortex. Nature Neuroscience.
3. Friston, K. (2010). The free-energy principle: a unified brain theory? Nature Reviews Neuroscience.
4. Assran, M. et al. (2023). Self-supervised learning from images with a joint-embedding predictive architecture (I-JEPA). CVPR.

Paper: https://arxiv.org/abs/2503.21796 HTML: https://arxiv.org/html/2503.21796v2