| name | supervised-memory-training |
| category | neuroscience |
| description | Supervised Memory Training (SMT) methodology for parallel RNN pretraining without backpropagation through time. |
| activation | RNN training, supervised memory, parallel training, predictive state, memory transition, BPTT replacement |
supervised-memory-training
Description
Supervised Memory Training (SMT) methodology for parallel RNN pretraining without backpropagation through time. Based on arXiv: 2606.06479.
Activation Keywords
- RNN training
- supervised memory
- parallel training
- predictive state
- memory transition
- BPTT replacement
Source Paper
- arXiv: 2606.06479
- Title: Pretraining Recurrent Networks without Recurrence
- Published: 2026-06-04
Core Methodology
Key Concepts
Supervised Memory Training (SMT) trains nonlinear RNNs without recurrent credit propagation by reducing to supervised learning on one-step memory transition labels. SMT acquires memory labels by training a Transformer-based encoder on a predictive state objective, retaining only information from the past necessary to predict the future. Decouples what to remember from how to update memory. Enables time-parallel RNN training with stable O(1) length gradient path without unrolling the RNN.
Mathematical Framework
- Core Innovation: Supervised Memory Training (SMT) trains nonlinear RNNs without recurrent credit propagation by reducing to supervised learning on one-step memory transition labels.
- Key Result: Derived from the abstract analysis of the paper's contribution.
Usage Patterns
Pattern 1: Supervised Memory Training
Apply this methodology when analyzing or implementing the described approach.
Pattern 2: Evaluation and Comparison
Compare against baseline approaches to validate improvements.
Instructions for Agents
Step 1: Understand the Core Innovation
Read the paper abstract and identify the key methodological contribution.
Step 2: Identify Applicable Domains
Determine if the methodology applies to the current problem domain.
Step 3: Apply or Evaluate
Either apply the methodology directly or use it as a comparison baseline.
Error Handling
- If the methodology requires specific hardware (e.g., trapped-ion qubits), note limitations.
- If the approach is theoretical only, mark as such.
Resources
