| name | distilling-feedback-memory |
| title | Distilling Feedback into Memory-as-a-Tool |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2601.05960 |
| keywords | ["agent-learning","feedback-integration","memory-systems","cost-efficiency","iterative-refinement"] |
| description | Convert inference-time feedback into persistent, retrievable guidelines stored as agent memory. Framework enables LLMs to improve performance over time by systematically accumulating and applying learned critiques. Augmented models rapidly match test-time refinement performance while drastically reducing inference cost. Memory-as-tool pattern enables agents to learn from feedback without expensive retraining. |
Problem
Standard LLM refinement approaches face limitations:
- High Inference Cost: In-context test-time refinement requires multiple forward passes, multiplying inference cost
- Knowledge Loss: Feedback provided during inference is not retained for future tasks
- Repeated Mistakes: Agents don't systematically avoid past errors across separate problems
- No Explicit Learning: Models can't accumulate guidelines or rubrics learned from feedback
- Inefficiency: Each task gets similar refinement budget even if pattern is clear
Agents need ways to learn from feedback that are cheaper than test-time refinement and don't require retraining.
Solution
Distilling Feedback into Memory introduces Memory-as-a-Tool:
- Feedback Capture: During inference, capture structured feedback from external evaluators or self-reflection
- "Don't assume the user meant X when they said Y"
- "Always verify numerical claims before asserting them"
- Persistent Guidelines: Convert feedback into reusable rubrics stored in file-based memory
- Tool Integration: Agents can call memory to retrieve relevant learned guidelines during task execution
- Rubric-Feedback Bench: New benchmark with structured rubric feedback for measuring guideline internalization
When to Use
- Iterative Agent Improvement: Agents that process multiple similar tasks and should improve over time
- Cost-Sensitive Deployment: Systems where refinement cost must be minimized
- Learning Agents: Systems where feedback is available but retraining is infeasible
- Safety Enhancement: Systems that must learn and apply safety guidelines from feedback
- Domain Adaptation: Agents adapting to new domains with user-provided feedback
When NOT to Use
- For one-shot tasks where no feedback is available
- When sufficient retraining capacity exists (direct fine-tuning is more effective)
- In systems with no feedback mechanism
- For tasks where learned guidelines might become stale
Core Concepts
The framework operates on the principle that feedback is learnable as stored guidelines:
