distilling-feedback-memory

name	distilling-feedback-memory
title	Distilling Feedback into Memory-as-a-Tool
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2601.05960
keywords	["agent-learning","feedback-integration","memory-systems","cost-efficiency","iterative-refinement"]
description	Convert inference-time feedback into persistent, retrievable guidelines stored as agent memory. Framework enables LLMs to improve performance over time by systematically accumulating and applying learned critiques. Augmented models rapidly match test-time refinement performance while drastically reducing inference cost. Memory-as-tool pattern enables agents to learn from feedback without expensive retraining.

Problem

Standard LLM refinement approaches face limitations:

1. High Inference Cost: In-context test-time refinement requires multiple forward passes, multiplying inference cost
2. Knowledge Loss: Feedback provided during inference is not retained for future tasks
3. Repeated Mistakes: Agents don't systematically avoid past errors across separate problems
4. No Explicit Learning: Models can't accumulate guidelines or rubrics learned from feedback
5. Inefficiency: Each task gets similar refinement budget even if pattern is clear

Agents need ways to learn from feedback that are cheaper than test-time refinement and don't require retraining.

Solution

Distilling Feedback into Memory introduces Memory-as-a-Tool:

1. Feedback Capture: During inference, capture structured feedback from external evaluators or self-reflection
   • "Don't assume the user meant X when they said Y"
   • "Always verify numerical claims before asserting them"
2. Persistent Guidelines: Convert feedback into reusable rubrics stored in file-based memory
3. Tool Integration: Agents can call memory to retrieve relevant learned guidelines during task execution
4. Rubric-Feedback Bench: New benchmark with structured rubric feedback for measuring guideline internalization

When to Use

• Iterative Agent Improvement: Agents that process multiple similar tasks and should improve over time
• Cost-Sensitive Deployment: Systems where refinement cost must be minimized
• Learning Agents: Systems where feedback is available but retraining is infeasible
• Safety Enhancement: Systems that must learn and apply safety guidelines from feedback
• Domain Adaptation: Agents adapting to new domains with user-provided feedback

When NOT to Use

• For one-shot tasks where no feedback is available
• When sufficient retraining capacity exists (direct fine-tuning is more effective)
• In systems with no feedback mechanism
• For tasks where learned guidelines might become stale

Core Concepts

The framework operates on the principle that feedback is learnable as stored guidelines: