Feed actual task results back into agent memory for calibration. Compares predicted vs actual outcomes, records accuracy scores, and tracks estimation quality, prediction quality, and decision quality over time to improve future agent performance.
Feed actual task results back into agent memory for calibration. Compares predicted vs actual outcomes, records accuracy scores, and tracks estimation quality, prediction quality, and decision quality over time to improve future agent performance.
["Run outcome-reflection immediately after a task completes — delay degrades accuracy","Include predicted outcome in task metadata at task creation time (so comparison is possible)","Score on three independent dimensions — estimation, prediction, decision — never aggregate prematurely","Persist calibration scores to memory so future tasks benefit from the history","Flag high-miss tasks for reflection-agent followup to identify root cause"]
Closes the feedback loop between prediction and reality in agent task execution. After a task completes, this skill compares the predicted outcome (recorded at planning time) against the actual outcome (observed at completion), scores the accuracy on three dimensions, and persists the calibration record to memory for future use.
Over time, accumulated calibration data reveals systematic biases (e.g., consistent underestimation of implementation tasks) that planners can use to improve future predictions.
When to Use
Invoke immediately after any task that had a predicted outcome recorded:
After HIGH/EPIC pipeline completion (planner predictions vs actual)
After estimation tasks (token counts, time estimates, file counts)
After architectural decision tasks (expected impact vs observed impact)
After any task where the planner recorded explicit predictions in task metadata
Do not invoke for:
Tasks where no prediction was recorded (nothing to compare)
Trivial tasks (single-file edits) — overhead not justified
Ongoing tasks — invoke only after TaskUpdate(completed)
Iron Law
NO CALIBRATION WITHOUT A PRIOR PREDICTION
If no prediction was recorded at planning time, outcome-reflection cannot score accuracy. The fix is to ensure planners record predictions. See plan-generator for prediction metadata format.
Calibration Dimensions
Dimension 1: Estimation Accuracy
Measures how accurately the agent estimated measurable quantities:
Scored by reading task completion metadata and comparing against task creation metadata.
Dimension 3: Decision Quality
Measures whether the decisions made during the task were appropriate in retrospect:
Did the chosen approach work without significant pivots?
Were there rework loops that a better decision would have avoided?
Did the agent's confidence match the actual difficulty?
Score: 0.0–1.0 (1.0 = no rework, smooth execution; 0.0 = multiple pivots or failure)
Calibration Record Format
Each outcome-reflection run produces one calibration record:
{"taskId":"task-N","taskType":"implementation|planning|estimation|architecture|security","completedAt":"ISO-8601","agentType":"developer|planner|architect|...","predictions":{"estimatedTokens":5000,"estimatedFiles":3,"estimatedSteps":5,"predictedOutcome":"Add JWT auth with refresh tokens","predictedBlockers":["Redis not available"],"confidence":"Medium"},"actuals":{"actualTokens":7200,"actualFiles":5,"actualSteps":8,"actualOutcome":"Added JWT auth with refresh tokens; added Redis fallback","actualBlockers":["Redis not available","JWT library version mismatch"],"reworkLoops":1},"scores":{"estimationAccuracy":0.72,"predictionQuality":0.85,"decisionQuality":0.8,"overall":0.79},"flags":[],"notes":"Token estimate was 44% low. Consider 1.5x buffer for JWT auth tasks."}
Workflow
Step 0: Load Memory and Task Data
# Check calibration history for this agent type
grep -r "outcome-reflection" C:/dev/projects/agent-studio/.claude/context/memory/learnings.md 2>/dev/null | tail -10
Never invoke without a prior prediction — meaningless calibration score
Never aggregate scores before trending — premature aggregation hides per-dimension biases
Never skip low-score flagging — misses compound if root cause is not investigated
Never use calibration to blame agents — use it to improve planning accuracy
Never run on trivial tasks — calibration overhead is not justified for single-file edits
Related Skills
plan-generator — records predictions at planning time (prerequisite for calibration)
instinct-learning — records atomic learned patterns (complementary to calibration)
reflection-agent — investigates high-miss tasks flagged by outcome-reflection
verification-before-completion — gate that runs before completion (runs before this skill)
Memory Protocol
Before starting: Read .claude/context/memory/learnings.md to find prior calibrations for the same agent type and task type. This provides baseline context.
After completing: Append calibration record to .claude/context/memory/learnings.md. Use MemoryRecord tool for structured pattern recording. Do NOT write directly to patterns.json.
Assume interruption: If calibration context is lost, re-read the completed task's metadata from TaskGet — all predictions and actuals should be in task metadata.
