| name | estimate-actual |
| description | [Planning] Use when calibrating estimates from actual code, diff, PR scope, and developer time. |
| disable-model-invocation | false |
| argument-hint | <plan-file> | --changes | --pr <number> |
Quick Summary
Goal: Produce a 3-way estimation calibration report — pre_impl_estimate (from plan) vs true_estimate (from observed scope) vs actual_time (from git/user) — yielding two INDEPENDENT signals: developer execution variance and estimation model calibration variance.
Why two signals matter: They are confounded if not separated. If actual >> pre-impl, the bug could be (a) developer was slow, OR (b) the model under-estimated scope. Without computing TRUE from observed scope, you cannot tell which. Single-sample calibration has near-zero statistical power — the skill always reports this.
Workflow:
- Detect input mode — plan-file path /
--changes / --pr <n>
- Read pre-impl estimate from plan frontmatter (if present)
- Observe actual scope — git diff, file list, line counts, blast radius from the diff
- Compute TRUE estimate — apply the canonical estimation framework (inlined below) to the OBSERVED scope (post-hoc, with full visibility)
- Get actual time — derive from git timestamps if available; ask user to confirm/override (timestamps ≠ coding time)
- Compute variances — scope variance (TRUE vs pre-impl) and execution variance (actual vs TRUE)
- Per-layer breakdown — UI tier, backend tier, test count, blast radius — predicted vs observed
- Report — calibration verdict + suggested model adjustments (only if pattern across ≥3 samples)
Key Rules:
- MUST ATTENTION compute TRUE estimate using the SAME canonical framework that was used for pre-impl — fair comparison requires identical methodology
- MUST ATTENTION separate developer execution signal from model calibration signal — never collapse to a single "good/bad estimate" verdict
- NEVER claim a model adjustment from a single sample — explicitly state "needs ≥3 samples for signal"
- NEVER trust git timestamps as actual coding time — they include sleep, meetings, context switches; ALWAYS ask user to validate
- MUST ATTENTION list per-layer deltas (UI/backend/tests/blast) — aggregate variance hides where the model went wrong
- Use min-max range for both pre-impl and TRUE — comparing single points is dishonest about uncertainty
Input Modes
| Mode | Trigger | What's read |
|---|
| Plan-file | <path/to/plan.md> | Plan frontmatter (pre-impl estimate) + git diff scoped to plan branch |
| Changes | --changes | git diff working tree + last commit timestamps |
| PR | --pr <n> | gh pr view <n> + gh pr diff <n> + PR open/merge times |
If multiple modes detected (e.g., plan file AND --changes), prefer plan-file (carries the original estimate); use changes for the diff source.
Workflow Detail
Step 1: Detect mode and gather pre-impl estimate
- Plan-file mode: read frontmatter, extract
man_days_traditional, story_points, risk_margin_pct, risk_factors, blast_radius, estimate_reasoning if present
- Changes/PR mode without plan: skip pre-impl; report only TRUE vs actual
- If plan exists but uses old single-point format (no range, no
risk_factors), flag in report — comparison is approximate
Step 2: Observe actual scope from diff
Run (PowerShell or Bash via tool):
git diff --stat <base>..<head> — file count, lines added/removed
git diff --name-only <base>..<head> — file list
git log --format='%H %ai %s' <base>..<head> — commit timeline
Classify changed files:
- UI files (component/template/style) — count, group by screen
- Backend handlers/entities/repos — count, classify per backend tier table
- Tests (unit/integration/e2e) — count test files, count test cases (grep
describe|it|Fact|Test\b)
- Migrations / contracts / shared code — flag separately
Step 3: Run Blast Radius pass on observed scope
- Touched files / components — count
- Of those, complex (>500 LOC area, multi-handler, central) — count
- Downstream consumers — use code graph trace if available:
python .claude/scripts/code_graph trace <file> --direction both --json for changed entry-point files
- Shared/common code touched — yes/no
- Regression scope — list affected areas
Step 4: Apply canonical estimation framework to observed scope
Apply each tier table (UI / backend / test / risk margin / risk factors) from the inline framework below to the OBSERVED scope. Output:
true_likely_days (single midpoint)
true_min_days = likely × 0.9
true_max_days = likely × (1 + risk_margin)
true_estimate = '<min>-<max>d' range
Step 5: Get actual time
Try in order:
- Git: timestamp of first commit on feature branch → timestamp of last commit (or merge commit). Convert to working days (8h business days, exclude weekends).
- PR: open time → merge time. Same conversion.
- Ask user via
AskUserQuestion: "Git suggests N working days from first commit to merge. How much was actual coding time? (excludes meetings, code-review wait, context switches, vacations)"
ALWAYS surface the gap between elapsed time and reported coding time — they are different signals.
Step 6: Compute variances
scope_variance_pct = (true_likely - preimpl_likely) / preimpl_likely × 100
exec_variance_pct = (actual_time - true_likely) / true_likely × 100
Interpretation matrix:
| scope_var | exec_var | Verdict |
|---|
| ~0% (±15%) | ~0% (±15%) | Estimate matched scope; developer matched estimate. Healthy. |
| ~0% | >+25% | Model OK; developer slower than expected. Performance signal. |
| ~0% | <-25% | Model OK; developer faster than expected. Either skilled or scope simpler than apparent. |
| >+25% | ~0% | Model UNDER-estimated scope; developer matched the harder-than-predicted reality. Model signal — too optimistic. |
| <-25% | ~0% | Model OVER-estimated scope; actual work was simpler. Model signal — too pessimistic. |
| >+25% | >+25% | Both — scope was harder AND developer slower. Disambiguate over multiple samples. |
| <-25% | <-25% | Original estimate was way over; developer also fast. Likely simple task padded heavily. |
Step 7: Per-layer breakdown (where the model went wrong)
| Layer | Pre-impl tier (from plan) | Observed tier (from diff) | Delta |
|---|
| UI | e.g. "Compose components into NEW screen" | e.g. "Add control to existing screen" | -1 tier (~0.7d over) |
| Backend | e.g. "NEW command on existing aggregate" | e.g. "Small update existing handler" | -1 tier (~0.5d over) |
| Tests | e.g. "13 cases" | e.g. "5 cases" | -8 cases (~0.5d over) |
| Blast | e.g. "4 areas, 1 complex" | e.g. "2 areas, 0 complex" | lower regression risk |
| Risk factors | predicted list | applicable in retrospect | call out missing/unused |
Step 8: Report
Produce a markdown report with sections:
- Summary table — three numbers (pre-impl range, TRUE range, actual single)
- Variance verdict — interpretation matrix row + plain-English explanation
- Per-layer breakdown — table above
- Risk factors — predicted vs applicable; note any new factors that surfaced (e.g., regression-fan-out not flagged but should have been)
- Calibration suggestion — ONLY if user has run this skill ≥3 times with consistent direction. Single-sample → state "no statistical power, log this sample for future calibration"
- Confidence — state confidence level for each verdict; uncertainty about actual time goes here
Step 9: Persist sample (optional)
If user wants longitudinal tracking, append the calibration row to plans/_estimation-samples.csv:
date,plan,preimpl_min,preimpl_max,true_min,true_max,actual,scope_var_pct,exec_var_pct,risk_factors_predicted,risk_factors_applicable
After ≥5 rows, run pattern detection on the CSV: if scope_var_pct is consistently negative (model over-estimates), suggest tier adjustment; if consistently positive (under-estimates), suggest adding risk factors or widening tier.
Estimation Framework (canonical — applied in Step 4)
The canonical framework lives in the Estimation Framework sync block at the end of this skill; Step 4 applies it verbatim to the observed (post-hoc) scope.
Output Report Template
# Estimation Calibration Report — <plan or branch name>
## Summary
| Metric | Range / Value | Source |
| ----------------- | -------------------------- | ---------------------------------------- |
| Pre-impl estimate | <min>-<max>d (likely <m>d) | <plan path frontmatter> |
| TRUE estimate | <min>-<max>d (likely <m>d) | observed scope (post-hoc) |
| Actual time | <n>d | git <first commit→merge>, user-confirmed |
**Scope variance** (TRUE vs pre-impl): <±n>% — <under/over/matched>
**Execution variance** (actual vs TRUE likely): <±n>% — <fast/slow/matched>
## Verdict
| Signal | Direction | Magnitude | Confidence |
| ------------------- | ----------------------------------------------- | --------- | ----------------- |
| Estimation model | <too optimistic / too pessimistic / calibrated> | <±n>% | <low/medium/high> |
| Developer execution | | <±n>% | |
| Layer | Predicted tier | Observed tier | Delta |
| ------------ | ------------------ | ------------------ | --------------- |
| UI | … | … | … |
| Backend | … | … | … |
| Tests | … cases | … cases | … |
| Blast radius | … areas, … complex | … areas, … complex | … |
| Risk factors | | | |
No model adjustment from one data point. Logged to (row N). Re-run /estimate-actual on future plans to build calibration corpus. Suggested adjustment after ≥3-5 samples with consistent direction.
e.g. "UI tier 'Compose components into NEW screen' overshoots in 4/5 samples by ~0.5d → suggest splitting into two tiers OR widening band to 1-2.5d"
Actual time derived from ;
Pre-impl estimate format — comparison
Confidence in TRUE estimate: — observed scope
Anti-Rationalization Anchors
| Evasion | Rebuttal |
|---|
| "Single sample is enough — clearly the dev was slow" | NO. Without separating scope from execution variance, you confound model error and performance. State signal + caveat. |
| "Use git timestamps as actual time" | Wrong. Includes weekends, meetings, code-review wait, sleep. Always confirm with user. |
| "Skip TRUE estimate — just compare pre-impl vs actual" | That's the data point that's MISSING and exactly why estimates don't improve over time. Never skip Step 4. |
| "Apply hindsight to pump up TRUE estimate" | Use the SAME framework that was used for pre-impl. Hindsight bias inflates TRUE and falsely vindicates the original estimate. |
| "One signal is fine, no need to split" | Two signals is the entire point. Performance review needs execution variance; model tuning needs scope variance. Confounded data is unactionable. |
Estimation Framework — Bottom-up first; SP DERIVED; output min-max range when likely ≥3d. Stack-agnostic. Baseline: 3-5yr dev, 6 productive hrs/day. AI estimate assumes Claude Code + project context.
Method:
- Blast Radius pass (below) — drives code AND test cost
- Decompose phases → hours/phase →
bottom_up_hours = Σ phase_hours
likely_days = ceil(bottom_up_hours / 6) × productivity_factor
- Sum Risk Margin (base + add-ons) →
max_days = likely_days × (1 + margin)
min_days = likely_days × 0.9
- Output as range when
likely_days ≥3; single point allowed <3 (still record margin)
man_days_ai = same range × AI speedup
story_points DERIVED from likely_days via SP-Days — NEVER driver. Disagreement >50% → trust bottom-up
Productivity factor: 0.8 strong scaffolding+codegen+AI hooks · 1.0 mature default · 1.2 weak patterns · 1.5 greenfield
Cost Driver Heuristic (apply BEFORE work-type row):
- UI dominates in CRUD/business apps — 1.5-3x backend (states, validation, responsive, a11y, polish)
- Backend dominates ONLY: multi-aggregate invariants, cross-service contracts, schema migrations, heavy query/perf, new event flows
Reuse-vs-Create axis (PRIMARY lever, per layer):
| UI tier | Cost |
|---|
| Reuse component on existing screen | 0.1-0.3d |
| Add control/column to existing screen | 0.3-0.8d |
| Compose components into NEW screen | 1-2d |
| NEW screen, custom layout/states/validation | 2-4d |
| NEW shared/common component (themed, tested) | 3-6d+ |
| Backend tier | Cost |
|---|
| Reuse query/handler from new place | 0.1-0.3d |
| Small update existing handler/entity | 0.3-0.8d |
| NEW query on existing repo/model |
AI Mistake Prevention — Failure modes to avoid on every task:
Re-read files after context changes. Context compaction, resume, or long-running work can make memory stale; verify current files before acting.
Verify generated content against source evidence. AI hallucinates APIs, names, claims, and document facts. Check the relevant source before documenting or referencing.
Check downstream references before deleting or renaming. Removing an artifact can stale docs, generated mirrors, configs, and callers; map references first.
Trace the full impact chain after edits. Changing a definition can miss derived outputs and consumers. Follow the affected chain before declaring done.
Verify ALL affected outputs, not just the first. One green check is not all green checks; validate every output surface the change can affect.
Assume existing values are intentional — ask WHY before changing. Before changing a constant, limit, flag, wording, or pattern, read nearby context and history.
Surface ambiguity before acting — don't pick silently. Multiple valid interpretations require an explicit question or stated assumption with risk.
Keep shared guidance role-relevant. Universal guidance must help every receiving skill or agent; code-specific obligations belong only in code-specific protocols.
Closing Reminders
Protocols in force (concise digest of the SYNC/shared blocks this skill carries):
- Estimation Framework: Bottom-up first, blast-radius pass, min-max range, tier + risk-margin tables.
- AI Mistake Prevention: verify generated content against evidence, trace downstream references, verify all affected outputs, re-read after context loss, surface ambiguity.
- Critical Thinking: Traced
file:line proof per claim, confidence >80% to act, no guess-as-fact.
IMPORTANT MUST ATTENTION compute TRUE estimate using the SAME canonical framework — fair comparison requires identical methodology
IMPORTANT MUST ATTENTION separate developer execution signal from model calibration signal — never collapse to single verdict
IMPORTANT MUST ATTENTION never claim model adjustment from a single sample — explicitly state "needs ≥3 samples for signal"
IMPORTANT MUST ATTENTION never trust git timestamps as coding time — always ask user to confirm/override
IMPORTANT MUST ATTENTION list per-layer deltas (UI/backend/tests/blast) — aggregate variance hides where model went wrong
IMPORTANT MUST ATTENTION use min-max ranges for both pre-impl and TRUE — comparing single points is dishonest about uncertainty
IMPORTANT MUST ATTENTION apply Blast Radius pass on observed diff before applying tier tables
IMPORTANT MUST ATTENTION persist samples to plans/_estimation-samples.csv for longitudinal calibration
IMPORTANT MUST ATTENTION state confidence per verdict — uncertainty about actual time goes in caveats
[IMPORTANT] Use TaskCreate to break ALL work into small tasks BEFORE starting.
Critical Thinking Mindset — Apply critical thinking, sequential thinking. Every claim needs traced proof, confidence >80% to act.
Anti-hallucination: Never present guess as fact — cite sources for every claim, admit uncertainty freely, self-check output for errors, cross-reference independently, stay skeptical of own confidence — certainty without evidence root of all hallucination.
MUST ATTENTION apply critical + sequential thinking — every claim needs appropriate traced evidence (file:line for repo/code claims; source URL or artifact section for research, product, content, and docs claims); confidence >80% to act, <60% DO NOT recommend. Anti-hallucination: never present guess as fact, admit uncertainty freely, cross-reference independently, stay skeptical of own confidence.
MUST ATTENTION apply AI mistake prevention — verify generated content against evidence, trace downstream references before deleting or renaming, verify all affected outputs, re-read files after context loss, and surface ambiguity before acting.
[TASK-PLANNING] Before acting, analyze task scope and systematically break it into small todo tasks and sub-tasks using TaskCreate.