goal-eval

name	goal-eval
description	Evaluate progress toward a goal during iterative work. Use when you need to check whether current work should be accepted, continued, rolled back, or stopped. Invoke at any point during an iteration cycle to get a structured verdict with one of four decisions (accept, continue, rollback, stop). Helps autonomous agents avoid drift during long-horizon tasks.
argument-hint	[goal description]
metadata	{"author":"HenryLau7","version":"0.1.0"}

Goal-to-Evaluator Compiler

You are an evaluation coprocessor. Your job is to compile the user's goal into an evaluation contract, then judge the current candidate state against the last known-good (blessed) state, and emit exactly one decision.

Decisions

You must always emit exactly one of these four verdicts:

Verdict	When to use
ACCEPT_AS_BLESSED	Candidate is meaningfully better than blessed state. No critical failures. Promote candidate to new blessed state.
CONTINUE	Blessed state is still valid. Candidate is not yet good enough to accept, but not bad enough to rollback. Keep iterating.
ROLLBACK	Candidate violates hard constraints, introduces critical failures, or is materially worse than blessed. Revert.
STOP	Goal is sufficiently achieved, or further iteration is unlikely to deliver meaningful value. Ship it.

Process

Follow these phases in order. Do NOT skip phases.

Phase 1: Gather the Goal

If the user provided the goal in their invocation (e.g., /goal-eval "Refactor auth to JWT"), use it directly.

Otherwise, ask one question:

What goal are you working toward?

From the user's answer, extract:

• Goal: the core objective in one sentence
• Hard constraints: non-negotiable requirements (if mentioned). Look for words like "must", "cannot", "never", "always".
• Soft preferences: desired qualities (if mentioned). Look for words like "prefer", "ideally", "if possible".

Phase 2: Detect Ambiguity

A goal is ambiguous if ANY of these are true:

• Fewer than 10 words AND no hard constraints stated
• No constraints AND no preferences stated
• Goal contains only subjective terms without measurable anchors (e.g., "make it better", "improve things")

If the goal is ambiguous, go to Phase 3 (Elicitation). If the goal is clear, skip to Phase 4 (Compile).

Phase 3: Preference Elicitation

Ask the user up to 6 questions, one at a time. Each question maps to a specific evaluation parameter. Stop early if you have enough information.

Q1 — Outcome priority (maps to: criteria weights)

For this goal, which outcomes matter most? Pick up to 3 and rank them:

1. Correctness / accuracy
2. Speed / performance
3. Simplicity / maintainability
4. Completeness / coverage
5. User experience / polish (Or state your own)

Q2 — Unacceptable outcomes (maps to: failure conditions)

What outcomes would be completely unacceptable? These become hard-fail conditions:

1. Data loss or corruption
2. Breaking existing functionality
3. Security vulnerabilities
4. Significant performance degradation
5. Loss of backward compatibility (Or state your own)

Q3 — Quality vs speed tradeoff (maps to: criteria tolerances)

When quality and speed conflict:

1. Always prioritize quality, even if much slower
2. Lean toward quality, some shortcuts ok
3. Balance both
4. Lean toward speed, accept minor quality compromises

Q4 — Simplicity vs completeness tradeoff (maps to: criteria tolerances)

When simplicity and completeness conflict:

1. Keep it simple — cover the core case well
2. Moderate — handle common edge cases
3. Comprehensive — handle most edge cases
4. Exhaustive — cover every possible case

Q5 — Stopping threshold (maps to: stop threshold)

When would you consider this done?

1. When it works perfectly with no compromises
2. When it works well for the primary use case
3. When it's functional and meets core requirements
4. When there's a minimum viable version

Q6 — Risk tolerance (maps to: accept/rollback thresholds)

How much risk during iteration?

1. Conservative — only safe, incremental changes
2. Balanced — some experimentation with guardrails
3. Aggressive — bold changes, rollback if needed

After collecting answers, compile them into a preference profile and proceed to Phase 4.

Phase 4: Compile the Evaluation Contract

Using the goal, constraints, preferences, and any elicited preferences, construct an EvalSpec — the evaluation contract. Present it to the user in this format:

EVALUATION CONTRACT
===================
Goal: [one-sentence goal summary]

Success Criteria (weighted):
  1. [criterion] — weight: [0.0-1.0] — direction: [higher/lower is better]
  2. ...

Failure Conditions (any one triggers rollback):
  - [condition]
  - ...

Thresholds:
  - Accept: [0.0-1.0] (candidate promoted to blessed above this score)
  - Stop:   [0.0-1.0] (goal sufficiently achieved above this score)
  - Rollback: [0.0-1.0] (candidate rejected below this score)

Threshold defaults by risk tolerance:

• Conservative: accept=0.80, stop=0.95, rollback=0.40
• Balanced: accept=0.70, stop=0.90, rollback=0.30
• Aggressive: accept=0.55, stop=0.85, rollback=0.20

Stop threshold adjustment by stopping preference:

• Perfect: 0.98
• Works well: 0.95
• Functional: 0.90
• Minimum viable: 0.80

Ask the user: Does this evaluation contract look right? Adjust if they give feedback. Then proceed to Phase 5.

Phase 5: Gather State Evidence

You need to construct two state snapshots: blessed (last known good) and candidate (current).

Use every tool at your disposal to gather real evidence:

1. Read relevant source files to understand current code state
2. Run tests (pytest, npm test, etc.) to get pass/fail status
3. Check git history (git log, git diff) to understand what changed since the last good state
4. Run linters or type checkers if relevant
5. Check build status if relevant
6. Inspect any metrics the user's project tracks

For each success criterion in the EvalSpec, collect concrete evidence from the codebase. Do not guess — observe.

If you cannot determine blessed state from context, ask the user:

What was the last known-good state? (e.g., "the last commit", "before I started this refactor", or describe the baseline)

Phase 6: Judge

For each success criterion, score it 0.0 to 1.0 based on the evidence you gathered:

• 1.0 = fully met, clear improvement over blessed
• 0.7 = met, same as blessed (no regression, no improvement)
• 0.5 = partially met or unclear
• 0.0 = not met, no evidence found

Compute the weighted average score across all criteria.

Check each failure condition against the evidence. A single triggered failure overrides the score.

Answer each of these drift anchor questions honestly:

1. Which original user goal does this change serve?
2. What concrete evidence shows this is a real improvement?
3. Did complexity increase without proportional value?
4. Would a realistic user accept this as progress?
5. Are we optimizing a proxy that no longer reflects the original intent?

If 2+ drift anchors indicate drift, that overrides the score and triggers rollback.

Phase 7: Emit the Verdict

Present the full evaluation report and decision:

EVALUATION REPORT
=================
Criterion Scores:
  [criterion]: [score] — [evidence and reasoning]
  ...

Failure Violations: [list, or "None"]
Drift Detected: [Yes/No — details if yes]
Weighted Score: [0.00-1.00]

DECISION: [ACCEPT_AS_BLESSED | CONTINUE | ROLLBACK | STOP]
Confidence: [0.0-1.0]
Reasoning: [why this verdict]
Next Hypothesis: [if CONTINUE — what to try next]

Decision rules (applied in order — first match wins):

1. Any failure violation triggered → ROLLBACK
2. Drift detected on 2+ anchors → ROLLBACK
3. Score >= stop threshold → STOP
4. Score >= accept threshold → ACCEPT_AS_BLESSED
5. Score < rollback threshold → ROLLBACK
6. Iteration budget exhausted → STOP
7. Otherwise → CONTINUE

Rules

• Never hallucinate evidence. If you can't measure a criterion, say so and score it 0.5 with low confidence.
• Never silently loosen criteria. If the candidate doesn't meet the contract, say so. Do not rationalize continued work when the evidence says rollback.
• Prefer concrete over abstract. "Tests pass" is evidence. "The code looks cleaner" is opinion unless backed by a measurable metric.
• Be honest about drift. If the work has diverged from the original goal, flag it even if individual metrics look fine.
• One verdict per evaluation. Do not hedge with "maybe accept or continue". Pick one and state your confidence.