Apply the fixed model-role and thinking-effort policy whenever work is delegated to subagents or a model/effort choice is made: the orchestrator (top-tier model) DIRECTLY sets service direction, reviews plan/spec documents (with multi-persona-review), improves shipped features, and hunts performance/security problems; plan/spec/planning documents are AUTHORED by Opus, which also owns core implementation and V&V, at xhigh or above; repetitive implementation and E2E tests go to Sonnet at high or above — never delegate below those effort floors. Use whenever you are about to spawn an Agent/Task/Workflow worker, pick a model for a subtask, set a thinking/effort level, assign verification, or hand off orchestration because the current model's quota is exhausted. Trigger on "위임해", "에이전트로 돌려", "오케스트레이션", "모델 역할분담", "어떤 모델로", "effort 얼마로", "thinking level", "서브에이전트", or in English "delegate this", "spawn an agent for", "which model should", "route this task", "verify with", "orchestrate". Fire even when the user does

2026-07-072

gh-issue-workflow

uzysjung/uzys-agent-harness

Treats GitHub Issues as the async backlog + decision channel between user and AI agent. Use when a non-blocking todo / bug / decision needs to persist beyond the chat session. Enforces 5-section body template (Background / Given / Decision / AC / Next) so issues become reusable agent context, not just sticky notes.

2026-06-262

north-star

uzysjung/uzys-agent-harness

Defines and enforces a project's long-term direction (North Star Statement, NSM, Will/Won't, 4-gate decision heuristic). Use when starting a new project, when scope creep is suspected, or when a non-obvious feature request needs prioritization. Sits one layer above SPEC/PRD — answers 'why and where to', not 'what and how'.

2026-06-262

ui-visual-review

uzysjung/uzys-agent-harness

Captures screenshots of key UI flows after E2E tests pass, runs an agent-side first-pass diff (regressions, console errors, layout shifts), then surfaces a checklist for the user's final approval. Use after E2E tests pass on a UI track (csr-*, ssr-*, full). Adapts the GoalTrack screenshot-review pattern into a repeatable workflow.

2026-06-262

gap-analysis-e2e

uzysjung/uzys-agent-harness

Two-mode chained gap analysis. DETECT scans the current service end-to-end through three lenses — north-star alignment, correctness (bugs), and user-perspective (UX) — and enumerates concrete, severity-ranked gaps. Then BENCHMARK researches how reference/benchmark services actually solved each high-ranked gap and PROPOSES a closing approach. Use when the user says any of: "북극성 기준으로 부족한 점", "사용자 관점에서 부족한 점", "다른 벤치마크 서비스는 이 부분을 어떻게 해결했는지", "갭분석", "레퍼런스 서비스랑 비교해서 부족한 점 찾아줘" — or the English equivalents: "gap analysis", "what are we missing vs the ideal/north-star", "benchmark against reference services". Fires for both Korean and English phrasing. NOT for a whole-codebase multi-dimension audit (use ultracode-service-audit) or single-artifact prose review (use multi-persona-review) — this is the narrower gap-vs-benchmark loop.

2026-06-212

multi-persona-review

uzysjung/uzys-agent-harness

A panel-review skill that critiques ONE artifact (launch post, README, doc, markdown, plan, design) via 3-5 disjoint user-perspective personas running in parallel, then synthesizes deduped, severity-ranked improvement points (P0/P1/P2). Use when the user says "작성글을 사용자 관점의 페르소나를 여러명 만들어서 (손넷 모델정도로) 피드백 받아바", "다면 리뷰 해볼까", "페르소나로 리뷰", "여러 관점으로 피드백", or in English "multi-persona review", "review this from different user perspectives", "get persona feedback on this post/README/doc", "panel review this artifact". Lighter than a full service audit — point it at ONE artifact, not a whole codebase. NOT for a whole-codebase multi-dimension audit (use ultracode-service-audit) or a single-axis gap-vs-benchmark loop (use gap-analysis-e2e).

2026-06-212

name	eval-harness
description	Formal evaluation framework for Claude Code sessions implementing eval-driven development (EDD) principles
origin	ECC
tools	Read, Write, Edit, Bash, Grep, Glob

Eval Harness Skill

A formal evaluation framework for Claude Code sessions, implementing eval-driven development (EDD) principles.

When to Activate

Setting up eval-driven development (EDD) for AI-assisted workflows
Defining pass/fail criteria for Claude Code task completion
Measuring agent reliability with pass@k metrics
Creating regression test suites for prompt or agent changes
Benchmarking agent performance across model versions

Philosophy

Eval-Driven Development treats evals as the "unit tests of AI development":

Define expected behavior BEFORE implementation
Run evals continuously during development
Track regressions with each change
Use pass@k metrics for reliability measurement

Eval Types

Capability Evals

Test if Claude can do something it couldn't before:

[CAPABILITY EVAL: feature-name]
Task: Description of what Claude should accomplish
Success Criteria:
  - [ ] Criterion 1
  - [ ] Criterion 2
  - [ ] Criterion 3
Expected Output: Description of expected result

Regression Evals

Ensure changes don't break existing functionality:

[REGRESSION EVAL: feature-name]
Baseline: SHA or checkpoint name
Tests:
  - existing-test-1: PASS/FAIL
  - existing-test-2: PASS/FAIL
  - existing-test-3: PASS/FAIL
Result: X/Y passed (previously Y/Y)

Grader Types

1. Code-Based Grader

Deterministic checks using code:

# Check if file contains expected pattern
grep -q "export function handleAuth" src/auth.ts && echo "PASS" || echo "FAIL"

# Check if tests pass
npm test -- --testPathPattern="auth" && echo "PASS" || echo "FAIL"

# Check if build succeeds
npm run build && echo "PASS" || echo "FAIL"

2. Model-Based Grader

Use Claude to evaluate open-ended outputs:

[MODEL GRADER PROMPT]
Evaluate the following code change:
1. Does it solve the stated problem?
2. Is it well-structured?
3. Are edge cases handled?
4. Is error handling appropriate?

Score: 1-5 (1=poor, 5=excellent)
Reasoning: [explanation]

3. Human Grader

Flag for manual review:

[HUMAN REVIEW REQUIRED]
Change: Description of what changed
Reason: Why human review is needed
Risk Level: LOW/MEDIUM/HIGH

Metrics

pass@k

"At least one success in k attempts"

pass@1: First attempt success rate
pass@3: Success within 3 attempts
Typical target: pass@3 > 90%

pass^k

"All k trials succeed"

Higher bar for reliability
pass^3: 3 consecutive successes
Use for critical paths

Eval Workflow

1. Define (Before Coding)

## EVAL DEFINITION: feature-xyz

### Capability Evals
1. Can create new user account
2. Can validate email format
3. Can hash password securely

### Regression Evals
1. Existing login still works
2. Session management unchanged
3. Logout flow intact

### Success Metrics
- pass@3 > 90% for capability evals
- pass^3 = 100% for regression evals

2. Implement

Write code to pass the defined evals.

3. Evaluate

# Run capability evals
[Run each capability eval, record PASS/FAIL]

# Run regression evals
npm test -- --testPathPattern="existing"

# Generate report

4. Report

EVAL REPORT: feature-xyz
========================

Capability Evals:
  create-user:     PASS (pass@1)
  validate-email:  PASS (pass@2)
  hash-password:   PASS (pass@1)
  Overall:         3/3 passed

Regression Evals:
  login-flow:      PASS
  session-mgmt:    PASS
  logout-flow:     PASS
  Overall:         3/3 passed

Metrics:
  pass@1: 67% (2/3)
  pass@3: 100% (3/3)

Status: READY FOR REVIEW

Integration Patterns

Pre-Implementation

/eval define feature-name

Creates eval definition file at .claude/evals/feature-name.md

During Implementation

/eval check feature-name

Runs current evals and reports status

Post-Implementation

/eval report feature-name

Generates full eval report

Eval Storage (.md + .log Pair Format)

각 평가 항목은 <topic>.md (설계) + <topic>.log (실행 결과) 쌍으로 저장. 강제. 단독 .md만 있으면 재현 불가.

.claude/
  evals/
    feature-xyz.md        # Eval definition (Capability/Regression/Test 3섹션 필수)
    feature-xyz.log       # Eval run history (실행 시각, grader, pass/fail)
    session-YYYYMMDD.md   # 세션 단위 회고 + 차기 backlog
    session-YYYYMMDD.log  # 동일 세션의 grader 출력
    baseline.json         # Regression baselines (선택)

Vantage 프로젝트의 .claude/evals/*.{md,log} 구조를 일반화한 것.

.md 파일 의무 섹션 (3개)

# Eval: <topic>

## Capability
[새 능력 — Claude/agent가 무엇을 할 수 있는지]
- AC: [측정 가능 기준]
- Grader: code-based / model-based / human

## Regression
[기존 기능 보호 — 변경으로 깨지면 안 되는 baseline]
- Baseline: <SHA or checkpoint>
- Tests: [목록]

## Test
[실행 절차 — 누가 다시 돌려도 동일 결과 나와야 함]
- Setup: [사전 조건]
- Run: `bash run-eval.sh <topic>` 또는 명시적 명령
- Expected: [기대 출력]

.log 파일 형식

각 실행마다 append. 시간순 누적.

=== 2026-04-19 14:32 (run #1) ===
Capability: 3/3 PASS (pass@1)
Regression: 5/5 PASS (pass^3)
Status: SHIP READY

=== 2026-04-20 09:15 (run #2 — after refactor) ===
Capability: 3/3 PASS
Regression: 4/5 PASS (login-flow regressed at SHA abc123)
Status: BLOCKED — fix login-flow first

Best Practices

Define evals BEFORE coding - Forces clear thinking about success criteria
Run evals frequently - Catch regressions early
Track pass@k over time - Monitor reliability trends
Use code graders when possible - Deterministic > probabilistic
Human review for security - Never fully automate security checks
Keep evals fast - Slow evals don't get run
Version evals with code - Evals are first-class artifacts

Example: Adding Authentication

## EVAL: add-authentication

### Phase 1: Define (10 min)
Capability Evals:
- [ ] User can register with email/password
- [ ] User can login with valid credentials
- [ ] Invalid credentials rejected with proper error
- [ ] Sessions persist across page reloads
- [ ] Logout clears session

Regression Evals:
- [ ] Public routes still accessible
- [ ] API responses unchanged
- [ ] Database schema compatible

### Phase 2: Implement (varies)
[Write code]

### Phase 3: Evaluate
Run: /eval check add-authentication

### Phase 4: Report
EVAL REPORT: add-authentication
==============================
Capability: 5/5 passed (pass@3: 100%)
Regression: 3/3 passed (pass^3: 100%)
Status: SHIP IT