원클릭으로 Manus에서 모든 스킬 실행

run-iteration-eval

스타11

포크1

업데이트2026년 6월 1일 11:17

Run the Logic-Lens content-eval pipeline for one iteration and produce a scored summary.json — use to measure a skill change. Wraps scripts/run-content-evals.sh (runner, costs tokens) and scripts/grade-iteration.py (grader, free, re-runnable). ALWAYS sync the plugin cache first. Use when the user wants to "run the evals", "score this iteration", "measure the skill change", "smoke-test before the full run", or "re-grade existing outputs".

설치

Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.

Manus에서 실행

출처

hyhmrright

hyhmrright/logic-lens

GitHub 저장소 열기 Creator 저장소 보기

다운로드

Manus에서 실행

run-iteration-eval

Measures a skill change by running the content cases in evals/content/v2/evals-v2.json through claude -p and grading the outputs. Outputs land in skills-workspace/iteration-<TAG>/.

The runner and grader are split on purpose: running calls Claude and costs tokens; grading is pure regex Python and is free to re-run on outputs that already exist. Never re-run the runner just to re-score — re-grade instead.

Steps

Sync the cache first — non-negotiable. The runner loads the skill from the plugin cache, not skills/. Run the sync-skill-cache skill (or its script directly). If you skip this, the eval grades the previously-published skill and the entire run is wasted:
```
bash .claude/skills/sync-skill-cache/scripts/sync-cache.sh
```

Pick a scope. Full runs cost real tokens; scope down while iterating:

SMOKE=1 bash scripts/run-content-evals.sh              # one case per mode (~$0.10) — fast sanity
CASES="200 201 202" bash scripts/run-content-evals.sh  # only the cases a diagnosis flagged
TAG=myfix bash scripts/run-content-evals.sh            # full run, named tag
bash scripts/run-content-evals.sh                      # full run, tag = git short SHA

The runner is idempotent — a case with an existing output.md is skipped. Delete the eval-<id>/ dir to force a re-run of that case.

Read summary.json in the iteration dir. It carries overall pass rate plus the per-mode and per-subscore (logic vs format) breakdown. The logic subscore reflects reasoning quality; format reflects Output-Skeleton compliance and is the historical bottleneck with high single-run variance. Judge a change on the right subscore — a format wobble is not a reasoning regression.
Re-grade without re-running (free) after editing the grader or to recompute on existing outputs:
```
python3 scripts/grade-iteration.py skills-workspace/iteration-<TAG>
```
Single-iteration grade without the full suite — when you already have outputs and only want the score table, grade-iteration.py <iteration-dir> is the cheapest path (see scripts/README.md).

Variance caveat

logic-review single-run scores are variance-dominated (see project memory). One run is a signal, not a verdict — for a decision near the margin, run the affected cases 2–3× or widen the case set before concluding a change helped or hurt. Hand the result to iteration-guard for the ship/rollback call rather than eyeballing a single number.

이 저장소의 다른 Skills

같은 저장소

iterate-skill

hyhmrright/logic-lens

Run the Logic-Lens skill-improvement loop end to end — baseline → diagnose failures → edit → sync cache → re-eval → verify net gain → iterate until clean. Use whenever the goal is to RAISE a skill's eval score or fix a failing eval mode: "improve logic-review", "the format compliance is failing, fix it", "iterate on this skill until the evals pass", "raise the score", "re-run the loop on the latest failures", "run another iteration", "tune the skill description / disambiguation table against the evals". Also use to RESUME a prior loop ("continue improving from where we left off", "do another pass", "iterate further"). Do NOT use for: a one-off question about a skill, shipping a release (use bump-version), or scaffolding a brand-new skill (use new-skill).

2026-06-0111

sync-skill-cache

hyhmrright/logic-lens

Sync the Logic-Lens working-copy skills/ into the installed plugin cache so content-evals test the EDITED skill, not the last published one. ALWAYS run this after editing any skills/**/SKILL.md or guide/_shared file and BEFORE running content-evals — otherwise the eval silently grades stale content and every token is wasted. Also use to verify the cache is in sync ("is the cache fresh?", "did my edit take effect in evals?", "check skill cache drift").

2026-06-0111

logic-review

hyhmrright/logic-lens

Find logic bugs in a single file or function via semi-formal execution tracing (Premises → Trace → Divergence → Trigger → Remedy). Trigger when a user shares code and suspects something is wrong without naming a concrete failure — phrases like "review this", "does this look right", "check this function", "audit this code", "tests pass but prod fails". SCOPE HARD RULE: one file or one function only. For a directory or whole module use logic-health; for a confirmed failure (stack trace, failing test, specific wrong value) use logic-locate; for two versions use logic-diff; for repo-wide autonomous fixing use logic-fix-all. Do NOT trigger for: style/formatting, security scanning, performance, test generation, architecture or design questions.

2026-05-3011

bump-version

hyhmrright/logic-lens

Bump the Logic-Lens version across all six metadata locations at once (package.json, the four plugin manifests, and the README badge), then validate. Use when cutting a release or when `npm run validate` reports a version mismatch.

2026-05-2811

new-skill

hyhmrright/logic-lens

Scaffold a new logic-* skill in the Logic-Lens repo and wire it into every place a skill must be registered, so no step is missed. Use when adding a seventh (or later) skill to Logic-Lens.

2026-05-2811

logic-diff

hyhmrright/logic-lens

Compare two code versions for semantic equivalence via semi-formal tracing of both versions side-by-side. Trigger when the user shares a refactor, rewrite, migration, or A/B implementation and wants to confirm behavior is unchanged — "did I break anything", "is this equivalent", "are these equivalent", "semantically equivalent", "are these two implementations semantically equivalent", "check my refactor", "same behavior after the change?", "does my rewrite produce the same output", "switched from X to Y — same results?". SCOPE HARD RULE: requires two code versions (A and B). A single version for bug-finding uses logic-review; one version + a failing test uses logic-locate; explaining what one piece of code does uses logic-explain; codebase audit uses logic-health. Do NOT trigger for: single-version review, performance comparison, design-quality comparison, or "which is better-written" questions.

2026-05-1511

name	run-iteration-eval
description	Run the Logic-Lens content-eval pipeline for one iteration and produce a scored summary.json — use to measure a skill change. Wraps scripts/run-content-evals.sh (runner, costs tokens) and scripts/grade-iteration.py (grader, free, re-runnable). ALWAYS sync the plugin cache first. Use when the user wants to "run the evals", "score this iteration", "measure the skill change", "smoke-test before the full run", or "re-grade existing outputs".
disable-model-invocation	true

run-iteration-eval

Measures a skill change by running the content cases in evals/content/v2/evals-v2.json through claude -p and grading the outputs. Outputs land in skills-workspace/iteration-<TAG>/.

Steps

Sync the cache first — non-negotiable. The runner loads the skill from the plugin cache, not skills/. Run the sync-skill-cache skill (or its script directly). If you skip this, the eval grades the previously-published skill and the entire run is wasted:
```
bash .claude/skills/sync-skill-cache/scripts/sync-cache.sh
```

Pick a scope. Full runs cost real tokens; scope down while iterating:

SMOKE=1 bash scripts/run-content-evals.sh              # one case per mode (~$0.10) — fast sanity
CASES="200 201 202" bash scripts/run-content-evals.sh  # only the cases a diagnosis flagged
TAG=myfix bash scripts/run-content-evals.sh            # full run, named tag
bash scripts/run-content-evals.sh                      # full run, tag = git short SHA

The runner is idempotent — a case with an existing output.md is skipped. Delete the eval-<id>/ dir to force a re-run of that case.

Read summary.json in the iteration dir. It carries overall pass rate plus the per-mode and per-subscore (logic vs format) breakdown. The logic subscore reflects reasoning quality; format reflects Output-Skeleton compliance and is the historical bottleneck with high single-run variance. Judge a change on the right subscore — a format wobble is not a reasoning regression.
Re-grade without re-running (free) after editing the grader or to recompute on existing outputs:
```
python3 scripts/grade-iteration.py skills-workspace/iteration-<TAG>
```
Single-iteration grade without the full suite — when you already have outputs and only want the score table, grade-iteration.py <iteration-dir> is the cheapest path (see scripts/README.md).