一键在 Manus 中运行任何 Skill

eval-enforcement

星标3

分支0

更新时间2026年7月13日 21:18

Gate any new MCP tool, skill, agent persona, voice profile, or LLM-backed surface behind a behavior eval before shipping. Also enforces skill shape, memory architecture, and workflow-script placement. Invoke when adding, renaming, or relaxing any of the above.

安装

用 Codex 或 Claude 帮你安装复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。

在 Manus 中运行

来源

ethos71

ethos71/forget-the-thunderdome

打开 GitHub 仓库查看创作者相关仓库

下载

在 Manus 中运行

Eval Enforcement

Owns the four contracts dom pins on every consumer project: behavior evals, skill structure, memory architecture, and workflow-script placement. This skill is dom's own — it both demonstrates the shape the universal test_skill_shape.py enforces, and routes incoming "can we ship this?" questions to the right answer.

The work itself is done by the @dom-eval-enforcer agent (.github/agents/dom-eval-enforcer.md). This skill is the discovery surface — when an agent or human is unsure whether a change is ship-ready, they land here.

Process

Classify the change. Is the diff adding any of:
- An @mcp.tool() registration? → behavior eval required.
- A new .github/skills/<name>/ directory? → shape eval gates it.
- A new agent persona / voice doc / system prompt? → eval required.
- A new LLM-backed route or surface? → eval required.
- A move of docs/robots/MEMORY.md? → ship-block.
- A workflow that calls a script? → placement check.
Run the relevant gate.
- scripts/check-skill-shape.sh — runs test_skill_shape.py on the current repo.
- scripts/check-memory.sh — runs test_memory_architecture.py.
- scripts/check-policy.sh — runs both, plus any project-local evals under .github/evals/.
Consult @dom-eval-enforcer with the diff if the change touches a behavior contract (persona text, tool semantics, voice profile). The agent returns one of: "ship it", "add an eval first", "don't ship". See references/enforcer-verdicts.md for verdict shapes.
Write or update the eval. Land in the right bucket under .github/evals/<bucket>/: ai/ (LLM surfaces), tools/ (MCP tools), skills/ (skill shape), voice/ (personas), ui/ (routes). Reference: references/eval-buckets.md.
Re-run until pass/skip is clean. Exit-code-only evidence is not acceptable — the eval must compare against real data content.

Key Files

.github/agents/dom-eval-enforcer.md — Policy agent persona.
.github/evals/skills/test_skill_shape.py — Universal skill-shape eval.
.github/evals/skills/test_memory_architecture.py — Universal memory eval.
docs/eval-policy.md — The formal contract.
docs/robots/MEMORY.md — Mid-term tier records prior enforcement decisions.

Related Skills

(none yet — dom currently ships exactly one skill of its own; consumer projects like smartballz layer dozens more)

Hard Rules

An eval that returns 0 without comparing real data is dead weight. Delete it or fix it; don't ship it.
A skill without a references/*.md and a scripts/*.sh is a stub, not a skill. The shape eval will fail.
docs/robots/MEMORY.md is sacred — renaming or relocating it is a ship-block.
Workflow-only helpers belong in .github/scripts/. Operator/cron scripts belong in bin/. Don't mix.
Never soften an eval threshold to unblock a ship. If the eval fails on current data, fix the data or the behavior.

同仓库更多 Skills

同仓库

dom-delegate

ethos71/forget-the-thunderdome

@dom's task-delegation system. Use when @dom needs to hand work to a subordinate bot (@smartballz, @job, @grof2, @nyx, etc.) — it writes a structured task into that bot's root TODO.md or memory; the bot implements it. @dom NEVER implements. Also use bot-side to read your own delegated tasks (dom intake). Commands: dom delegate/tasks/intake/bots.

2026-07-133

dom-usage

ethos71/forget-the-thunderdome

Report AI token usage, cost, and latency per model and audit that Ollama routing is being enforced. Use when you want to know what tokens/cost went to which model (Ollama vs Haiku/Sonnet/Opus via OpenRouter), what Ollama saved vs paid tiers, the cost-vs-time tradeoff, or to verify the four-tier waterfall is keeping cheap/free work on local Ollama. Invoked as `dom usage` (named dom-usage to avoid colliding with Claude Code's built-in /usage). Reads the log that delegate_task/ask_local write.

2026-07-133

form-parser

ethos71/forget-the-thunderdome

Extracts and classifies form fields from job posting pages. Identifies form system (Workday, Lever, Greenhouse, LinkedIn, custom HTML). Maps resume data to standard fields. Suggests answers for custom fields. Outputs JSON schema showing what the user needs to fill in.

2026-07-133

job-discovery

ethos71/forget-the-thunderdome

Discovers new job postings matching the user's criteria from major job boards. Runs daily to find roles matching profile.yaml's search section (target_roles, keywords, min_salary, remote_only). Outputs to the discovered_jobs table in data/job_tracker.db for curation.

2026-07-133

job-search-validation

ethos71/forget-the-thunderdome

How to validate a job posting URL before adding it to the pipeline or recommending it to the user. Built from real failures during search sessions — HTTP 200 ≠ active posting, and most ATS systems serve empty pages to scrapers.

2026-06-113

name	eval-enforcement
description	Gate any new MCP tool, skill, agent persona, voice profile, or LLM-backed surface behind a behavior eval before shipping. Also enforces skill shape, memory architecture, and workflow-script placement. Invoke when adding, renaming, or relaxing any of the above.

Eval Enforcement

Process

Classify the change. Is the diff adding any of:
- An @mcp.tool() registration? → behavior eval required.
- A new .github/skills/<name>/ directory? → shape eval gates it.
- A new agent persona / voice doc / system prompt? → eval required.
- A new LLM-backed route or surface? → eval required.
- A move of docs/robots/MEMORY.md? → ship-block.
- A workflow that calls a script? → placement check.
Run the relevant gate.
- scripts/check-skill-shape.sh — runs test_skill_shape.py on the current repo.
- scripts/check-memory.sh — runs test_memory_architecture.py.
- scripts/check-policy.sh — runs both, plus any project-local evals under .github/evals/.
Consult @dom-eval-enforcer with the diff if the change touches a behavior contract (persona text, tool semantics, voice profile). The agent returns one of: "ship it", "add an eval first", "don't ship". See references/enforcer-verdicts.md for verdict shapes.
Write or update the eval. Land in the right bucket under .github/evals/<bucket>/: ai/ (LLM surfaces), tools/ (MCP tools), skills/ (skill shape), voice/ (personas), ui/ (routes). Reference: references/eval-buckets.md.
Re-run until pass/skip is clean. Exit-code-only evidence is not acceptable — the eval must compare against real data content.

Key Files

.github/agents/dom-eval-enforcer.md — Policy agent persona.
.github/evals/skills/test_skill_shape.py — Universal skill-shape eval.
.github/evals/skills/test_memory_architecture.py — Universal memory eval.
docs/eval-policy.md — The formal contract.
docs/robots/MEMORY.md — Mid-term tier records prior enforcement decisions.

Related Skills

(none yet — dom currently ships exactly one skill of its own; consumer projects like smartballz layer dozens more)

Hard Rules

An eval that returns 0 without comparing real data is dead weight. Delete it or fix it; don't ship it.
A skill without a references/*.md and a scripts/*.sh is a stub, not a skill. The shape eval will fail.
docs/robots/MEMORY.md is sacred — renaming or relocating it is a ship-block.
Workflow-only helpers belong in .github/scripts/. Operator/cron scripts belong in bin/. Don't mix.
Never soften an eval threshold to unblock a ship. If the eval fails on current data, fix the data or the behavior.