القائمة
Structured evaluation using a swarm of small agents. The orchestrator assesses the target, designs a test plan mixing verifiable questions, trick questions, and navigation tasks, proposes it for user approval, then dispatches and synthesises scored results.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
Identify changed code and dispatch reuse, quality, and efficiency reviewer agents in parallel; return the raw findings list.
Review changed code for reuse, quality, and efficiency, and produce a triaged list of findings (no edits). Use after writing code, for simpler code with fewer bugs and better performance.
Provides guidance on good software design principles, with trade-off calibration, examples, and common failure modes. Applies when making decisions while writing, designing, or refactoring code, or evaluating trivial code for clarity, maintainability, and common mistakes.
Reviews code with a team against established software design principles, and returns triaged findings (no edits). Use when evaluating code, or after writing or refactoring complex code, systems or technical designs: for code that is clearer, easier to maintain, and avoids common mistakes.
Shared setup for shaping sessions. Determines if an artefact exists or needs creating, creates it if needed, starts the transcript, reads the type taxonomy. Called by the parent router or by any sub-skill invoked directly.
Shape an artefact through structured Q&A. Routes to the right sub-skill: brainstorm (new/unclear artefacts), refine (open decisions), or discover (exploration-driven artefacts like People and Ideas).
| name | swarm-test:evaluate |
| description | Structured evaluation using a swarm of small agents. The orchestrator assesses the target, designs a test plan mixing verifiable questions, trick questions, and navigation tasks, proposes it for user approval, then dispatches and synthesises scored results. |
Cheap, parallel quality gate. Haiku agents are small and fast but not brilliant — if they can't figure something out from the work product alone, a real user or contributor will struggle too. Their confusion is the measuring instrument.
The orchestrator (you) reads the target, assesses what's testable, and designs a test plan by composing from the scenario types and test shapes below. The user approves the plan before dispatch.
Present the plan to the user for approval before dispatching. The proposal has four parts, in this order:
1. Goal — What are we testing, and what does success look like? One or two sentences.
2. Approach — How do you propose to test it? What you'll do, concretely — the shape, the mix, the agent count. Keep it short and plain. No framework jargon (the user doesn't know or care about "comprehension scenarios" or "counter-factual"). Describe what the agents will do in plain language.
3. Reasoning — Write exactly 2-3 bullets explaining why this approach is effective for this specific target. Each bullet: one sentence, under 15 words. Name the mechanism, then what it proves or what risk it catches.
4. Scenarios — The specific tests. Factual scenarios show expected answers inline. Counter-factuals show (expected: does not exist) — don't label them as counter-factuals, but do mark the expected answer so the user can verify. Group scenarios whichever way makes them easiest to scan (by theme, by area, or flat list — whatever fits).
Each part is doing one job. The user reads the goal to check scope, the approach to understand what's going to happen, the reasoning to evaluate whether it's well-designed, and the scenarios to approve the specifics.
The user's request guides the orchestrator:
The orchestrator uses judgment, not rules. A documentation test might be pure comprehension if the docs are procedural, or mostly factual if the docs are reference material. The user's intent and the target's nature determine the mix.
Each type is defined once. The orchestrator composes from these. Dispatch all agents using the smallest available model (in Claude Code: model: haiku).
Tests whether agents can navigate, find, and understand information. Each scenario is a unique task a real person would try to accomplish.
When to use: Always applicable. The baseline layer.
Agent count: 1 agent per scenario (adjust count to the target's scope).
Prompt template:
You are testing [docs/code/design] quality. You are in [repo] at [path].
Your task: [specific scenario]. Start by reading [entry point] to orient
yourself, then find the answer using only the [docs/code/etc] available.
Report:
1. The path you took (which files, in order)
2. What you found (or couldn't find)
3. Rate navigation: DIRECT / EXPLORED / STRUGGLED / LOST
4. One concrete suggestion to improve what you tested
Scoring (navigation):
| Score | Label | Meaning |
|---|---|---|
| 3 | DIRECT | Straight to it |
| 2 | EXPLORED | Some searching, got there |
| 1 | STRUGGLED | Many wrong turns |
| 0 | LOST | Couldn't find it or gave up |
Aggregate: (sum of scores) / (scenarios × 3) as a percentage.
Design guidance:
Tests whether agents can retrieve a specific, verifiable answer from the source material.
When to use: When the orchestrator can derive single correct answers from the target. Reference docs, config files, taxonomies, APIs with defined behaviour — all good candidates. Procedural or subjective content is not.
Agent count: 1 agent per question for a quick check. For meaningful measurement, note in the test plan that a Monte Carlo approach (separate skill) can run the same questions at scale.
Prompt template:
You are testing [docs/code/design] quality. You are in [repo] at [path].
Your task: [specific question]. Start by reading [entry point] to orient
yourself, then find the answer using only the [docs/code/etc] available.
Reply in this exact format:
ANSWER: [your specific answer]
CONFIDENCE: [HIGH / MEDIUM / LOW]
PATH: [files you read, in order]
SUGGESTION: [one concrete improvement]
Scoring (outcome): The orchestrator compares the agent's answer against the expected answer and scores as a percentage — how much of the expected answer was captured? "Got 4 of 5 status values" = 80%. This gives more signal than a coarse pass/fail across runs.
Design guidance:
Tests whether agents confabulate — inventing answers for things that don't exist. Uses the same prompt template as factual — the agent has no hint that absence is a valid answer. The orchestrator knows the answer doesn't exist; the agent doesn't.
When to use: Whenever factual scenarios are used. Counter-factuals are the control group — without them, you can't distinguish "the agent found the right answer" from "the agent is good at sounding confident."
Agent count: Same as factual — paired with them in the test plan.
Scoring: The orchestrator evaluates the response — this tests the material's clarity, not the agent's behaviour:
Design guidance:
How scenarios are arranged and dispatched.
The default. A mix of scenario types dispatched to independent agents. Each agent gets one scenario.
When to use: Most tests. Documentation quality, code comprehension, smoke tests, fuzz tests.
Agent count: Sum of all scenarios across types. Typical range: 8-14.
Output — comprehension scenarios:
┌─────┬──────────────────────────────────────────┬────────────┬──────────────────────────────────────┐
│ # │ Scenario │ Navigation │ Path / Finding │
├─────┼──────────────────────────────────────────┼────────────┼──────────────────────────────────────┤
│ 1 │ Find how to add a temporal type │ STRUGGLED │ index → standards → naming → colours │
│ │ │ │ → extending (4 wrong turns) │
├─────┼──────────────────────────────────────────┼────────────┼──────────────────────────────────────┤
│ 2 │ How to archive a living artefact │ DIRECT │ index → archiving standards │
└─────┴──────────────────────────────────────────┴────────────┴──────────────────────────────────────┘
Navigation: 63% (19/30)
Output — factual scenarios:
┌─────┬──────────────────────────────┬───────────────────────────────────┬─────────────────────────────────────────────┬───────┐
│ # │ Question │ Expected Answer │ Agent Answer │ Score │
├─────┼──────────────────────────────┼───────────────────────────────────┼─────────────────────────────────────────────┼───────┤
│ 3 │ Research naming pattern │ yyyymmdd-research~{Title}.md │ yyyymmdd-research~{Title}.md │ 100% │
├─────┼──────────────────────────────┼───────────────────────────────────┼─────────────────────────────────────────────┼───────┤
│ 4 │ Design status values │ 8 values (proposed...rejected) │ Got 6 of 8, missed superseded & parked │ 75% │
└─────┴──────────────────────────────┴───────────────────────────────────┴─────────────────────────────────────────────┴───────┘
Factual accuracy: 88% (avg across 4 questions)
Output — counter-factual scenarios:
┌─────┬──────────────────────────────┬─────────────────────────────────────────────────────┬───────────────────┐
│ # │ Question │ Agent Answer │ Boundary │
├─────┼──────────────────────────────┼─────────────────────────────────────────────────────┼───────────────────┤
│ 5 │ Vault sync schedule config │ sync_interval in preferences (fabricated) │ Confabulation │
├─────┼──────────────────────────────┼─────────────────────────────────────────────────────┼───────────────────┤
│ 6 │ Plugin rename process │ "I couldn't find a rename process" │ Clear boundary │
├─────┼──────────────────────────────┼─────────────────────────────────────────────────────┼───────────────────┤
│ 7 │ Retrospective naming pattern │ "No retrospective type, but maybe captures?" │ Ambiguous boundary│
└─────┴──────────────────────────────┴─────────────────────────────────────────────────────┴───────────────────┘
Counter-factual: 1 clear boundary, 1 ambiguous, 1 confabulation
Two groups receive identical questions. Only one variable differs between them. Tests whether the variable matters.
When to use: When the user is comparing two approaches, configurations, bootstrap methods, or documentation structures.
Agent count: Questions x 2 (one per group). Typical range: 12-20 (6-10 questions, 2 groups). The question set should include both factual and counter-factual.
Output:
┌─────┬──────────────────────────────┬───────────────────────────────────┬──────────────────┬──────────────────────────────────┐
│ # │ Question │ Correct Answer │ Group A │ Group B │
├─────┼──────────────────────────────┼───────────────────────────────────┼──────────────────┼──────────────────────────────────┤
│ 1 │ Research naming pattern │ yyyymmdd-research~{Title}.md │ CANNOT DETERMINE │ yyyymmdd-research~{Title}.md │
├─────┼──────────────────────────────┼───────────────────────────────────┼──────────────────┼──────────────────────────────────┤
│ 2 │ Vault sync schedule config │ DOES NOT EXIST │ CANNOT DETERMINE │ DOES NOT EXIST │
├─────┼──────────────────────────────┼───────────────────────────────────┼──────────────────┼──────────────────────────────────┤
│ 3 │ Plugin rename process │ DOES NOT EXIST │ Has rename tool │ DOES NOT EXIST │
│ │ │ │ (fabricated) │ │
└─────┴──────────────────────────────┴───────────────────────────────────┴──────────────────┴──────────────────────────────────┘
┌─────┬──────────────────────────────┬──────────┬──────────┐
│ # │ Question │ Group A │ Group B │
├─────┼──────────────────────────────┼──────────┼──────────┤
│ 1 │ Research naming pattern │ ✗ │ ✓ │
├─────┼──────────────────────────────┼──────────┼──────────┤
│ 2 │ Vault sync schedule config │ ✗ │ ✓ │
├─────┼──────────────────────────────┼──────────┼──────────┤
│ 3 │ Plugin rename process │ ✗✗ │ ✓ │
├─────┼──────────────────────────────┼──────────┼──────────┤
│ │ Accuracy │ 0/3 0% │ 3/3 100% │
└─────┴──────────────────────────────┴──────────┴──────────┘
✗ = incorrect/unknown ✗✗ = fabricated answer (confabulation)
Design guidance:
Group findings into patterns, not individual complaints:
After synthesis, present a concise summary of actionable findings — what's broken, what's risky, and what to do about it. This is the section the user reads first. Structure as a numbered list of findings, each with a one-line description and a concrete recommendation. Example:
parked appears alongside adopted in the lifecycle table but only adopted has a +Status/ folder. Recommend making the distinction explicit in the taxonomy.After presenting findings, ask the user how they'd like to proceed:
Agent failures: If an agent times out, crashes, or returns garbage, drop the result and note it in findings. Do not retry — a failure is data (the scenario may be too complex for a single haiku agent).
When fixing:
When re-evaluating: use a smaller, focused round (4-6 agents) targeting the fixed areas. One iteration is usually enough. Two if the first round found structural problems. Never more than three.