Menu
Run one iteration of the doc improvement loop: generate or accept questions, answer them in parallel, verify, grade, diagnose, fix the docs, and re-evaluate until convergence.
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Baseado na classificação ocupacional SOC
Install the MadAgents agent system into a folder on this machine, to be run without a container. Use when the user wants MadAgents outside Apptainer — in a project folder, on a login node, or anywhere they already have MadGraph — or wants to refresh an existing install.
Find documentation problems that caused agent mistakes — gaps, inaccuracies, or ambiguities. Uses grade context to guide diagnosis.
Edit MadGraph documentation with parallel review: style, quality, and factual verification. Revises until all checks pass or max rounds reached.
Generate evaluation questions about MadGraph and related tools with verified reference answers, using web research for real-world use cases.
Get a writable copy of the MadGraph documentation.
Extract factual claims from text and verify each one using execution, source inspection, or physics reasoning.
| name | train-docs |
| description | Run one iteration of the doc improvement loop: generate or accept questions, answer them in parallel, verify, grade, diagnose, fix the docs, and re-evaluate until convergence. |
Run one cycle of the documentation improvement loop with parallel question processing.
$ARGUMENTS
Parse key=value pairs from the input. All are optional — use defaults if not specified.
focus=NLO matchingrequirements=must involve Pythia8Answerers and verifiers must be spawned as teammates (via TeamCreate), NOT as subagents (via the Agent tool). Teammates run as independent Claude Code sessions that can dispatch their own subagents. Subagents cannot dispatch further subagents — so a verifier spawned as a subagent would be unable to delegate to madgraph-operator, script-operator, etc.
agent_type: "orchestrator" — full MadAgents orchestrator.agent_type: "verifier" — MadAgents orchestrator with built-in verification workflow and restricted subagent roster.Teammates start in /output (the lead's working directory). Always specify absolute paths (e.g., /output/train/q000/) when telling a teammate where to write its output.
Create ./train/ with a subdirectory per question: q000/, q001/, etc. Each question directory gets: answer/, verify/, grade/, diagnose/.
Execute all steps in sequence without pausing for confirmation — the user will be consulted only where the skill explicitly says "Discuss with user."
If questions were provided, use them. Otherwise invoke /generate-questions with the count, focus, and requirements parameters.
The generated questions include reference_answer fields. These are unverified and must not be visible to answerer subagents.
Write the full questions data (including reference_answer fields) to ./train/questions.json. This file will be hidden from answerers and re-answerers via hide_paths.
Number each question: q000, q001, q002, ...
Before answering, call hide_paths to hide the questions file (which contains reference answers answerers must not see). Pass: /output/train/questions.json. Save the returned token.
Create a team for the answerers. For each question, add a teammate to this team with agent_type: "orchestrator". Give it the question and its working directory. All teammates work in parallel.
{question text}
Work in
/output/train/q{N}/answer/. Ensure your answer is complete and concrete before finishing — all steps, commands, and parameters should be fully specified.
After ALL teammates finish, shut down the team. Before calling TeamDelete, kill all tmux panes belonging to the team's teammates (they may linger after the teammate finishes). Then:
show_paths with the saved token to restore the questions file.get_transcript with team_name, agent_name, output_path set to /output/train/q{N}/answer/transcript.json, and answer_path set to /output/train/q{N}/answer/response.md. This extracts both the transcript and the answer in one call — no need to save the response manually.Create a team for the verifiers. For each question, add a teammate to this team with agent_type: "verifier". Tell each to work in its question directory:
Work in
/output/train/q{N}/verify/. Verify the factual claims in the answer at/output/train/q{N}/answer/response.md. Also check for user-facing written artifacts the answerer produced (answer files, scripts, code) in/output/train/q{N}/answer/.If a claim database exists at
/output/.eval/claim_db.json, reuse the exact wording of known claims where the same fact appears. Use known claims as context, not automatic skips. Re-verify if there is any doubt. Do NOT generalize from known claims across contexts.Output: Write verdicts to
/output/train/q{N}/verify/verdicts.json— a JSON array where each object has:claim,correct(true/false/null),method(execution/inspection/physics_reasoning/null),evidence(list),explanation.Also write remember selections to
/output/train/q{N}/verify/remember.json— a flat array of 0-based indices marking which newly verified claims are genuinely new and worth caching. Skip inconclusive claims (correct=null).
After ALL verifier teammates finish, shut down the team. Before calling TeamDelete, kill all tmux panes belonging to the team's teammates.
Consolidate claim database: For each question, read remember.json to find which newly verified claims should be cached. Append them to /output/.eval/claim_db.json (load existing DB or start with [], assign incremental IDs, add claim, correct, method, evidence, explanation, count: 1 fields). Deduplicate: skip any claim whose text already exists in the DB (exact match). This is done sequentially to avoid race conditions.
For each question simultaneously, invoke a subagent (no worktree isolation needed — grading and diagnosis only read files and write JSON). Tell each:
Grade and diagnose this answer. Read the inputs, then:
Step 1 — Grade: Assess the answer against the verification verdicts.
Inputs:
- Question: {question text}
- Verification summary: {count correct, incorrect, inconclusive claims}
- Verdicts:
/output/train/q{N}/verify/verdicts.json- Answer:
/output/train/q{N}/answer/response.md- Transcript:
/output/train/q{N}/answer/transcript.json(for assessing workflow and efficiency)Output: Write to
/output/train/q{N}/grade/grade.json:{"grade": "CORRECT|INCORRECT|INCONCLUSIVE", "tags": [], "explanation": "..."}Grades:
- CORRECT: The answer correctly answers the question. Errors in reasoning or wrong intermediate facts do not affect this — those are captured by tags.
- INCORRECT: Wrong final answer, misleading conclusions, refusal, or no meaningful response.
- INCONCLUSIVE: Verification results are insufficient to determine correctness. Use only when you genuinely cannot make the call.
Tags (zero or more, independent of grade):
- has_mistakes: Wrong facts, flawed reasoning, or incorrect intermediate steps that do not invalidate the final answer.
- inefficient: The agent spent significant effort that better documentation would have prevented. Read the transcript to assess this. Web searches and source inspection for question-specific resources are expected — flag only when extra effort traces back to a documentation problem. If no transcript file is available, do not assign this tag.
- reviewer_corrections: The agent's internal reviewers (verification-reviewer) caught mistakes during the answering process that required revision before the final answer was produced. Even if the final answer is correct, this indicates the docs were unclear or misleading enough to cause initial errors. Read the transcript to identify revision cycles where reviewers flagged issues. If no transcript file is available, do not assign this tag.
Step 2 — Diagnose (only if grade is INCORRECT or any tag is present):
Identify documentation issues that caused or contributed to the problems.
- Read
/output/train/q{N}/verify/verdicts.jsonfor incorrect claims- Read
/output/train/q{N}/answer/response.mdand/output/train/q{N}/answer/transcript.json— look for revision cycles where reviewers flagged issues- Check whether
/madgraph_docs/covers the topic correctly and clearly- If the docs are missing, wrong, or ambiguous — write a finding. If the docs already cover it correctly — skip (model error, not actionable).
- If the
inefficienttag is present, also look for unnecessary effort in the transcript caused by doc gaps: web searches for basic MadGraph operations, trial-and-error with commands, reading MG5 source code for info the docs should provide.- If the
reviewer_correctionstag is present, identify what the reviewers flagged and trace it back to a documentation problem. What did the agent get wrong initially? Was the doc misleading, incomplete, or ambiguous on that point?Rules: Identify root causes, not symptoms. Findings must be generalizable. Recommendations should be practical, not sweeping rewrites. If no issues are documentation problems, write empty lists.
Output: Write to
/output/train/q{N}/diagnose/diagnoses.json:{"doc_gap": [...], "doc_incorrect": [...], "doc_ambiguous": [...]}Each finding has
problem,correct_info, andrecommendationfields. Include all three categories. Empty lists are fine. If the answer is CORRECT with no tags, write empty lists for all categories.
After ALL agents finish, proceed.
Read each grade.
If ALL questions are CORRECT with empty tags → stop here. The docs supported clean correct answers for every question. Report the results to the user and skip to the Summary.
Collect the questions that need improvement: grade is INCORRECT, or any tag is present (has_mistakes, inefficient, or reviewer_corrections). INCONCLUSIVE with no tags does NOT need improvement.
Read diagnoses.json files only from questions that need improvement (from step 5). Merge findings:
doc_gap, doc_incorrect, doc_ambiguousWrite merged findings to ./train/diagnoses_merged.json.
Read ./train/diagnoses_merged.json. For each question that needed improvement, read its reference_answer from ./train/questions.json. Reference answers are unverified — they may contain errors. Tell the doc-editor to use them as inspiration for understanding what the docs should cover, not as ground truth.
Tell the doc-editor to prefer minimal, topic-general changes over problem-specific patches. Write about the topic naturally — not about the failure that motivated the change. If more specificity is needed, the review cycle will ask for it.
Apply documentation improvements using the /edit-docs workflow, passing the merged diagnoses and reference answer context. The edit-docs workflow handles:
Do NOT skip or short-circuit the edit-docs revision loop. All three reviewers must pass before applying. Once all pass, apply the changes so re-answerers see the updated docs.
Unverified claims warning: After the verifier finishes, read its verdicts file. If any claim has correct: null (inconclusive — verification failed to confirm or deny), STOP and print a warning to the user listing each unverified claim with its explanation. Ask the user how to proceed: keep the change as-is, drop the unverified claim, or rewrite the relevant section. Do NOT apply the changes until the user responds.
Re-answer only the questions that failed (from step 5). The answerer teammates must not see previous evaluation artifacts (grades, verdicts, diagnoses, previous answers).
Before re-answering, call hide_paths to move all eval artifacts out of the container's view. Pass every relevant path for each failed question N (use absolute container paths for MCP tools):
/output/train/questions.json/output/train/q{N}/answer/output/train/q{N}/verify/output/train/q{N}/grade/output/train/q{N}/diagnose/output/train/diagnoses_merged.jsonSave the returned token.
Answer: Create a team for the re-answerers. For each failed question, add a teammate to this team with agent_type: "orchestrator" (fresh context — it will see the updated docs). Tell each the question and its working directory. All teammates work in parallel.
{question text}
Work in
/output/train/q{N}/answer/. Ensure your answer is complete and concrete before finishing — all steps, commands, and parameters should be fully specified.
After ALL teammates finish, shut down the team. Before calling TeamDelete, kill all tmux panes belonging to the team's teammates. Then:
show_paths with the saved token to restore the hidden artifacts.get_transcript with team_name, agent_name, output_path set to /output/train/q{N}/answer/transcript.json, and answer_path set to /output/train/q{N}/answer/response.md.Then re-run ALL three verification steps for each re-evaluated question (same as steps 3–4):
agent_type: "verifier". Dispatch one teammate per re-evaluated question with the same instructions as step 3.For each re-evaluated question, check its new grade:
CORRECT with no tags → mark as converged (exclude from further iterations)If ALL re-evaluated questions converged → doc changes confirmed. Discuss results with user.
If some questions still fail → keep the applied doc changes (they may have partially helped) and repeat from step 6 with only the non-converged questions. Read diagnoses only from the latest iteration's non-converged questions — do not re-merge stale diagnoses from earlier iterations. Max 2 iterations total.
If questions still fail after 2 iterations → stop and report the remaining issues to the user. Doc changes from all iterations are kept.
After completion, report: