Ejecuta cualquier Skill en Manus
con un clic

Ejecuta cualquier Skill en Manus con un clic

rerun-corpus

Estrellas3

Forks1

Actualizado12 de julio de 2026 a las 06:09

Audit and selectively re-run the deterministic analysis engine across all 26 corpus architectures. Default (no args) shows a staleness check — which reports are missing metadata, which arch types need engine fixes re-applied, and the current corpus score table. Pass --ai-only to re-run only AI/agentic architectures, --all to re-run everything, or a single arch name to re-run one. Use after engine changes (self_validation.py, ground_truth_generator.py, rapids_driven_controls.py) to confirm scores are consistent and no regressions landed.

Instalación

Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.

Ejecutar en Manus

Fuente

BerdTan

BerdTan/threatassessor

Abrir repositorio de GitHub Ver repositorios del creador

Descarga

Ejecutar en Manus

Ocupaciones relacionadasSOC

Basado en la clasificación ocupacional SOC

Analistas de garantía de calidad de software y probadoresOcupaciones informáticas y matemáticas·SOC 15-1253

Explorador de archivos

2 archivos

SKILL.md

readonly

name	rerun-corpus
description	Audit and selectively re-run the deterministic analysis engine across all 26 corpus architectures. Default (no args) shows a staleness check — which reports are missing metadata, which arch types need engine fixes re-applied, and the current corpus score table. Pass --ai-only to re-run only AI/agentic architectures, --all to re-run everything, or a single arch name to re-run one. Use after engine changes (self_validation.py, ground_truth_generator.py, rapids_driven_controls.py) to confirm scores are consistent and no regressions landed.
allowed-tools	Bash(python3:) Bash(source:)

Corpus Re-run

Audits /report/ against the current engine version and re-runs analysis where needed.

Re-run is deterministic only (python3 -m chatbot.main --gen-arch-truth) — no LLM calls, no Expert Review. Takes ~15–20 sec per architecture.

Run

# Staleness audit only (default — read-only, fast)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py

# Re-run AI/agentic architectures only (highest impact after engine changes)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --ai-only

# Re-run a single architecture
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py 21_agentic_ai_system

# Re-run all 26 architectures (~8 min)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --all

What the audit checks

Signal	Stale if
`run_id` / `run_ts`	Missing from `ground_truth.json` metadata (pre engine-metadata fix)
`pattern_sources`	Missing — engine couldn't show which analysis engines fired
`ai_system` validation	Arch type is `ai_system` but `run_ts` predates the self_validation + hop-coverage fix (2026-07-12)
Score floor	TATB overall < 70 (below Solid)

Output format

STALE  22_generic_name_with_ai_nodes  type=ai_system  no run_id  → needs re-run
OK     21_agentic_ai_system           type=ai_system  run_id=21_agentic_ai_system_20260712T060119
OK     05_legacy_flat_network         type=web_app    run_id=05_legacy_flat_network_20260712T...
META   03_aws_3tier                   type=web_app    no run_id  (metadata only — score unaffected)
...

Summary: 1 STALE (need re-run), 24 META (display only), 1 OK
Re-run stale:  python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --ai-only
Re-run all:    python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --all

After re-running, a TATB corpus table is printed with before/after scores.

When to run

After any change to self_validation.py, ground_truth_generator.py, rapids_driven_controls.py, per_node_ttp_mapper.py, or exhaustive_mitigation_mapper.py
After updating MITRE data (/update-data)
To confirm a corpus-wide score regression didn't land

Related skills

/tatb-corpus — score all architectures from existing reports (read-only)
/tatb-score — score a single architecture
/backfill-aivss — refresh AIVSS governance scores after a harness change

Más de este repositorio

mismo repositorio

tatb-score

BerdTan/threatassessor

Run TATB (TA Test Benchmark) on a single architecture and print the four-rubric scorecard — Threat-Relevant, TTP-Accurate, Risk-Defensible, Plan-Actionable. Uses the same scoring logic as the dashboard TATB tab. Pass an architecture name as argument, or omit for the most recent report. Read-only. ~3 seconds (includes MITRE API fetch).

2026-07-123

tatb-corpus

BerdTan/threatassessor

Score all architectures in /report/ across the four TATB rubrics and print a visual corpus map — bar table, heatmap, and pipeline stage attribution. Shows which architectures are Excellent/Solid/Weak/Draft on each dimension and identifies which pipeline stages to fix for the largest corpus-wide gain. No UI needed. ~10-15 seconds (MITRE fetch batched once).

2026-07-113

run-er

BerdTan/threatassessor

Run Expert Review (MoE critics + ScrumMaster) on any architecture that has an existing analysis. Three modes — sequential (3 core critics), full_moe (5 critics + SM, generates Plan-Actionable data), or single-critic (re-run one critic by name). Streams live critic results with scores, gaps, and final confidence. ~60–120s for full_moe, ~30–60s for sequential.

2026-07-113

tatb-loop

BerdTan/threatassessor

TATB feedback loop — observe corpus or single-arch TATB scores, diagnose weak signals, propose targeted pipeline stage fixes, gate on human approval, apply changes, re-run TATB to verify improvement, and log the before/after delta to docs/DECISIONS.md. The agent does NOT autonomously commit code; every code change requires explicit human approval.

2026-07-043

quick-test

BerdTan/threatassessor

Runs a fast ~15 second integration sanity check — verifies dependencies, rate limiter, MITRE data load (≥835 techniques), OpenRouter API key, API connection, and one real embedding call. Use at session start or after installing dependencies to confirm everything is working before coding.

2026-07-023

aivss-gate

BerdTan/threatassessor

Show per-critic AIVSS gate config, per-agent model assignments (HarnessModelGuardian), and last SIEM event summary. Use to verify gate thresholds are set, check which agents have custom models configured, and inspect the most recent inbound/internal/outbound scores. Read-only — no side effects.

2026-06-273

name	rerun-corpus
description	Audit and selectively re-run the deterministic analysis engine across all 26 corpus architectures. Default (no args) shows a staleness check — which reports are missing metadata, which arch types need engine fixes re-applied, and the current corpus score table. Pass --ai-only to re-run only AI/agentic architectures, --all to re-run everything, or a single arch name to re-run one. Use after engine changes (self_validation.py, ground_truth_generator.py, rapids_driven_controls.py) to confirm scores are consistent and no regressions landed.
allowed-tools	Bash(python3:) Bash(source:)

Corpus Re-run

Audits /report/ against the current engine version and re-runs analysis where needed.

Re-run is deterministic only (python3 -m chatbot.main --gen-arch-truth) — no LLM calls, no Expert Review. Takes ~15–20 sec per architecture.

Run

# Staleness audit only (default — read-only, fast)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py

# Re-run AI/agentic architectures only (highest impact after engine changes)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --ai-only

# Re-run a single architecture
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py 21_agentic_ai_system

# Re-run all 26 architectures (~8 min)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --all

What the audit checks

Signal	Stale if
`run_id` / `run_ts`	Missing from `ground_truth.json` metadata (pre engine-metadata fix)
`pattern_sources`	Missing — engine couldn't show which analysis engines fired
`ai_system` validation	Arch type is `ai_system` but `run_ts` predates the self_validation + hop-coverage fix (2026-07-12)
Score floor	TATB overall < 70 (below Solid)

Output format

STALE  22_generic_name_with_ai_nodes  type=ai_system  no run_id  → needs re-run
OK     21_agentic_ai_system           type=ai_system  run_id=21_agentic_ai_system_20260712T060119
OK     05_legacy_flat_network         type=web_app    run_id=05_legacy_flat_network_20260712T...
META   03_aws_3tier                   type=web_app    no run_id  (metadata only — score unaffected)
...

Summary: 1 STALE (need re-run), 24 META (display only), 1 OK
Re-run stale:  python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --ai-only
Re-run all:    python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --all

After re-running, a TATB corpus table is printed with before/after scores.

When to run

After any change to self_validation.py, ground_truth_generator.py, rapids_driven_controls.py, per_node_ttp_mapper.py, or exhaustive_mitigation_mapper.py
After updating MITRE data (/update-data)
To confirm a corpus-wide score regression didn't land

Related skills

/tatb-corpus — score all architectures from existing reports (read-only)
/tatb-score — score a single architecture
/backfill-aivss — refresh AIVSS governance scores after a harness change