Jeden Skill in Manus ausführen
mit einem Klick

Jeden Skill in Manus mit einem Klick ausführen

rerun-corpus

Sterne3

Forks1

Aktualisiert12. Juli 2026 um 06:09

Audit and selectively re-run the deterministic analysis engine across all 26 corpus architectures. Default (no args) shows a staleness check — which reports are missing metadata, which arch types need engine fixes re-applied, and the current corpus score table. Pass --ai-only to re-run only AI/agentic architectures, --all to re-run everything, or a single arch name to re-run one. Use after engine changes (self_validation.py, ground_truth_generator.py, rapids_driven_controls.py) to confirm scores are consistent and no regressions landed.

Installation

Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.

In Manus ausführen

Quelle

BerdTan

BerdTan/threatassessor

GitHub-Repository öffnen Creator-Repositorys ansehen

Download

In Manus ausführen

Verwandte BerufeSOC

Basierend auf der SOC-Berufsklassifikation

Softwarequalitätssicherungsanalysten und -testerInformatik- und Mathematikberufe·SOC 15-1253

Datei-Explorer

2 Dateien

SKILL.md

readonly

name	rerun-corpus
description	Audit and selectively re-run the deterministic analysis engine across all 26 corpus architectures. Default (no args) shows a staleness check — which reports are missing metadata, which arch types need engine fixes re-applied, and the current corpus score table. Pass --ai-only to re-run only AI/agentic architectures, --all to re-run everything, or a single arch name to re-run one. Use after engine changes (self_validation.py, ground_truth_generator.py, rapids_driven_controls.py) to confirm scores are consistent and no regressions landed.
allowed-tools	Bash(python3:) Bash(source:)

Corpus Re-run

Audits /report/ against the current engine version and re-runs analysis where needed.

Re-run is deterministic only (python3 -m chatbot.main --gen-arch-truth) — no LLM calls, no Expert Review. Takes ~15–20 sec per architecture.

Run

# Staleness audit only (default — read-only, fast)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py

# Re-run AI/agentic architectures only (highest impact after engine changes)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --ai-only

# Re-run a single architecture
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py 21_agentic_ai_system

# Re-run all 26 architectures (~8 min)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --all

What the audit checks

Signal	Stale if
`run_id` / `run_ts`	Missing from `ground_truth.json` metadata (pre engine-metadata fix)
`pattern_sources`	Missing — engine couldn't show which analysis engines fired
`ai_system` validation	Arch type is `ai_system` but `run_ts` predates the self_validation + hop-coverage fix (2026-07-12)
Score floor	TATB overall < 70 (below Solid)

Output format

STALE  22_generic_name_with_ai_nodes  type=ai_system  no run_id  → needs re-run
OK     21_agentic_ai_system           type=ai_system  run_id=21_agentic_ai_system_20260712T060119
OK     05_legacy_flat_network         type=web_app    run_id=05_legacy_flat_network_20260712T...
META   03_aws_3tier                   type=web_app    no run_id  (metadata only — score unaffected)
...

Summary: 1 STALE (need re-run), 24 META (display only), 1 OK
Re-run stale:  python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --ai-only
Re-run all:    python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --all

After re-running, a TATB corpus table is printed with before/after scores.

When to run

After any change to self_validation.py, ground_truth_generator.py, rapids_driven_controls.py, per_node_ttp_mapper.py, or exhaustive_mitigation_mapper.py
After updating MITRE data (/update-data)
To confirm a corpus-wide score regression didn't land

Related skills

/tatb-corpus — score all architectures from existing reports (read-only)
/tatb-score — score a single architecture
/backfill-aivss — refresh AIVSS governance scores after a harness change

Mehr aus diesem Repository

gleiches Repository

tatb-score

BerdTan/threatassessor

Run TATB (TA Test Benchmark) on a single architecture and print the four-rubric scorecard — Threat-Relevant, TTP-Accurate, Risk-Defensible, Plan-Actionable. Uses the same scoring logic as the dashboard TATB tab. Pass an architecture name as argument, or omit for the most recent report. Read-only. ~3 seconds (includes MITRE API fetch).

2026-07-123

tatb-corpus

BerdTan/threatassessor

Score all architectures in /report/ across the four TATB rubrics and print a visual corpus map — bar table, heatmap, and pipeline stage attribution. Shows which architectures are Excellent/Solid/Weak/Draft on each dimension and identifies which pipeline stages to fix for the largest corpus-wide gain. No UI needed. ~10-15 seconds (MITRE fetch batched once).

2026-07-113

run-er

BerdTan/threatassessor

Run Expert Review (MoE critics + ScrumMaster) on any architecture that has an existing analysis. Three modes — sequential (3 core critics), full_moe (5 critics + SM, generates Plan-Actionable data), or single-critic (re-run one critic by name). Streams live critic results with scores, gaps, and final confidence. ~60–120s for full_moe, ~30–60s for sequential.

2026-07-113

tatb-loop

BerdTan/threatassessor

TATB feedback loop — observe corpus or single-arch TATB scores, diagnose weak signals, propose targeted pipeline stage fixes, gate on human approval, apply changes, re-run TATB to verify improvement, and log the before/after delta to docs/DECISIONS.md. The agent does NOT autonomously commit code; every code change requires explicit human approval.

2026-07-043

quick-test

BerdTan/threatassessor

Runs a fast ~15 second integration sanity check — verifies dependencies, rate limiter, MITRE data load (≥835 techniques), OpenRouter API key, API connection, and one real embedding call. Use at session start or after installing dependencies to confirm everything is working before coding.

2026-07-023

aivss-gate

BerdTan/threatassessor

Show per-critic AIVSS gate config, per-agent model assignments (HarnessModelGuardian), and last SIEM event summary. Use to verify gate thresholds are set, check which agents have custom models configured, and inspect the most recent inbound/internal/outbound scores. Read-only — no side effects.

2026-06-273

name	rerun-corpus
description	Audit and selectively re-run the deterministic analysis engine across all 26 corpus architectures. Default (no args) shows a staleness check — which reports are missing metadata, which arch types need engine fixes re-applied, and the current corpus score table. Pass --ai-only to re-run only AI/agentic architectures, --all to re-run everything, or a single arch name to re-run one. Use after engine changes (self_validation.py, ground_truth_generator.py, rapids_driven_controls.py) to confirm scores are consistent and no regressions landed.
allowed-tools	Bash(python3:) Bash(source:)

Corpus Re-run

Audits /report/ against the current engine version and re-runs analysis where needed.

Re-run is deterministic only (python3 -m chatbot.main --gen-arch-truth) — no LLM calls, no Expert Review. Takes ~15–20 sec per architecture.

Run

# Staleness audit only (default — read-only, fast)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py

# Re-run AI/agentic architectures only (highest impact after engine changes)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --ai-only

# Re-run a single architecture
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py 21_agentic_ai_system

# Re-run all 26 architectures (~8 min)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate \
  && python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --all

What the audit checks

Signal	Stale if
`run_id` / `run_ts`	Missing from `ground_truth.json` metadata (pre engine-metadata fix)
`pattern_sources`	Missing — engine couldn't show which analysis engines fired
`ai_system` validation	Arch type is `ai_system` but `run_ts` predates the self_validation + hop-coverage fix (2026-07-12)
Score floor	TATB overall < 70 (below Solid)

Output format

STALE  22_generic_name_with_ai_nodes  type=ai_system  no run_id  → needs re-run
OK     21_agentic_ai_system           type=ai_system  run_id=21_agentic_ai_system_20260712T060119
OK     05_legacy_flat_network         type=web_app    run_id=05_legacy_flat_network_20260712T...
META   03_aws_3tier                   type=web_app    no run_id  (metadata only — score unaffected)
...

Summary: 1 STALE (need re-run), 24 META (display only), 1 OK
Re-run stale:  python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --ai-only
Re-run all:    python3 .claude/skills/rerun-corpus/scripts/rerun-corpus.py --all

After re-running, a TATB corpus table is printed with before/after scores.

When to run

After any change to self_validation.py, ground_truth_generator.py, rapids_driven_controls.py, per_node_ttp_mapper.py, or exhaustive_mitigation_mapper.py
After updating MITRE data (/update-data)
To confirm a corpus-wide score regression didn't land

Related skills

/tatb-corpus — score all architectures from existing reports (read-only)
/tatb-score — score a single architecture
/backfill-aivss — refresh AIVSS governance scores after a harness change