Run any Skill in Manus with one click

tatb-score

Stars3

Forks1

UpdatedJuly 12, 2026 at 06:01

Run TATB (TA Test Benchmark) on a single architecture and print the four-rubric scorecard — Threat-Relevant, TTP-Accurate, Risk-Defensible, Plan-Actionable. Uses the same scoring logic as the dashboard TATB tab. Pass an architecture name as argument, or omit for the most recent report. Read-only. ~3 seconds (includes MITRE API fetch).

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

BerdTan

BerdTan/threatassessor

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Information Security AnalystsComputer and Mathematical Occupations·SOC 15-1212

File Explorer

2 files

SKILL.md

readonly

name	tatb-score
description	Run TATB (TA Test Benchmark) on a single architecture and print the four-rubric scorecard — Threat-Relevant, TTP-Accurate, Risk-Defensible, Plan-Actionable. Uses the same scoring logic as the dashboard TATB tab. Pass an architecture name as argument, or omit for the most recent report. Read-only. ~3 seconds (includes MITRE API fetch).
allowed-tools	Bash(python3:) Bash(source:)

TATB Score — Single Architecture

Scores one architecture across all four TATB rubrics and prints a detailed breakdown with sub-signals, evidence, and stage attribution. Identical logic to the dashboard TATB tab.

Run

# Most recent report
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate && python3 .claude/skills/tatb-score/scripts/tatb-score.py

# Named architecture
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate && python3 .claude/skills/tatb-score/scripts/tatb-score.py 01_minimal_vulnerable_2

# JSON output (for piping or agent consumption)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate && python3 .claude/skills/tatb-score/scripts/tatb-score.py 01_minimal_vulnerable_2 --json

What it shows

Section	Contents
Overall	Weighted average, rating band (Excellent/Solid/Weak/Draft), data availability
Threat-Relevant	Node binding %, node coverage %, technique variety %, generic-fallback flag
TTP-Accurate	Validation depth (CONFIRMED/PLAUSIBLE/FAILED), MITRE alignment %, cross-critic %, MoE lift
Risk-Defensible	Technique mitigation coverage, hop layer completeness, residual exposure (MONITOR/MITIGATE count)
Plan-Actionable	Item completeness, measurable outcomes, sprint spreadability, control specificity, AP closure
Stage attribution	Which pipeline stages to investigate for each low-scoring signal

Failure fixes

Error	Fix
`ground_truth.json not found`	Run analysis first from dashboard or CLI
MITRE fetch fails (alignment shows 50%)	API server must be running: `./scripts/api/api_start.sh`
Plan shows N/A	No ScrumMaster data — run `full_moe` scenario from dashboard Expert Review
Risk hop_pct = 50% (neutral)	No ADRs with hop data — run `full_moe` to generate ADRs

Notes

MITRE alignment requires the API server (localhost:8000) to be running — it fetches /api/v1/technique-mitigations. Falls back to 50% neutral if unavailable.
Plan-Actionable requires 08_scrum_master.json — only present after full_moe run.
Scores are not written back to disk — this is a read-only diagnostic tool.

Feedback loop

Low scores point directly to pipeline stages:

Signal	Stage	What to fix
node_binding < 80%	AnalysisStage → ground_truth_generator	Entry-node extraction; check parsed_nodes key names
tech_variety < 60%	AnalysisStage → RAPIDS + path-finding	Expand keyword lists or hop depth
val_pct < 70%	QualityStage → self_validation	Heuristic thresholds too permissive
mitre_pct < 70%	ReportStage → exhaustive_mitigation_mapper	Missing M-ID mappings for some techniques
hop_pct < 60%	ReportStage → ADR generator	dir_category assignment not covering all four layers
closure_pct < 80%	ScrumMasterStage → SM prompt	SM not referencing AP-IDs in action items

Related skills

/tatb-corpus — score all architectures at once and show corpus-wide patterns
/aivss-score — AIVSS-specific deep-dive
/quick-test — integration sanity check

More from this repository

same repository

rerun-corpus

BerdTan/threatassessor

Audit and selectively re-run the deterministic analysis engine across all 26 corpus architectures. Default (no args) shows a staleness check — which reports are missing metadata, which arch types need engine fixes re-applied, and the current corpus score table. Pass --ai-only to re-run only AI/agentic architectures, --all to re-run everything, or a single arch name to re-run one. Use after engine changes (self_validation.py, ground_truth_generator.py, rapids_driven_controls.py) to confirm scores are consistent and no regressions landed.

2026-07-123

tatb-corpus

BerdTan/threatassessor

Score all architectures in /report/ across the four TATB rubrics and print a visual corpus map — bar table, heatmap, and pipeline stage attribution. Shows which architectures are Excellent/Solid/Weak/Draft on each dimension and identifies which pipeline stages to fix for the largest corpus-wide gain. No UI needed. ~10-15 seconds (MITRE fetch batched once).

2026-07-113

run-er

BerdTan/threatassessor

Run Expert Review (MoE critics + ScrumMaster) on any architecture that has an existing analysis. Three modes — sequential (3 core critics), full_moe (5 critics + SM, generates Plan-Actionable data), or single-critic (re-run one critic by name). Streams live critic results with scores, gaps, and final confidence. ~60–120s for full_moe, ~30–60s for sequential.

2026-07-113

tatb-loop

BerdTan/threatassessor

TATB feedback loop — observe corpus or single-arch TATB scores, diagnose weak signals, propose targeted pipeline stage fixes, gate on human approval, apply changes, re-run TATB to verify improvement, and log the before/after delta to docs/DECISIONS.md. The agent does NOT autonomously commit code; every code change requires explicit human approval.

2026-07-043

quick-test

BerdTan/threatassessor

Runs a fast ~15 second integration sanity check — verifies dependencies, rate limiter, MITRE data load (≥835 techniques), OpenRouter API key, API connection, and one real embedding call. Use at session start or after installing dependencies to confirm everything is working before coding.

2026-07-023

aivss-gate

BerdTan/threatassessor

Show per-critic AIVSS gate config, per-agent model assignments (HarnessModelGuardian), and last SIEM event summary. Use to verify gate thresholds are set, check which agents have custom models configured, and inspect the most recent inbound/internal/outbound scores. Read-only — no side effects.

2026-06-273

name	tatb-score
description	Run TATB (TA Test Benchmark) on a single architecture and print the four-rubric scorecard — Threat-Relevant, TTP-Accurate, Risk-Defensible, Plan-Actionable. Uses the same scoring logic as the dashboard TATB tab. Pass an architecture name as argument, or omit for the most recent report. Read-only. ~3 seconds (includes MITRE API fetch).
allowed-tools	Bash(python3:) Bash(source:)

TATB Score — Single Architecture

Scores one architecture across all four TATB rubrics and prints a detailed breakdown with sub-signals, evidence, and stage attribution. Identical logic to the dashboard TATB tab.

Run

# Most recent report
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate && python3 .claude/skills/tatb-score/scripts/tatb-score.py

# Named architecture
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate && python3 .claude/skills/tatb-score/scripts/tatb-score.py 01_minimal_vulnerable_2

# JSON output (for piping or agent consumption)
cd "$(git rev-parse --show-toplevel)" && source .venv/bin/activate && python3 .claude/skills/tatb-score/scripts/tatb-score.py 01_minimal_vulnerable_2 --json

What it shows

Section	Contents
Overall	Weighted average, rating band (Excellent/Solid/Weak/Draft), data availability
Threat-Relevant	Node binding %, node coverage %, technique variety %, generic-fallback flag
TTP-Accurate	Validation depth (CONFIRMED/PLAUSIBLE/FAILED), MITRE alignment %, cross-critic %, MoE lift
Risk-Defensible	Technique mitigation coverage, hop layer completeness, residual exposure (MONITOR/MITIGATE count)
Plan-Actionable	Item completeness, measurable outcomes, sprint spreadability, control specificity, AP closure
Stage attribution	Which pipeline stages to investigate for each low-scoring signal

Failure fixes

Error	Fix
`ground_truth.json not found`	Run analysis first from dashboard or CLI
MITRE fetch fails (alignment shows 50%)	API server must be running: `./scripts/api/api_start.sh`
Plan shows N/A	No ScrumMaster data — run `full_moe` scenario from dashboard Expert Review
Risk hop_pct = 50% (neutral)	No ADRs with hop data — run `full_moe` to generate ADRs

Notes

MITRE alignment requires the API server (localhost:8000) to be running — it fetches /api/v1/technique-mitigations. Falls back to 50% neutral if unavailable.
Plan-Actionable requires 08_scrum_master.json — only present after full_moe run.
Scores are not written back to disk — this is a read-only diagnostic tool.

Feedback loop

Low scores point directly to pipeline stages:

Signal	Stage	What to fix
node_binding < 80%	AnalysisStage → ground_truth_generator	Entry-node extraction; check parsed_nodes key names
tech_variety < 60%	AnalysisStage → RAPIDS + path-finding	Expand keyword lists or hop depth
val_pct < 70%	QualityStage → self_validation	Heuristic thresholds too permissive
mitre_pct < 70%	ReportStage → exhaustive_mitigation_mapper	Missing M-ID mappings for some techniques
hop_pct < 60%	ReportStage → ADR generator	dir_category assignment not covering all four layers
closure_pct < 80%	ScrumMasterStage → SM prompt	SM not referencing AP-IDs in action items

Related skills

/tatb-corpus — score all architectures at once and show corpus-wide patterns
/aivss-score — AIVSS-specific deep-dive
/quick-test — integration sanity check