Menu
Red-team a Harbor verifier for answer leakage, reward hacking, shared-container tampering, artifact manipulation, and semantic under-testing.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Build or repair a deterministic Harbor task environment with explicit resources, network policy, health checks, and safe solver-visible boundaries.
Implement a legitimate Harbor Oracle solution and validate its behavior against an independent verifier without weakening the task contract.
Calibrate a frozen tb-hard task using Harbor Oracle and real-agent rollouts, then distinguish authentic difficulty from ambiguity, flakiness, leakage, or infrastructure failure.
Create or evolve a canonical Harbor task inside a stable tb-hard case directory. Use after the design brief names observable acceptance criteria and at least three difficulty dimensions.
Audit a calibrated Harbor task for deterministic purchaser export, leakage, evidence completeness, and release readiness.
Research Harbor and Terminal-Bench 2 patterns to design an original tb-hard task. Use before approving a design brief or when checking whether a candidate is too shallow or derivative.
| name | harbor-redteam-verifier |
| description | Red-team a Harbor verifier for answer leakage, reward hacking, shared-container tampering, artifact manipulation, and semantic under-testing. |
| argument-hint | <case_id> |
| user-invocable | true |
| context | fork |
| agent | verifier-security-reviewer |
| model | sonnet |
| effort | high |
| allowed-tools | Read Write Edit Glob Grep Bash |
Target case: $ARGUMENTS
Treat the solver as adversarial. Inspect task/, Docker build boundaries, task metadata, verifier mode, artifact declarations, sidecars, startup behavior, and test coverage.
Attempt to pass without solving by deleting functionality, hardcoding visible samples, tampering with wrappers/PATH/environment, reading image layers or Git history, abusing caches/processes/sidecars, or modifying shared verification state. Determine whether a separate verifier is necessary.
Write cases/<case_id>/private/reviews/redteam-report.md with severity, exploit path, grading impact, minimal remediation, and retest steps. Do not edit benchmark semantics or silently relax the verifier.
