菜单
Adversarial blind evaluation harness. Generates and runs hidden tests against code written by coding agents. The coding agent never sees the tests — only pass/fail results. Use when /plan creates tasks, or manually with "test-lab run <target>".
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Create receipt-backed persona dream packets from memory residue. Use when a persona should dream, reflect, or turn recent memories into persona insight; when create-movie/dream.py feels too heavy for the goal; when the desired output is a prompt, frame prompts, contact sheet, reflection, and memory write receipt rather than a full movie; or when a downstream movie workflow needs a dream_packet.json input.
Use when the user asks to query project memory, ask an oracle, use supported browser-backed reviewers, run Tau roundtable/single-handler workflows, run persona/deep-review workflows, generate image prompts, check OS/project health through composed skills, or run an ask DAG. This skill is the executable /ask runtime; do not replace it with an informal subagent, plain web search, or hand-written review.
Build structured evidence cases using Claims-Arguments-Evidence (CAE) trees. AGENT-DRIVEN composable orchestrator: the agent decomposes the question, calls existing skills for data collection and verification, then DECIDES the verdict. Python runner.py is a thin data collector and persistence layer.
Monitor Herdr-visible Codex/Claude agent panes for stalled, blocked, unknown, or confused state, then ask each stalled agent whether it needs human intervention or can self-unblock with brave-search or webgpt.
Internal Tau-owned LLM proxy on localhost:4001. Surfaces: chat/batch completions, scillm exec, OpenCode serve (coding delegate), OpenCode transport (DAG/SSE), standing Codex agents. Chutes, Gemini, Claude/Codex OAuth, OpenCode Go, Ollama. Auto-routes by model name. ZIP/PDF, JSON repair, batch pools. Project agents must not call this skill directly unless the human explicitly asks to operate SciLLM or the work is Tau/SciLLM maintenance.
Operate and verify the local T'au project at ${HOME}/workspace/experiments/tau. Use for Tau loop, harness, watchdog cron, GitHub issue orchestration, TUI, Memory-first chat, and E2E proof/status tasks. This skill is a light wrapper around the Tau repo and must report mocked/live proof boundaries explicitly.
| name | test-lab |
| description | Adversarial blind evaluation harness. Generates and runs hidden tests against code written by coding agents. The coding agent never sees the tests — only pass/fail results. Use when /plan creates tasks, or manually with "test-lab run <target>". |
| provides | ["skill-validation","adversarial-testing"] |
| composes | ["memory","best-practices-python","best-practices-react","best-practices-kde","best-practices-skills","task-monitor"] |
| triggers | ["test-lab","blind test","run hidden tests"] |
| metadata | {"short-description":"Blind evaluation harness — agents never see their tests"} |
| taxonomy | ["testing","quality"] |
STOP. READ THIS ENTIRE SKILL.MD BEFORE CALLING ANY ENDPOINT.
Enforces an information barrier between the agent writing code and the tests verifying it.
Coding agents write code and tests in the same context, then optimize for passing their own tests rather than actual correctness. ImpossibleBench (arXiv:2510.20270) showed GPT-5 cheats 76% of the time when it can see tests, but near-zero when tests are hidden.
# Generate hidden tests from a task file
test-lab/run.sh generate <plan-file>
# Run hidden tests against a target directory
test-lab/run.sh run <target-dir> [--domain python|react|kde|skills]
# Rich table report from results JSON
test-lab/run.sh report <results-json>
# Verify a specific task with retry budget
test-lab/run.sh verify-task <task-id> <target-dir> --max-retries N
TASK: Add Redis caching module
STATUS: FAIL (attempt 2/5)
FAILURES:
- [conventions] Entry point missing dotenv loading
- [style] File exceeds 800 line limit
- [testing] No sanity test with real fixtures
PASSES: 7/10
No test file paths. No assertion source. No expected values.
| Skill | Integration Point |
|---|---|
/plan | Auto-generates hidden tests after task file creation |
/orchestrate | Runs test-lab verify-task as blocking gate after each task |
test-lab-guard.ts | Extension blocks agent reads of test dirs |
| Scenario | Behavior |
|---|---|
| test-lab not installed | /plan warns, tasks proceed without blind eval |
| Extension not loaded | Tests don't run automatically, can be run manually |
| Generator fails for a domain | That domain's tests skipped, others still run |
| No applicable best-practices | Task passes trivially (no hidden tests) |
| All retries exhausted | Task marked BLOCKED, human notified via agent-inbox |