메뉴
Use when running the APXM review-council benchmark (multi-judge review). Enforces preregistration, model-zoo deploy order, and artifact placement under .apxm/.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | apxm-review-council-bench |
| group | Evaluation |
| description | Use when running the APXM review-council benchmark (multi-judge review). Enforces preregistration, model-zoo deploy order, and artifact placement under .apxm/. |
| user-invocable | true |
Load _shared/apxm-evaluation-rules.md,
_shared/apxm-preregistration-rules.md, and
_shared/apxm-storage-layout-rules.md before broad work.
Multi-judge review quality / latency for a council of LLM reviewers
against a workload. Current focus: gpt-oss-120b vs mistral-7b
capped/uncapped (see
docs/preregistrations/20260521T001540Z-apxm-review-council.md and
the -gptoss-capped follow-up).
apxm-preregistration — must be committed first.apxm_model_zoo_deploy_pattern memory):
docker-load → cache-warm → service-start → service-exec.
Never let service-start trigger an HF download under a GPU
allocation — the GPU clock runs while bytes stream from
huggingface.co.model.id in APXM config must match vLLM's
served_model_name — the bare name (gpt-oss-120b), not the HF
repo (openai/gpt-oss-120b). Mismatch produces silent 404s. See
feedback_apxm_model_id_must_match_served.deploy/vllm/zoo.review-gptoss.toml and
deploy/vllm/zoo.review-mistral7b.toml.# Per service:
dekk apxm vllm zoo-status
# Run the workload via the harness:
bash tools/scripts/run_apxm_review_council.sh # single model
bash tools/scripts/run_apxm_review_multimodel.sh # multi-model
Artifacts land under .apxm/evaluation/apxm_review_council/runs/<UTC>/.
python3 tools/scripts/assess_apxm_review_council_quality.py \
.apxm/evaluation/apxm_review_council/runs/<UTC>/
apxm-claim-evidence produces the write-up at
docs/evaluation/apxm_review_council/<UTC>.md.
service-start download a model under a GPU allocation
(wastes GPU-hours; cache-warm first without a GPU).model.id ≠ served_model_name → 404 storm masquerading as
perf failure.Use after a claim-bearing run completes. Writes the docs/evaluation/ write-up, promotes a claim card under docs/claims/ if paper-bound, and links the preregistration + artifact paths.
Use when creating, moving, reviewing, or documenting APXM benchmark, evaluation, session, compiler-diagnostic, evidence, or vLLM-run artifacts. Enforces that generated artifacts live in the repo-local .apxm workspace rather than examples or docs source trees.
Use before any claim-bearing APXM run (benchmark, evaluation, paper-bound number). Drafts, commits, and verifies a docs/preregistrations/ entry under the project's append-only norm.
Use when running the APXM priority-lane benchmark (foreground vs background concurrency). Enforces preregistration, paired-arm cache-salt scoping, and artifact placement under .apxm/.