torch-external-review

name	torch.external.review
description	Validate an external ask — a definition-of-ready check. Confirms the request is complete, well-formed, internally coherent, and carries the stricter intake an external ask requires, then writes a review file.
user-invocable	true
allowed-tools	Read, Write, Glob, Bash

You validate external asks. This is a definition-of-ready check — a single pass over the request as filed.

The only question you answer: is this request complete, well-formed, coherent, and specific enough on testing — enough for a tech lead to act on? You do NOT judge whether the build is a good idea, how hard it is, or whether the combination is achievable. Tech leads and staff own that.

You gate a request only when it is malformed: a missing field, an empty list, a placeholder, a self-contradiction, a past deadline, a hand-wavy testing or validation method, or a vague definition of complete.

Step 0: Parse Arguments

Parse $ARGUMENTS for:

• --headless: suppress the end-of-run summary (for CI callers)
• Remaining args: one or more space-separated IDs (EXT-NNN or AIPCC-NNNNN). If empty, default to every EXT-*.md file in artifacts/external-tasks/ with status: Draft or status: Ready.

Persist the ID list:

python3 scripts/state.py write-ids tmp/external-review-ids.txt <all_IDs>

Step 1: Read Each Request

For each ID, confirm artifacts/external-tasks/<ID>.md exists (Glob), then read it — both frontmatter and body:

python3 scripts/frontmatter.py read artifacts/external-tasks/<ID>.md

frontmatter.py read validates the frontmatter against the schema as it reads. If it exits non-zero, the request is malformed at the schema level — an automatic fail. Record it (Step 3) with the error message and move to the next ID.

If the frontmatter is valid, also Read the file body for the coherence check.

Step 2: Run the Check

Six checks. All must pass for the request to be ready.

1. Completeness. Every required field is present and meaningful — schema validation in Step 1 already caught missing/mistyped fields, so here catch what it cannot: empty lists, and placeholder values (TBD, ???, TODO, <...>).
2. Internal coherence. The request must not contradict itself — the body names a customer, hardware, or accelerator that disagrees with the frontmatter; the stated origin contradicts the body; etc.
3. Deadline sanity. requested_completion is a valid ISO 8601 date (YYYY-MM-DD) and not already in the past. Compare to today (python3 scripts/state.py timestamp).
4. Validation definitions. validation_definitions is non-empty and each entry is concrete — names a suite or an actual gate, not "test it" or "the usual".
5. Model validation. model_validation is concrete and names the model(s) to validate against, not a vague "validate it".
6. Definition of complete. Each entry in definition_of_complete is a concrete, checkable criterion — and the set includes how/where the result is delivered (target index, registry, location). A definition of complete with no delivery criterion is incomplete.

Step 3: Write the Review

For each ID, write artifacts/external-reviews/<ID>-review.md.

First Write the findings as the file body — plain markdown, no frontmatter:

### Definition-of-ready check — <ID>

**Completeness**: <ok | list of empty / placeholder fields>
**Internal coherence**: <ok | list of contradictions>
**Deadline**: <ok | past date | malformed date>
**Validation definitions**: <ok | empty | vague entries — say which>
**Model validation**: <concrete | vague — say what's missing>
**Definition of complete**: <ok | vague entries | no delivery criterion>

**To fix before this can be filed**: <none | numbered list>

Then set the frontmatter (frontmatter.py set on the existing file prepends validated frontmatter and keeps the body):

python3 scripts/frontmatter.py set artifacts/external-reviews/<ID>-review.md \
    ext_id=<ID> \
    pass=<true|false> \
    recommendation=<submit|revise> \
    needs_attention=<true|false>

• pass = true iff all six checks pass.
• recommendation = submit if pass, else revise.
• needs_attention = true if the request passes but something is still worth a human glance.

If Step 1 found invalid frontmatter, write the review with the schema error instead and skip the findings body:

python3 scripts/frontmatter.py set artifacts/external-reviews/<ID>-review.md \
    ext_id=<ID> pass=false recommendation=revise needs_attention=true \
    error="frontmatter_invalid: <msg>"

Step 4: Promote to Ready

For each ID where recommendation == submit, flip the task status:

python3 scripts/frontmatter.py set artifacts/external-tasks/<ID>.md status=Ready

For recommendation == revise, leave status: Draft.

Step 5: Finalize

Rebuild the index once:

python3 scripts/frontmatter.py rebuild-index --pipeline external

If --headless is set, output "torch.external.review step completed." and stop. Otherwise summarize per-ID:

• submit → the request is ready for /torch.external.submit.
• revise → list what the requester must fix (from the review file body).

$ARGUMENTS