sw-team-agent-qa

name	sw-team-agent-qa
description	Validation procedures for agentic deliverables in the SW-TEAM framework. Loaded by the SW-TEAM Agent QA agent at session startup to validate spec templates (field completeness, placeholder format), SKILL.md files (FIXED/ADAPT contract, Layer A cross-references), .agent.md files (frontmatter, tools, handoff self-containment), and integration artefacts (copilot-instructions.md, KB starters). Use when running Agent QA for any SW-TEAM dev phase or validating a single deliverable type. Load alongside a domain-context SKILL (e.g., {domain}-builder-context) for package name, output folder structure, and the domain validation scenario.
user-invocable	false

SW-TEAM Agent QA — Validation Procedures

Session Startup (Prepend to Procedure 1)

Before any other action:

0. Load domain-context SKILL: #tool:read_file → .github/skills/{domain}-builder-context/SKILL.md (or the relevant <domain>-builder-context/SKILL.md) This provides: package name, output folder structure, validation scenario for the integration test, KB file prefixes, and worker agent names.
1. Load this SKILL: #tool:read_file → .github/skills/sw-team-agent-qa/SKILL.md

---

Scope

Deliverable type	Validates	Does NOT validate
Spec templates (templates/*.md)	Field completeness, placeholder format, no project-specific values	Whether {package}/ code correctly parses the template
SKILL.md (.github/skills/*/SKILL.md)	FIXED/ADAPT contract, Layer A refs exist, output paths match folder structure	Whether the generated .py script runs correctly
.agent.md (.github/agents/*.agent.md)	Frontmatter syntax, tools, handoff prompts, user-invocable setting	Whether the agent behaves correctly in Copilot
Integration (copilot-instructions.md, kb/*.md)	Content matches framework doc content sketch, KB structure matches KB protocol	Whether Copilot reads the file at runtime
{package}/ Python code	Not in scope — Code QA handles this	—
Test files (tests/*.py)	Not in scope — Code QA handles this	—

Critical boundary: if a {package}/ function is missing that a SKILL.md references, report this as an Agent QA failure (broken Layer A reference). Do not write the missing function — escalate to the SCRUM Master.

---

Procedure 1: Validate a Spec Template

Input: path to a spec template (e.g., templates/project_spec_template.md)

1. Read the template file.
2. Read the framework spec inventory (from domain-context SKILL §FIXED — Framework document, the spec inventory section) to find the corresponding spec and its required fields.
3. Extract the "Key fields" for this spec.
4. For each required field:
   • Field is present in the template with a placeholder (e.g., <YYYY-MM-DD>)
   • Field has a description explaining what to fill in
   • Placeholder does NOT contain project-specific values (no project names, no hardcoded column names, no specific dates)
5. Check structural completeness:
   • Template has a title with a <scope_id> or identifier placeholder
   • Template has a Status: field (to be set to LOCKED when approved)
   • All markdown tables have correct column alignment
6. Report PASS (all checks green) or FAIL (list specific failures).

---

Procedure 2: Validate a SKILL.md

Input: path to a SKILL.md (e.g., .github/skills/ingest-data/SKILL.md)

1. Read the SKILL.md file.
2. Frontmatter check:
   • Has YAML frontmatter with name and description
   • description mentions when to trigger the skill
3. FIXED/ADAPT contract check:
   • At least one ## FIXED section exists
   • At least one ## ADAPT section exists
   • FIXED sections contain NO project-specific values — only {placeholders}, spec[...] references, or framework constants
   • ADAPT sections have clear markers indicating what the agent fills (comments like # Agent fills this based on project_spec.md)
   • FIXED sections include validation logic (assertions, checks) that would catch errors in ADAPT sections
4. Layer A cross-reference check:
   • For every {package}.* function call in the scaffold:
     • Use #tool:search/textSearch to confirm the function exists in the package directory (from domain-context SKILL §FIXED — Package)
     • If the function does not exist: FAIL — report as broken Layer A ref
   • For every from {package}.* import statement:
     • Use #tool:file_search to confirm the module exists
5. Output path check:
   • Output paths in the scaffold match the folder structure from domain-context SKILL (§FIXED — Output folder structure)
   • The skill writes to paths using {scope_id} placeholders (not hardcoded paths)
   • Code snapshot path matches folder structure convention
6. Spec parameter sourcing check:
   • All numeric parameters come from load_spec_section() or equivalent
   • No hardcoded numeric literals for model parameters
   • FIXED sections may contain framework constants — these are acceptable
7. Report PASS or FAIL with specifics per check.

---

Procedure 3: Validate an .agent.md

Input: path to an .agent.md (e.g., .github/agents/domain-expert.agent.md)

1. Read the .agent.md file.
2. Frontmatter check:
   • name field present and non-empty
   • description field present and non-empty
   • tools field present and is a list
   • model field present
   • If worker agent: user-invocable: false is set
   • If worker agent: disable-model-invocation: true is set
3. Handoff check (if handoffs are declared):
   • For each handoff:
     • label is present and descriptive
     • agent references a known agent name
     • prompt is present and self-contained: a subagent receiving ONLY this prompt (no session history) must be able to understand what to do. Implicit references like "the current spec" or "what we discussed" are failures — subagents have no context.
4. System prompt check (agent body):
   • References the correct SKILL.md path(s) for loading
   • Includes a session startup sequence
   • Responsibilities match the framework doc agent definitions section for this agent
   • Does not contain responsibilities that belong to a different agent
5. Cross-agent consistency:
   • If orchestrator: declares the worker agent names (from domain-context SKILL §FIXED — Worker agent names)
   • If worker agent: does NOT declare handoffs to other agents
6. Report PASS or FAIL with specifics.

---

Procedure 4: Validate Integration Artefacts

Input: paths to copilot-instructions.md and/or kb/*.md starters

For copilot-instructions.md:

1. Read the file.
2. Check against the framework doc content sketch (from domain-context SKILL §FIXED — Framework document, the integration artefacts section):
   • <project_root> definition present
   • artefacts.jsonl protocol present (append-only, sequential ID format)
   • KB protocol present (entry ID format with prefix, dedup rule)
   • Agent invocation rules present (sequential, self-contained)
   • Output path conventions present (matching folder structure)
3. Check for forbidden content:
   • No project-specific values (no project names, no specific paths)
   • No model-specific configuration

For KB starters:

1. Read each file.
2. Check against the KB protocol (from domain-context SKILL §FIXED — KB file prefixes):
   • Section headers match the KB section definition in the framework doc
   • Entry ID format guidance is present
   • Entry format example is included (timestamp, agent, phase, description)
   • File contains structure only — no actual content entries
3. Report PASS or FAIL per file.

---

Procedure 5: Integration Test (Domain Validation Scenario)

Input: dev phase number (e.g., DP-2)

This is a mental walkthrough — Agent QA cannot run agents in Copilot. Instead, verify that all artefacts are consistent with what the domain validation scenario (from domain-context SKILL §FIXED — Validation scenario) expects.

1. Read framework doc for the relevant phase (from domain-context SKILL §FIXED — Framework document).
2. Read the dev phase validation test criteria (from domain-context SKILL §FIXED — Dev phase list).
3. For each validation criterion:
   • {package}/ function output → skip (Code QA's responsibility)
   • Spec template field → verify the template contains that field
   • SKILL.md scaffold → verify the scaffold would produce the expected output path and column structure described in the domain validation scenario
   • .agent.md → verify the agent's handoff prompt includes the information the scenario says the invocation prompt must contain
4. For SKILL.md scaffolds, trace the domain validation scenario example:
   • Do the {placeholders} in ADAPT sections map to the example values in the scenario?
   • Would the FIXED validation assertions catch the known issues in the scenario?
5. Report which §dev-plan criteria are covered by Agent QA (PASS/FAIL) and which are deferred to Code QA.

---

Track Consistency Check

Run this check when validating deliverables for any phase that introduces track-specific SKILL.md files or {package}/ modules (read the applicable phases from domain-context SKILL §FIXED — Modelling tracks).

For each modelling track defined in the domain-context SKILL:

1. SKILL.md file exists for this track at the expected path
2. Track-specific approach matches the spec (per domain-context SKILL §FIXED — Modelling tracks)
3. Layer A calls use ONLY {package}.modeling.<track>.* (no cross-track imports)
4. Evaluation diagnostics are track-specific (per domain-context SKILL)
5. No SKILL.md or spec template forces a common return type across tracks

Any violation is a P0 failure — report immediately.

---

Report Formats

Single Deliverable Report

## Agent QA — [deliverable type]: [file path]

**Phase**: DP-N
**Result**: ✅ PASS / ❌ FAIL

### Checks
- [x] / [ ] <check description>
- ...

### Failures (if any)
1. **[check name]**: <what was expected> vs <what was found>
   - File: <path>
   - Line: <N> (if applicable)
   - Fix: <specific recommendation>

### Layer A Cross-References (SKILL.md only)
| {package}/ function | Exists | Module path |
|---|---|---|
| `{package}.module.function_name` | ✅/❌ | `{package}/module/file.py` |

Full Phase Report

## Agent QA Report — DP-N: [Phase Name]

**Date**: YYYY-MM-DD
**Overall result**: ✅ ALL PASS / ❌ FAILURES FOUND

### Summary
| Deliverable | Type | Result | Failures |
|---|---|---|---|
| `templates/project_spec_template.md` | spec | ✅/❌ | 0 / N |
| `.github/skills/ingest-data/SKILL.md` | skill | ✅/❌ | 0 / N |