ai-qa

name	ai-qa
description	Use when configuring AI QA, testing Web or virtual mobile app behavior, capturing evidence, promoting platform case variants, replaying regressions, or reporting selected multi-platform runs with the ai-qa CLI.
metadata	{"aiQaSkillVersion":"3.1.1","aiQaProtocolRange":"^3.0.0","aiQaRecordingReceipt":true,"aiQaManagedChecksum":"bundled"}

AI QA Workflow

AI QA supports exactly web, ios-simulator, and android-emulator. Real devices are unsupported. The host owns project access, permissions, authentication, platform controllers, and file writes. The CLI never invokes controllers; it validates and records host-supplied readiness, actions, evidence, verdicts, cases, RunGroups, reports, and recording receipts.

Enter or re-enter a trusted project

Use this gate before readiness diagnosis:

1. Resolve the exact project root; derive both from your own runtime knowledge: your host label and your host's skill root (agentsHome), and confirm both with the user on first use.
2. Run ai-qa project upgrade check --stdin-json with {"host":"codex","agentsHome":"/Users/example/.agents"}, substituting your derived values.
3. For up_to_date, continue.
4. For migration_required, load every returned Migration Skill from its exact installed skillPath and execute the migrations automatically in order.
5. For migration_recovery_required, load the returned Migration Skill. History-writing commands are unavailable while project migration recovery is active. Inspect its declared targets and validation invariants. Retry ai-qa project upgrade complete when every invariant already holds; otherwise abort. Abort restores declared targets and CLI-owned project state while preserving undeclared config edits. Then rerun upgrade check and the returned migration chain.
6. For migration_conflict, stop without project writes.
7. Only after you complete the migration chain and upgrade check returns up_to_date, run ai-qa doctor --json --stdin-json with the same host context.

Run ai-qa skill sync to finish a stale forward Global Skill upgrade. Forward sync is automatic and retryable without confirmation; same-version managed repair or downgrade requires --confirm-managed-replacement.

Configure a project

1. Complete the trusted-project entry gate. For example, Claude Code uses claude-code and ~/.claude, while Codex uses codex and ~/.agents. Treat requiredAction.kind: configure-project as a blocking first-use gate.
2. Inspect project-owned instructions and derive only unambiguous values. Ask for a non-empty deployed platform selection, then collect every selected platform's required configuration. Never configure a physical iOS or Android device.
3. Inspect existing result-management procedures. Always ask the user to explicitly choose recordingPolicy.mode; neither local-only nor project-skill has a default. Select project-skill only after the user confirms the exact existing procedure.
4. Draft the complete schema 3 config and canonical project skill .ai-qa/project-skill/SKILL.md together. Use skill-creator for the Project Skill. Keep literal secrets out; config may name environment variables.
5. Run ai-qa config validate --stdin-json, validate the scratch Project Skill, and verify exact-root, target/parent symlink, and secret safety.
6. For each proposed file, display the missing destination's exact path and complete proposed content, or the existing destination's complete diff. Never render a synthetic diff for a missing destination. Obtain one confirmation for the config and canonical Project Skill, then write both files once and create the canonical .ai-qa/ directories. If the user cancels, write nothing. Never ask the user or the Agent to calculate a checksum.
7. Repeat the trusted-project entry gate for every configured platform. Resume QA only when every requested platform is ready.

Read shared-work-protocol.md for setup, lifecycle, evidence, case, RunGroup, report, and recording contracts.

Provision and refresh the project skill

The canonical project skill lives at .ai-qa/project-skill/SKILL.md. Each host reads its own rendering, registered in .ai-qa/project-skill/renderings.yaml.

• requiredAction: provision-host-skill — derive your host's project-skill location and format from your own conventions, render the canonical content into it (preserve managed markers; adapt format only as your host requires), show the complete proposed file or complete diff, obtain one confirmation, write the file, then run ai-qa project-skill register --stdin-json with {"host": <label>, "path": <project-relative path>}.
• requiredAction: refresh-host-skill — the canonical content changed. Re-render your host's file from the current canonical content, show the complete diff, confirm once, write, and register again.

Procedure edits always change the canonical file, never a rendering directly. Protocol 2.0.0 runs are immutable history: never resume them; start new runs. They remain readable for state, verified reports, and recording status, but repair, materialization, case promotion, and receipt registration are also forbidden.

Execute selected platforms

Before starting work, ask which configured platform subset the user wants now. Accept any non-empty subset of one, two, or three configured platforms. Configuration never selects execution platforms.

Every run supplies the current host context once. run start stdin includes host and agentsHome once: exploratory stdin is {"goal":"...","acceptanceCriteria":[...],"host":"codex","agentsHome":"/Users/example/.agents","readiness":{"platform":"web","status":"ready","checks":[]}}; regression stdin is {"platform":"web","status":"ready","checks":[],"host":"codex","agentsHome":"/Users/example/.agents"}. run-group start stdin is {"host":"codex","readiness":{"web":{"platform":"web","status":"ready","checks":[]}}}.

For each selected platform:

1. Read its controller reference and obtain host-recorded readiness.
2. Start a platform-owned run. One run has one platform, work order, journal, evidence directory, verdict, and report.
3. Follow the shared two-phase action and fresh post-action evidence chain. Invoke the controller only through the host, never through the CLI.
4. Promote reviewed exploratory work incrementally into the matching immutable platform variant.

Use web-controller.md for Web, ios-simulator-controller.md for iOS Simulator, and android-emulator-controller.md for Android Emulator.

For exploratory work, multi-platform exploratory QA starts one explicit exploratory run per selected platform with the confirmed goal and acceptance criteria. Complete and report each run independently, then promote each reviewed platform variant incrementally.

For regression, one selected platform starts one explicit regression run. Multi-platform regression uses a RunGroup with the explicit selected platforms and explicit cases or --all-active. Missing selected case variants are coverage gaps, not child runs. Generate and verify every child report before the aggregate report. An aggregate report preserves the complete matrix and never synthesizes a QA verdict.