菜单
Structural QA on pdd-to-test-prompts.md — header + total count match, ≥8 prompts each with required fields, all 7 adversarial categories, ≥20% adversarial share, plus training-gap / product-feedback / escalation prompts. Binary pass/fail; gates pdd-to-test-prompts-eval.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
Produce a domain-expert-facing before/after (pre-post) Google Doc for a DDD narrative iteration, and read the expert's edits back. Pull the verbatim CURRENT narration from canopy-web, apply reviewer feedback into a PROPOSED next version, publish a scannable side-by-side doc a non-engineer can review, then read their suggestion-mode edits via the structured Doc. Use when iterating a Connect DDD narrative on expert feedback (e.g. RF Surveys / Sophie), when asked to "make a before/after of the narration", "iterate the narrative on <expert>'s feedback", or "let <expert> sign off on the language".
Apply the two HQ-layer standing-instruction settings Nova can't set at build time — camera-only photo capture (appearance="acquire" on Deliver image uploads) and grid menu display on every module — to the deployed draft apps, then resolve the matching Phase-3 residuals. Runs between app-deploy and app-release.
Build the CommCare Learn (training) app from the PDD via Nova's /nova:autobuild. Captures nova_app_id and writes a structure summary.
Render a training deck spec.yaml into a Google Slides deck via the 14-stencil ACE template. Produces a presentable Slides URL.
Phase 3 § Step 2.8 — structural + install-time QA on the released Learn + Deliver CCZs. Downloads each CCZ via commcare_download_ccz, parses the zip + suite.xml + form XMLs, verifies form counts + Connect-marker presence match the Nova blueprint, then runs commcare-cli `validate` + `play` as install-time runtime gates. AVD-free, Connect-free — purely CCHQ-side. Halts loud on mismatch.
Run app smoke recipes against a local AVD and capture per-step screenshots for the training deck. Per-opp content only.
基于 SOC 职业分类
| name | pdd-to-test-prompts-qa |
| description | Structural QA on pdd-to-test-prompts.md — header + total count match, ≥8 prompts each with required fields, all 7 adversarial categories, ≥20% adversarial share, plus training-gap / product-feedback / escalation prompts. Binary pass/fail; gates pdd-to-test-prompts-eval. |
| disable-model-invocation | true |
Structural correctness checks on the pdd-to-test-prompts.md artifact written by pdd-to-test-prompts. Binary verdict. Eight static checks, runs <100ms via importable checks.ts.
The companion pdd-to-test-prompts-eval grades quality (specificity of expected answers, realism of adversarial prompts, etc.). See skills/_qa-template.md for the shared QA contract.
| Source | Artifact | Used for |
|---|---|---|
| Phase 1 producer | 2-scenarios/pdd-to-test-prompts.md | the test prompts file under structural check |
2-scenarios/pdd-to-test-prompts-qa_result.yaml — QA result per lib/qa-types.ts schema| # | id | type | description | auto-fix on fail |
|---|---|---|---|---|
| 1 | header_with_total_count | static | Title heading exists + Total prompts: N line matches actual prompt count | update header to match actual count or add/remove prompts |
| 2 | prompt_count_in_range | static | ≥8 and ≤80 prompts | add or consolidate prompts to fit |
| 3 | each_prompt_has_required_fields | static | Every prompt has Category, Question, Expected answer summary, Expected tags, Expected escalation | add the missing fields to named prompts |
| 4 | adversarial_coverage | static | All 7 adversarial categories (should-refuse, out-of-scope, hallucination-probe, leading-question, negative-frame, safety-critical, ambiguous-intent) each have ≥1 prompt | add prompts in the missing categories |
| 5 | adversarial_share_minimum | static | ≥20% of all prompts are adversarial | add adversarial prompts to reach the 20% threshold |
| 6 | training_gap_prompt_present | static | ≥1 prompt declares Expected tags: [training-gap] | add a prompt that should be tagged training-gap |
| 7 | product_feedback_prompt_present | static | ≥1 prompt declares Expected tags: [product-feedback] | add a prompt that should be tagged product-feedback |
| 8 | escalation_prompt_present | static | ≥1 prompt has non-trivial Expected escalation | add a prompt that should trigger bot escalation |
The static check functions live at skills/pdd-to-test-prompts-qa/checks.ts as importable TS. Same dispatch pattern as idea-to-pdd-qa (PR #149).
scripts/qa-run.ts --skill pdd-to-test-prompts-qa --artifact "$TMP" ....2-scenarios/pdd-to-test-prompts-qa_result.yaml.drive_read_file, drive_create_filenpx tsx scripts/qa-run.ts ...When --dry-run: reads happen normally; QA result IS written (internal artifact).
| Date | Change | Author |
|---|---|---|
| 2026-05-08 | Initial skill. Phase 1 PR #3 of the QA/Eval split migration (greenfield). 8 static checks. Companion pdd-to-test-prompts-eval ships in the same PR. | ACE team (0.13.90) |