| name | muggle-test-regenerate-missing |
| model | sonnet |
| description | Bulk-regenerate test scripts for every test case in a Muggle AI project that has no active script. Scans the project, finds test cases stuck in DRAFT or GENERATION_PENDING, shows the user the list, and on approval kicks off bulk remote script generation via the Muggle Test cloud. Use when the user wants to regenerate missing scripts, fill in missing test scripts, generate scripts for test cases without one, rebuild scripts for stale test cases, or bulk regenerate across a project. Handles discovery, filtering, confirmation, and remote dispatch end-to-end. |
Muggle Test — Regenerate Missing Test Scripts
Telemetry first step: see _shared/telemetry-emit.md. Use skillName: "muggle-test-regenerate-missing".
A bulk maintenance skill for Muggle AI projects. It finds every test case in a project that does not currently have an active (ready-to-run) test script, shows the list to the user, and on approval triggers a remote test script generation workflow for each one. Useful after creating a batch of new test cases or when cleaning up a project that has drifted.
Execution is remote only — Muggle Test's cloud generates the scripts in parallel against the project URL. The user's machine is not involved beyond making API calls.
Preferences
Gates run per preference-gates/README.md.
| Preference | Step | Decision it gates |
|---|
autoLogin | 1 | Reuse saved credentials when auth is required |
autoSelectProject | 2 | Reuse last-used Muggle Test project for this repo |
Concept: what counts as "no active script"
In the Muggle Test data model, a test case carries a status that reflects whether it has a usable script attached:
| Status | Meaning | Regenerate? |
|---|
ACTIVE | Has a generated, ready-to-run script | No — already good |
DRAFT | Created but never generated | Yes |
GENERATION_PENDING | Queued but generation never started | Yes |
GENERATING | Currently generating | No — generation is in flight; don't double-dispatch |
REPLAY_PENDING / REPLAYING | A replay is in flight; script exists | No — busy with replay |
DEPRECATED | Marked stale on purpose | No — user decision |
ARCHIVED | Hidden from normal flows | No — user decision |
The skill targets exactly DRAFT and GENERATION_PENDING by default. These are the two states that mean "no usable script attached and nothing running to produce one." GENERATING is deliberately excluded — a workflow is already in progress, and firing a second one races against the first and wastes budget.
Treat this filter as a default, not a law. If the user explicitly says "include generating ones too, they're stuck" or "include deprecated", respect the override — but don't widen the filter on your own.
UX Guidelines — Minimize Typing
Every selection-based question MUST use the AskUserQuestion tool (or the platform's equivalent structured selection tool). Never ask the user to "reply with a number" — always present clickable options.
- Selections (project, which test cases to include): Use
AskUserQuestion, with allow_multiple: true for the test case picker.
- Free-text inputs (project URL when creating, override filters): Only ask as plain text when the option set isn't finite.
- Batch related questions when independent. Don't ask sequentially what could be one screen.
Workflow
Step 1 — Authenticate (gated by autoLogin)
- Call
muggle-remote-auth-status.
- If authenticated and not expired → gate
autoLogin (per preference-gates/README.md):
always → proceed with saved session.never → muggle-remote-auth-login with forceNewSession: true, then muggle-remote-auth-poll.ask → run Picker 1 from preference-gates/autoLogin.md via AskUserQuestion; map the answer back to one of the actions above.
- If not authenticated or expired → call
muggle-remote-auth-login, then poll with muggle-remote-auth-poll.
- Do not skip auth and do not assume a stale token still works.
If auth keeps failing, suggest the user run muggle logout && muggle login from a terminal.
Step 2 — Select Project (gated by autoSelectProject)
A project is the unit on the Muggle AI dashboard that groups test cases, scripts, and runs. The user must pick the one to scan — never auto-select from repo name, branch, or URL heuristics.
The per-repo project cache lives at <cwd>/.muggle-ai/last-project.json (via the muggle-local-last-project-get / muggle-local-last-project-set MCP tools). Look for Muggle Test Last Project: id=… url=… name="…" in session context.
Gate autoSelectProject (per preference-gates/README.md). Cache: Muggle Test Last Project session line.
always + cache → use cached projectId, proceed to Step 3. No cache → fall through to ask.never → full project list; skip Picker 2.ask → project list picker (see gate file for spec + Picker 2 override). Skip Picker 2 if "Create new project".
Logic
- Call
muggle-remote-project-list (only when not satisfied by the always cache).
- Use
AskUserQuestion to present projects as clickable options. Include the project URL in each label so the user can disambiguate. Always include "Create new project" as the last option.
- Wait for explicit selection.
- If the user picks "Create new project": collect
projectName, description, and url, then call muggle-remote-project-create.
Store projectId and projectUrl only after the user confirms — both are needed downstream.
Step 3 — Scan Test Cases
Pull the full set of test cases for the project. The list endpoint is paginated.
- Call
muggle-remote-test-case-list with projectId. Start at page 1; continue requesting pages until the response indicates no more items. Use a generous pageSize (e.g. 100) to keep the call count low.
- Accumulate everything into a single in-memory array.
- Tell the user the totals as you go if the project is large (e.g., "Found 247 test cases across the project").
If the call returns zero test cases, stop and tell the user — there is nothing to regenerate. Suggest they create test cases first (point them at muggle-test or muggle-test-feature-local).
Step 4 — Filter to Missing Scripts
Apply the default filter: keep only test cases where status ∈ {DRAFT, GENERATION_PENDING}.
Then show a one-line summary. Also surface how many cases are currently GENERATING so the user knows about in-flight work (but don't include them in the candidate list unless they override the filter):
Project: <name> (<projectId>)
Total test cases: 247
With active script (skipped): 198
Currently generating (skipped, in-flight): 32
Needs regeneration: 17
• DRAFT: 12
• GENERATION_PENDING: 5
If after filtering the list is empty, congratulate the user — every test case already has an active script — and stop. Do not invent work.
Step 5 — Present and Confirm Selection
Use AskUserQuestion with allow_multiple: true to present every candidate test case as a clickable option. The user must explicitly approve which ones to regenerate.
For each option label, show enough context for the user to make a real decision:
[<status>] <title> — use case: <use case title>
For example:
[DRAFT] Sign up with valid email — use case: User Registration[GENERATION_PENDING] Add item to cart — use case: Checkout Flow
Default behavior:
- If there are ≤ 25 candidates, present all of them in a single
AskUserQuestion with everything pre-checked and let the user deselect anything they want to skip.
- If there are > 25 candidates, show the first 25 ranked by status priority (
DRAFT → GENERATION_PENDING), plus a tail option "Include all N — don't make me click each one". The user can also pick "Show next batch" to see more.
After selection, call AskUserQuestion once more for a final confirmation:
"About to start remote test script generation for N test cases against <projectUrl>. This will consume Muggle Test workflow budget. Proceed?"
- "Yes, start all N"
- "No, cancel"
Only proceed after the user picks "Yes".
Step 6 — Dispatch Remote Generations (Bulk)
Send a single bulk request instead of dispatching one workflow per test case:
- Call
muggle-remote-workflow-start-test-script-generation-bulk with:
projectId — from Step 2name — "muggle-test-regenerate-missing: bulk ({count} test cases)" where {count} is the number of selected test casestestCaseIds — array of all selected test case IDs from Step 5
- The backend handles looking up full test case details (goal, precondition, instructions, expectedResult, url), so there is no need to call
muggle-remote-test-case-get per test case.
- Parse the response to get the
items array with per-test-case status. Each item contains the test case ID, dispatch status, and (when successful) the workflow runtime ID.
Failure handling: the bulk API returns per-item status in the response items array. Individual test cases may fail (validation error, missing field, etc.) while others succeed. Surface failures in the Step 7 report — partial progress beats no progress.
Step 7 — Report
After the bulk dispatch returns, build a summary table from the response items array. Each item contains a test case ID, dispatch status, and (when successful) a workflow runtime ID. Cross-reference with the test case list from Step 3 to fill in titles and use case names:
Test Case Use Case Prev Status Dispatch Runtime
───────────────────────────────────────────────────────────────────────────────────────────────────
Sign up with valid email User Registration DRAFT ✅ started rt-abc123
Sign up with disposable email User Registration DRAFT ✅ started rt-def456
Add item to cart Checkout Flow GENERATION_PEND. ✅ started rt-ghi789
Apply expired coupon Checkout Flow GENERATION_PEND. ❌ failed —
───────────────────────────────────────────────────────────────────────────────────────────────────
Total: 17 dispatched | 16 started | 1 failed
For failures: include a one-line error excerpt from the item's error field and (where possible) a hint at the cause (e.g., "missing instructions field — edit the test case in the dashboard, then re-run this skill").
Per-item failure routing (regen)
For every item with a non-success dispatch status, follow _shared/failure-mode-handling.md section C (regen failure buckets: transient / infra / agent-course / product-uxux).
Because this skill is bulk-dispatch (no live user attention per item), batch the routing — don't ask the user per item:
- Group failed items by AI-classified bucket using the signal heuristics in the shared doc (most failures here will be
transient or infra since dispatch failures rarely surface product-uxux signals).
- Emit one
regen-failure-classified event per failed item via muggle-local-telemetry-event-emit.
- Present the buckets to the user via a single
AskUserQuestion summarizing counts: e.g., "12 transient (recommend retry), 3 infra (recommend report-bug). What do you want to do?" Options: "Retry transient items", "Report all to Muggle AI", "muggle-feedback for selected items", "Skip — leave them as-is".
- Emit one
regen-failure-resolved event per item with the user's batch decision applied.
Bulk regen does not run replays, so the section B (replay) router does not apply here.
Step 8 — Open the Dashboard
Open the Muggle AI dashboard so the user can watch progress visually. Point them at the scripts page (where DRAFT → GENERATING → ACTIVE transitions are visible), not /runs (which is for replay batches and shows nothing for a regen):
open "https://www.muggle-ai.com/muggleTestV0/dashboard/projects/{projectId}/scripts"
Tell them:
"I've opened the project's scripts page. Generation jobs typically take a few minutes each — your test cases will move from DRAFT through GENERATING to ACTIVE as scripts complete."
Step 9 (optional) — Poll Status
Only if the user explicitly asks for a status check, call muggle-remote-wf-get-ts-gen-latest-run for each runtime ID and report progress. Do not poll on a tight loop by default — the dispatch step is the actual goal of this skill, and the dashboard already shows live status better than a CLI loop ever can. Polling exists as a courtesy for users who don't want to leave the terminal.
If the user wants a one-shot snapshot, present a small table:
Test Case Runtime Status Steps so far
────────────────────────────────────────────────────────────
Sign up with valid email rt-abc123 RUNNING 6
Add item to cart rt-ghi789 COMPLETED 12
Tool Reference
| Phase | Tool |
|---|
| Auth | muggle-remote-auth-status, muggle-remote-auth-login, muggle-remote-auth-poll |
| Project | muggle-remote-project-list, muggle-remote-project-create |
| Scan | muggle-remote-test-case-list (paginated) |
| Dispatch | muggle-remote-workflow-start-test-script-generation-bulk |
| Status (optional) | muggle-remote-wf-get-ts-gen-latest-run, muggle-remote-wf-get-latest-ts-gen-by-tc |
| Browser | open (shell command) |
Non-negotiables
- The user MUST select the project — present projects via
AskUserQuestion, never infer from cwd, repo name, or URL guesses.
- The user MUST approve which test cases to regenerate — show the candidates via
AskUserQuestion, let them deselect, then confirm again before any dispatch. Bulk-regenerating without approval can waste meaningful workflow budget.
- Default filter is
DRAFT + GENERATION_PENDING — never include GENERATING, ACTIVE, DEPRECATED, ARCHIVED, REPLAYING, or REPLAY_PENDING unless the user explicitly says so. GENERATING already has a workflow in flight and dispatching another races against it. ACTIVE test cases already have working scripts. The rest reflect deliberate user decisions or in-flight replays the skill should not interfere with.
- Use the bulk endpoint for dispatch — call
muggle-remote-workflow-start-test-script-generation-bulk once with all selected test case IDs rather than dispatching one-by-one. The backend resolves full test case details internally.
- Failures don't abort the batch — the bulk API returns per-item status. Surface failures in the report. Partial progress beats no progress.
- Open the dashboard, don't poll by default — the runs page is the canonical view of progress. Only poll if the user explicitly asks.
- Use
AskUserQuestion for every selection — never ask the user to type a number.
- Can be invoked at any state — if the user already has a project chosen in conversation context, skip Step 2 and go straight to scanning.
