| name | canary-lab-export |
| description | Use when exporting a Canary Lab run as an evaluation — "export the evaluation", "produce the report for run 7cvh", "give me the evaluation archive", "review what the run proved" — through the export MCP tools (start/submit_external_evaluation_export, get/list/download_evaluation_export). The run must be terminal (passed, failed, or aborted). |
| type | skill |
Canary Lab — Evaluation Export
This client writes the reasoning; Canary Lab renders the final
evaluation.html + archive. These tools arrive via the Canary Lab MCP
server. If this client is already connected (the plugin connects with
full), skip this step. To configure a connection manually: npx canary-lab mcp --profile export (the composite lifecycle/full profiles carry the
same tools). The evaluation export is the deliverable of a run: a
self-contained archive (per-test reasoning, verdicts, Playwright artifacts —
video playback where the tests drive a browser) the user reviews from
evaluation.html without any Canary Lab UI running.
Workspace Bootstrap
- Read
~/.canary-lab/workspaces.json (Windows: %USERPROFILE%\.canary-lab\workspaces.json); one workspace → use it, several → ask which, none → ask the user to run npx canary-lab setup.
- Check
/mcp/health on the UI's port: read port from the workspace's canary-lab.config.json (fallback 7421), then curl -s http://127.0.0.1:<port>/mcp/health — success is a JSON response. Confirm projectRoot matches the selected workspace.
- If the health check fails, start
npx canary-lab ui from the workspace in a visible long-running terminal; if this client cannot run long-lived commands, ask the user to run npx canary-lab ui from the workspace and confirm when it's up.
Export Loop
- After the relevant run is terminal (passed, failed, or aborted), call
start_external_evaluation_export with the run id and requested language. If the user asks to export a failed or aborted run as-is, preserve that status in the report instead of trying to heal first. (list_runs / get_run find the run id when the user names a feature instead of a run.)
start_external_evaluation_export does not embed the run snapshot — call get_run(runId) for the summary/failures while authoring. Use the returned reportSchema to write the evaluation report/archive wording in this client.- Call
submit_external_evaluation_export with the taskId returned by start_external_evaluation_export (task.taskId), plus either textSlots[] or rewrite. If you submit a rewrite, rewrite.cases must keep the exact count and order of the template's cases[] — one case per run entry; never merge, dedupe, or drop skipped or duplicate run entries (prefer editing textSlots[], which keeps the count correct automatically). The result includes an evaluation digest (featureTitle, summary, per-case title + confidence) — relay it to the user in chat; don't just say it's available in the UI.
- Use
get_evaluation_export, list_evaluation_exports, or download_evaluation_export for status and the full rendered evaluation.html. Canary Lab stores the artifact, but it does not rewrite, translate, or generate the report with an internal agent for external exports.
- End by pointing the user at the archive itself — opening
evaluation.html (and the video playback for browser-driven tests) is the review step the export exists for.
Guardrails
- Keep the same
session_id for the whole conversation.
- Never heal or re-run just to make an export look better — export the run the user named, status preserved.
