kane-cli

# Kane CLI — End-to-End Test Generation and Execution Skill KaneCLI is one E2E package with three pillars: 1. **Generate**: requirements → use-cases → acceptance criteria → scenarios → runnable `_test.md` tests (the assurance commands). 2. **Execute**: desktop web (Chrome, the default), Android emulator, iOS simulator, and real Android and iOS devices in the TestMu AI cloud. 3. **Prove**: a sealed evidence pack per run, and coverage (`kane-cli cover`) measured against acceptance criteria. **Working with a team?** On kane-cli 0.10.0+ a team works in a **shared project**: one Test Manager project that its folders are linked to. What one person saves there (use-cases, tests, Test Manager cases, coverage) the whole team sees. Test files stay in the folder where they were saved until they're brought over (`references/shared-projects.md` §7). In agent mode, a folder that isn't set up yet goes local unless you link it with `--project <id>`. So decide where the work goes before the first assurance command, and read `references/shared-projects.md` first. Use `kane-cli` to validate complete user journeys and design requirement-linked, coverage-accounted suites from a PRD, specification, or user-provided description via assurance commands. Never write test cases by hand. Do NOT use Playwright, Puppeteer, or Selenium directly. Use `--agent` for `run` and `testmd run`. `testrun run` emits NDJSON when **stdin** is not a TTY (0.8.21+ also takes `--agent`, which selects the same stream), so every `testrun run` line you write ends in `< /dev/null` (bash and zsh: macOS, Linux, Git Bash; in cmd.exe write `< NUL`; from PowerShell run it through cmd: `cmd /c "kane-cli testrun run … < NUL"`). Check the first stdout line: on 0.8.17+ it is `{"type":"stream_start"…}`. A prose plan there instead means the CLI is in terminal mode: it shows the human view and, after a real run, opens an evidence table that waits until `q` or Esc, so the process never exits on its own; stop it and rerun with the redirect. An empty stdout with a message on stderr is a usage error: read it. Assurance conversational commands use `--mode agent`. **Authoring test cases or scenarios?** Never write them by hand: every test case comes from the **assurance** commands — Read `references/assurance.md` first. - The user has **requirement documents** (a PRD, a spec, acceptance notes) → ingest them, then design. - The user only **describes** what to test, in chat → write their description, in their words, to a requirements file, ingest that file, then design (§6). The description is the requirement. Don't draft test cases in chat or scratch files: design produces structured, refinable, runnable `_test.md` output. --- ## 1. Every session: ready check, launch, result card. READ THIS FIRST A one-line "Test passed" instead of the result card is a bug. A run that starts with no ready check is a bug too. This section is first so you don't miss it. Follow it exactly. The order never changes: **ready check → launch line → the run → result card**. On a person's first session two things are added: a short tour sent with the launch line, and three choices asked after the first result. Nothing is asked before the first result. ### 1.1 Start with the ready check Before the first kane-cli command of a session, run the preflight script that ships with this skill and show the ready card. It is one short command and takes about two seconds: ```bash sh "<skill dir>/scripts/preflight.sh" ``` `<skill dir>` is the folder that holds this `SKILL.md`. **Read `references/ready-check.md`** for the card (a full table on the first session, one line afterwards), the problems that stop a run, and the sign-in flow. Two rules matter enough to repeat here: you start sign-in yourself with `kane-cli login --oauth`, and you **never ask for an access key or password in chat**. If the preflight shows no saved preferences (its `## agent-config` section is `none`, or has no `onboarding.completed_at`), this is the person's first session: **Read `references/first-run.md`** before you launch. ### 1.2 How to launch kane-cli **All platforms use your shell tool** (`Bash`) to launch kane-cli. Do NOT use `Monitor`: it creates excessive noise. **Tag every command with your runtime.** Put `KANE_CLI_USER_AGENT=<your-runtime>` in front of every `kane-cli` command you run. Pick a short, stable identifier for the AI assistant or IDE running this skill (e.g. `claude-code`, `codex`, `gemini`, `cursor`, `kiro`, `aider`) and use the same value for the whole session. Do it inline on each command: an `export` does not survive from one shell call to the next in most agent hosts. Do not show the prefix in commands you quote to the person. ```bash KANE_CLI_USER_AGENT=<your-runtime> kane-cli run "<objective>" --agent <other-flags> ``` On Windows PowerShell: `$env:KANE_CLI_USER_AGENT='<your-runtime>'; kane-cli run "<objective>" --agent <other-flags>`. **Watch mode.** Use the person's saved preference (`references/agent-config.md`). With none saved, show the browser unless the preflight says there is no display, an SSH session, or CI: then add `--headless`. **Keeping runs.** When the person's saved purpose is `suite` or `ask`, add `--name <short-slug>` to every one-off `run`. A named run is recorded as a `_test.md` while it runs, so keeping it afterwards costs nothing, and a run launched without a name cannot be kept without running again. With `one-off`, leave the flag out. Details: `references/first-run.md` §4. Run one-offs from a folder that isn't linked: a named run in a linked folder writes its test beside the project's and can mix two projects (`references/shared-projects.md` §9). Bash blocks until kane-cli exits, then hands you the complete stdout. Parse it, summarize what happened, and present the result card. Wait for process completion on `testmd run` too, but parse its own completion event, `test_md_done`. An intermediate `run_end` does not finish a saved test. Set a generous timeout (up to 600000ms) since browser runs can take a while. ### 1.3 Before you launch In one message, **before** invoking Bash, send the ready card and then: ```text Starting browser task: <one-line restatement of the user's objective>. ``` That line tells the user something is in progress. No todos needed: Bash returns all output at once and you summarize it below. On a first session, the tour from `references/first-run.md` goes in this same message, right after the launch line, so the person reads it while the run works. ### 1.4 After the run: summarize what happened Once Bash returns, parse the captured NDJSON stdout and present a **concise summary** of what happened. Not every event deserves a line. Surface what matters and skip the noise. (Skip the summary entirely when the person's preference is `results-only`.) Progress events have `step`/`status`/`remark` fields and **no `type` field**. #### What to surface | Show | Which events | How | |------|-------------|-----| | **Failures** | Any step with `status: "failed"` | `Step <n> failed: <remark>` | | **Flow changes** | `bifurcation`, `child_agent_start`, `child_agent_end` | Plain-language one-liner (e.g. "The agent split the objective into 2 sub-tasks") | | **Errors** | `error` typed events | `Error: <message>` | | **Unresolved variables** | `warning` with `code: "unresolved_variables"` | Before any progress: name the variables with no value and where a value goes (§3 **Unresolved variables**). The run continued | | **Overall progress** | All passing steps | One summary line: `<total> steps completed: <2–4 key actions from remarks>` | #### What to skip - Individual passing steps: fold them into the overall progress line - Internal field names (`step`, `status`, `remark`, `run_end`, `final_state`, `bifurcation`, `session_dir`, `stream_start`, `project_folder_auto_defaulted`, etc.): translate to plain language. A `project_folder_auto_defaulted` event fires before progress when the run-startup gate auto-resolves a project/folder; surface it as one line ("kane-cli auto-selected project X / folder Y for this run") and move on. Details: `references/test-manager.md`. For short runs (≤ 3 steps), you may list each step individually since there's nothing to fold. ### 1.5 The result card The terminal event has `type: "run_end"` and stable fields: `status`, `summary`, `one_liner`, `duration`, `credits_consumed`, `final_state`, `test_url`, `session_dir`, `run_dir`. **For a passing run, always emit this exact table** (substituting the field values): ```markdown | | | |-------|-------| | 🟢 **Result** | Passed | | 🎯 **Task** | <one_liner> | | ⏱️ **Duration** | <duration, as 1m 54s> | | 👣 **Steps taken** | <count of completed progress events (done/failed), retaining child/execution context> | | 💳 **Credits** | <credits_consumed, rounded> used · about <ready check balance minus used> left | | 📝 **What happened** | <summary> | | 📁 **Evidence** | Want to open the run evidence in your browser? | | 🔗 **Test case** | [Open in Test Manager](<test_url>) | | ➡️ **Next** | <two things you can do right now, e.g. add a payment step · run it headless in CI> | ``` **If `final_state` has values** (the user used "store as X", see §4), append a second table: ```markdown | 📦 What was found | Value | |-------------|----------------| | <key from final_state, humanized> | <value> | ``` **If the objective used assertions** ("assert …", "verify …"), append a pass/fail table per assertion derived from the run summary and step remarks. Every other result has its own card in **`references/cards.md`**: a run that didn't start, one that stopped early, a possible product bug, a saved test, and a suite (local or cloud grid). Read it before presenting any of those. The rules there apply to every card: one short sentence per cell, failures first, `➡️ Next` is an offer you can act on, and secret-looking values never go in chat. ### 1.6 On failure For exit code 1 (or `status: "failed"` in `run_end`), present the failure card. Never show raw paths or NDJSON. ```markdown | | | |-------|-------| | 🔴 **Result** | Failed at step <n> of <total> | | 🎯 **Task** | <one_liner, or the objective in a few words> | | ⏱️ **Duration** | <duration, as 1m 12s> | | 💳 **Credits** | <credits_consumed, rounded> used | | 📝 **What happened** | <plain-language description of the failing step's remark> | | 🔍 **Likely cause** | <your diagnosis: missing element, slow page, ambiguous objective, auth wall, etc.> | | 📁 **Evidence** | Want to open the run evidence in your browser? | | ➡️ **Next** | <a retry you can run now> · <walk through the failing step> | ``` The failing step's screenshot lives inside the run's evidence pack (the stderr hint names the pack path): extract it with `unzip <pack> "tests/*/steps/*/screenshot.png" -d <tmpdir>`, Read it, and show it **under** the card. For the pack layout and deeper diagnosis, see `references/debug.md`. Exit code 2 with no progress events means nothing ran: that is a `🟡 Didn't start` card, not a failure (`references/cards.md` §4). If progress events came before it, steps ran and the run stopped without a result: say so, and use the stopped-early card (`references/cards.md` §5). ### 1.7 After the first result: three choices, then save On a first session only, right after the first result card, save the defaults this run used, then ask the three choices from `references/first-run.md` §4 (watch mode, where results go, one-off or saved suite) as the **last thing in your turn**, and save the answers when they arrive. Some hosts hand control back before the person answers: end your turn there and save on their reply. On every later session none of this is asked again. **The live status strip (Claude Code only) has its own once-only question.** Onboarding is shared by every agent the person uses, but the strip exists only in Claude Code, so the person may have finished their first session in another agent without ever being asked. In Claude Code, in **any** session: if the preflight's `## agent-config` has no `strip.claude-code.offered_at`, kane-cli is 0.8.17 or newer, and `node=` is not empty, ask the strip question once, after that session's first result card, as the last thing in your turn. On a first session it simply rides along as the fourth choice. It is recommended, never turned on without a yes, and you record `offered_at` either way so it is never asked twice. **Read `references/live-strip.md` §3** for the wording. --- ## 2. Decision tree When the user's request involves a browser — or writing test cases: **Is kane-cli installed, signed in and ready?** - Unknown → run the preflight and show the ready card (§1.1, `references/ready-check.md`) - A problem that stops the run → offer the fix from the card; deeper setup lives in `references/setup-and-config.md` - Ready ↓ **Where does the work go?** (before any assurance command) - The folder holds its own `.context/` and no link → it works locally, as before (with a link too, kane-cli refuses it `BOTH_STORES`): `references/assurance.md` (to share that store with a team, `references/context-sync.md`). The rows below are for a folder without one - The folder holds `.testmuai/assurance.json`, or the person works with a team or names a Test Manager project → a **shared project**: Read `references/shared-projects.md` first. In a linked folder never use `kane-cli run` for a first result; its tests go to another project (§9 there) - The person works alone, or asks to keep the work on this machine → the local store: `references/assurance.md` - Not clear → ask once: "Should this work go to your team's Test Manager project, or stay in this folder?" **What does the user want?** - Tests that exploration (HyperExecute) designed, or a teammate's tests, in this folder → Read `references/shared-projects.md` §7 - Explore an app on HyperExecute, shape, stop or re-run an exploration, or check what it designed → Read `references/explore.md` first - A command was refused, or you need the shared-project event and code list → Read `references/agent-contract.md` - A single one-shot browser task → build a `kane-cli run --agent` command (§3 + §4) - A test they want to save / re-run / commit → Read `references/testmd.md` first, then use `kane-cli testmd` - Run a suite of saved tests (several `_test.md` at once) → Read `references/testrun.md` first, then use `kane-cli testrun run … < /dev/null` - Need test cases from a description the user gives in chat, with no document → **don't hand-write them**; write the description to a requirements file and design from it (§6) - Has requirement documents (PRD/spec) and wants a designed suite, coverage accounting, or suite upkeep → Read `references/assurance.md` first — the assurance commands (`context`/`design`/`cover`/`maintain reconcile`, kane-cli 0.6.1+; several features need newer releases, up to 0.8.14+ — the reference marks each) - A **designed** test (its `_test.md` carries an `assurance:` block) failed a run → Read `references/assurance.md` §6.1 **before touching the file** — an edit is adopted as the next design version on the next run; classify the failure first (app bug → report it; requirement changed → reconcile; wording → redesign through the CLI; capability missing → stop), and edit a step of an existing designed test by hand only after reading `references/objectives-cookbook.md` - Share a **local** context store through a repository, a bucket or a folder, join one, keep two stores level, or resolve a sync conflict → Read `references/context-sync.md` first (kane-cli 0.8.14+). A team in Test Manager uses a shared project instead (`references/shared-projects.md`); a linked folder refuses `kane-cli context sync`, `kane-cli context push` and `kane-cli context clone` (`REMOTE_UNSUPPORTED`), and a location pull (a location name, `--rebase`, `--yes` or `--answer`: `PULL_LOCATION_LINKED`); a plain `kane-cli context pull --mode agent` there brings the project's tests (`references/shared-projects.md` §7); the store is shared through a location, never by copying or git-merging `.context/` - Multiple independent browser tasks → Read `references/parallel.md` first - View, share, or validate run evidence (`.evidence` packs) → Read `references/evidence.md` - Judge a run or pack against quality gates (a policy YAML, `--policy`, `--policy-id`, `kane-cli evidence verdict`, `kane-cli policy …`), or gate CI on more than pass/fail → Read `references/policy.md` first (after 0.10.0-rc.3) - Debug a failed run → Read `references/debug.md` - Configure kane-cli or check directory layout → Read `references/setup-and-config.md`