e2e-exe-dev

# e2e on exe.dev For each unexpected failure, preserve the test evidence, then follow the companion [e2e-triage](../e2e-triage/SKILL.md) at a safe checkpoint and include its findings and next actions in the final E2E report. Research each distinct failure once, including when another E2E skill delegates here; do not delay active prompts, change acceptance results or submit public issues/comments without authorization. Install `e2e-triage` alongside this skill. If its relative link is unavailable, resolve it by skill name in the agent's installed catalog; if absent, report that triage was unavailable and continue the authorized test/report without installing anything implicitly. For an installation on an existing Mac, locally or over SSH, use [`e2e-macos`](../e2e-macos/SKILL.md). This skill's target workflow is Linux; its installer is shared with the Mac driver and its LaunchAgent helper. For the real public interactive installer, use the companion [e2e-wizard](../e2e-wizard/SKILL.md) and this driver's `--interactive` mode. The workflow below remains the headless test; its setup repairs and acceptance results do not qualify the wizard. Installs NanoClaw on a disposable exe.dev VM without a human at the keyboard and proves the whole loop: message in over the cli channel → host → container → Claude → reply out. It reuses the setup wizard's own step processes (`pnpm exec tsx setup/index.ts --step …`) in the wizard's order — no parallel install logic to drift — and replaces each prompt with an env var or a pre-seeded state. The assertion is the wizard's first-chat probe (`setup/lib/agent-ping.ts`): `pnpm run chat ping` must return a reply. Ships in [nanocoai/nanoclaw-oss-dev-tools](https://github.com/nanocoai/nanoclaw-oss-dev-tools) as `skills/e2e-exe-dev`, in the portable [Agent Skills](https://agentskills.io) format. Install it into any agent with `npx skills add nanocoai/nanoclaw-oss-dev-tools`, or in Claude Code with `/plugin install nanoclaw-e2e@nanoclaw-oss-dev-tools` (there it is invoked as `/nanoclaw-e2e:e2e-exe-dev`). In Codex, invoke `$e2e-exe-dev`. Resolve the scripts relative to this `SKILL.md`, then run them **from the root of the NanoClaw checkout you want tested**. The installed skill and the checkout being tested are separate directories. Install `e2e-wizard` alongside this skill; its shared read-only helper discovers the exact provider picker. Three scripts under `scripts/`: | Script | Runs on | Does | |---|---|---| | `exe-run.sh` | your machine | `ssh exe.dev new --json` (or `cp` from a base VM), pushes the selected credential and installer over SSH, clones the ref, runs the installer, exports evidence, optionally snapshots (`--snapshot`) or performs guarded deletion on pass (`--rm`) | | `e2e-install.sh` | the VM (or any Debian/Ubuntu box, or a CI runner) | the headless install + ping; exits 0 on pass | | `e2e-evidence.py` | VM and operator machine | creates, redacts and validates the headless evidence bundle before any requested teardown | ## When to use - A branch or PR touches setup, the container image, OneCLI wiring, the service, or the router/delivery path and unit tests can't prove it. - Someone reports an install failure you want to reproduce on a clean machine. - You want a base VM with Docker, OneCLI and the built image already in place so later runs skip the 3–10 minute build. Not for: wiring real channels (Slack, Telegram, …). The e2e agent talks over the always-on `cli` channel (`src/channels/cli.ts`) so no platform credentials are involved. Use `/manage-channels` on the VM afterwards if you want more. ## Prerequisites - An exe.dev account with `ssh exe.dev` working from your machine. exe.dev has two SSH destinations: `ssh exe.dev <cmd>` is the **lobby** (VM lifecycle only — no shell, no scp) and `ssh <ssh_dest>` is the **VM** (full shell). Pin the key for both so non-interactive runs never stall on key selection: ``` Host exe.dev *.exe.xyz IdentitiesOnly yes IdentityFile ~/.ssh/id_ed25519 ``` exe.dev publishes its own agent skill (`using-exe-dev`, in `skill/SKILL.md` of github.com/boldsoftware/exe.dev) and docs at https://exe.dev/docs.md (index) / https://exe.dev/docs/all.md (one page). `ssh exe.dev help <command>` is the authoritative flag reference; this skill only adds the NanoClaw side. - For the headless Claude path, an Anthropic API key or OAuth token in a local file, default `~/.nanoclaw-e2e/anthropic_key`. It goes to the VM over stdin into a `0600` file; the installer seeds it into the OneCLI vault exactly as `setup/auth.ts` does (`onecli secrets create --type anthropic --host-pattern api.anthropic.com`). `exe-run.sh` and `proxmox-run.py` refuse an empty or placeholder file, or one without an `sk-ant-` credential, before creating anything; with `--base` or a remote OneCLI vault they leave it to the installer, which checks the file only when it is about to hand it to the gateway (wizard runs already check the exact shape per auth method). - Key files for every credential-file method follow one convention: `~/.nanoclaw-e2e/<provider>_key`, mode `0600`, owned by you, one key and nothing else. `anthropic_key` (Claude; also OpenCode `custom` with `--opencode-provider anthropic`), `openrouter_key` and `deepseek_key` (OpenCode), `openai_key` (Codex `api`). The drivers take the path through `--credential-file` only; they never print, log or export a key, and the guest copy is removed before teardown. - A Debian/Ubuntu image on the VM (exe.dev's default). Both bootstrap helpers are apt-based: `setup/install-node.sh` (NodeSource) and `setup/install-docker.sh` (`get.docker.com` + `usermod -aG docker`). Docker runs inside exe.dev VMs. ## Choose the provider and authentication first Resolve the exact NanoClaw commit and inspect its provider picker before reading a credential or provisioning anything. The discovery helper belongs to the sibling `e2e-wizard` skill and reads Git objects, so uncommitted files cannot change the choices: ```bash COMMIT="$(git rev-parse --verify HEAD^{commit})" PROVIDER_HELPER=/absolute/path/to/installed/e2e-wizard/scripts/provider-options.py python3 "$PROVIDER_HELPER" --root "$PWD" --revision "$COMMIT" ``` Show the reported `providers` to the operator and ask which provider to test. Then inspect that provider and show its exact `auth_prompt` and `auth_methods`: ```bash python3 "$PROVIDER_HELPER" --root "$PWD" --revision "$COMMIT" --provider claude ``` For a bundled provider (`payload_kind: bundled`), use the NanoClaw SHA directly without `--payload-ref`. For a branch-owned provider, first fetch the `nc:copy from-branch:` payload named by its offered skill from the owning remote, then pass that fetched ref as `--payload-ref`. A skill may declare both (Codex on nanoclaw `290aa683` copies its runtime from the `providers` branch and bundles its auth hook): discovery reports `payload_kind: mixed`, lists every source with its commit under `payload_sources`, stamps each file with the commit it comes from, and takes `auth_source_commit` from the block that carries `setup/providers/<provider>.ts`. Record both `nanoclaw_commit` and `auth_source_commit`. Never guess a payload remote or silently choose a provider or auth method. The unattended headless installer currently supports Claude `api` and `oauth`. Its `existing` mode is only for an explicitly selected reused gateway whose vault already has a usable Anthropic credential; it is not a public-picker choice. Every other offered provider goes through the public wizard (`--interactive`), which automates the credential-file methods those providers expose and, with `--gateway onecli|iron-proxy`, installs the requested gateway on a seam ref. OpenCode runs require `--interactive` and `--opencode-model`; custom/self-hosted endpoints also use `--opencode-base-url` and an API-key file. Follow [OpenCode wizard coverage](../e2e-wizard/SKILL.md#opencode). Codex `device` pairing is the one human handoff this driver supports, behind `--supervised-human-auth` (see [Supervised authentication](../e2e-wizard/references/supervised-auth.md)); browser and Claude subscription methods still stop before allocation because their handoffs cannot be completed from here. `skip` cannot produce an E2E pass. Read a credential only after the operator chooses its matching method. ## Workflow 1. **Run it** from the checkout root, on the branch you want tested. Set `E2E_SKILL_DIR` to the actual directory containing this `SKILL.md`: ```bash E2E_SKILL_DIR=/absolute/path/to/installed/e2e-exe-dev cd /absolute/path/to/nanoclaw bash "$E2E_SKILL_DIR/scripts/exe-run.sh" \ --provider claude --auth-method api --credential-file /path/to/anthropic-key \ --result-file /path/to/results/exe-headless.json bash "$E2E_SKILL_DIR/scripts/exe-run.sh" --ref origin/main --name nc-main \ --provider claude --auth-method oauth --credential-file /path/to/anthropic-oauth-token \ --result-file /path/to/results/exe-main.json bash "$E2E_SKILL_DIR/scripts/exe-run.sh" --ref feat/iron --gateway iron-proxy \ --provider claude --auth-method oauth --credential-file /path/to/anthropic-oauth-token \ --result-file /path/to/results/exe-iron.json ``` `--gateway onecli|iron-proxy` picks the gateway on a seam ref ([below](#refs-on-the-gateway-seam)); an unknown value stops before any VM work; `--interactive` rejects it. Keep `--result-file` on every live run so the report and sanitized evidence exist before the post-run retention choice. For an already authorized disposable run, add `--rm` (the result parent directory must exist). `--rm` is rejected without `--result-file`; `<result-file>.artifacts` must be a new local path so teardown cannot precede validated evidence persistence. `--ref` resolves in the local checkout to an exact commit before creating a VM. Fetch locally first if you want an updated `origin/main`. The VM fetches that SHA from `--repo` (default: local `origin`), checks it out detached, and verifies `HEAD` matches. The commit must be fetchable from that repository; local-only commits and uncommitted edits are not uploaded. The installer comes from the installed skill, so compatible older NanoClaw refs can be tested too. 2. **Read the result.** The last thing printed on success is ``` === NANOCLAW E2E: RESULT === STATUS: pass COMMIT: <full tested SHA> SERVICE_TYPE: systemd-user | systemd-system | nohup PING: ok REPLY: <first 200 chars of the agent's reply> RESULT: logs/e2e/result.json === END === ``` Exit codes: `0` pass · `1` a step failed (its `=== NANOCLAW SETUP: … ===` block is printed right above the failure; raw output in `~/nanoclaw/logs/e2e/<step>.log` on the VM) · `2` no reply / auth error · `3` the host never opened `data/cli.sock` or the socket was unreachable. The driver uses `64` for invalid arguments, `65` for checkout/ref errors, `66` for an unreadable key, `69` for unconfirmed VM creation/reachability, `70` for unconfirmed snapshot creation, and `74` for local result writing/export failure; other command failures can return their own nonzero exit code. After installer preflight, `logs/e2e/result.json` records pass/failure, exit code, tested commit, tracked changes present at start, phase, service type, ping classification, timestamps, the installed gateway (`gateway`), the requested one (`requested_gateway`) and whether the seam's gateway step ran (`gateway_seam`); the summary block prints a `GATEWAY:` line. An in-progress run has status `running`. It replaces a previous run's result and contains no credentials or reply text. With `--result-file`, after argument parsing and destination validation the driver replaces any old local report with `running` before checking the checkout, credentials or VM. It exports a matching completed installer result and a sanitized evidence directory before snapshot/removal. The evidence binds the run ID, NanoClaw SHA, provider, auth method, provider auth-source SHA, gateway kind, dev-tools SHA when available, and exact harness digest; an installer result naming a different gateway fails the export and keeps the VM. The checkout's HEAD must be the tested SHA, or that SHA plus only setup's own `setup: apply <skill>` commits (recorded as `checkout_head`); any other commit fails the export. A driver-only failure report has a null `gateway`. It includes checksummed setup/runtime logs and a service/container/socket state snapshot. Otherwise the driver records the driver failure, exit code, phase and requested ref/commit; `commit` is null because no tested revision was confirmed. An interrupted run may remain `running`. A failed export keeps the VM. Once exported, the installer result describes the test; snapshot/removal errors are reported by the driver's exit code. On the VM, a checkout failure on a base leaves a `running` record with phase `checkout` and the requested commit, replacing any old pass. Failures before installer preflight on a fresh VM may have no remote result file; the local driver report still records the failure. 3. **Bake a base VM** once the run is green, then clone it for every later run: ```bash bash "$E2E_SKILL_DIR/scripts/exe-run.sh" --snapshot my-nanoclaw-e2e-base \ --provider claude --auth-method api --credential-file /path/to/anthropic-key \ --result-file /path/to/results/base.json bash "$E2E_SKILL_DIR/scripts/exe-run.sh" --base my-nanoclaw-e2e-base --ref my-branch \ --provider claude --auth-method api --credential-file /path/to/anthropic-key \ --result-file /path/to/results/cached.json ``` VM names are global across exe.dev (they become `<name>.exe.xyz`), so pick a distinctive snapshot name. Default run names include a random suffix. A taken name or invalid create/copy response stops the run; an existing VM is never adopted as a replacement. A failed snapshot keeps the tested VM, even with `--rm`. Every step is idempotent against the cloned state: `onecli --reuse` when `.env` already has `ONECLI_URL`, `auth --check` short-circuits on an existing vault secret, `container` rebuilds only what changed, `init-cli-agent` reuses the group and wiring. Cached checkouts are pointed at the requested repository and exact SHA; tracked edits in a base checkout cause a failure instead of being overwritten. 4. **Poke at it.** `ssh <ssh_dest>` (printed at the end; `ssh exe.dev ls --json` lists it), then in `~/nanoclaw`: `pnpm run chat hi`, `bin/ncl groups list`, `tail -f logs/nanoclaw.log`. 5. **Choose retention after evidence is safe.** Recommend deleting a disposable, run-owned VM after a pass and retaining an unexpected failure until triage is complete. Offer the operator that choice after reporting the result. With `--rm`, the driver revalidates its run marker, confirms no harness controller is active, calls `ssh exe.dev rm <name> --json`, verifies the name is absent from `ssh exe.dev ls --json`, and writes `teardown-receipt.json` into the validated artifact directory. Any failed export, checksum, redaction, local persistence, ownership, controller, or absence check blocks or leaves teardown unconfirmed. For manual cleanup, repeat the same checks and record the receipt; inventory lookup alone never proves ownership. ### Running the installer somewhere else `e2e-install.sh` has no exe.dev dependency. On a CI runner or any fresh box: ```bash git clone https://github.com/nanocoai/nanoclaw-oss-dev-tools.git E2E_SKILL_DIR="$(pwd)/nanoclaw-oss-dev-tools/skills/e2e-exe-dev" git clone https://github.com/nanocoai/nanoclaw.git cd nanoclaw NANOCLAW_E2E_PROVIDER=claude NANOCLAW_E2E_AUTH_METHOD=api \ NANOCLAW_E2E_AUTH_SOURCE_COMMIT="$(git rev-parse HEAD)" \ NANOCLAW_E2E_KEY_FILE=/path/to/key bash "$E2E_SKILL_DIR/scripts/e2e-install.sh" ``` The target needs `git` and `python3` before starting the installer. The exe.dev driver installs these if missing; provision them yourself on other hosts. Env it honors: `NANOCLAW_E2E_ROOT`, `NANOCLAW_E2E_KEY_FILE`, `NANOCLAW_E2E_GATEWAY` (`onecli`, the default, or `iron-proxy`; only refs on the gateway seam can select Iron — see below), `NANOCLAW_ONECLI_API_HOST` + `NANOCLAW_ONECLI_API_TOKEN` (remote gateway instead of a local install — the same vars `setup/auto.ts` reads), `NANOCLAW_DISPLAY_NAME`, `NANOCLAW_E2E_TZ` (default `UTC`), `NANOCLAW_E2E_KEEP_AGENT` (default `1`), `NANOCLAW_E2E_FORCE_AUTH` (`1` replaces an existing vault secret with the key file — token rotation on a `--base` VM whose snapshot still holds the old one, or a rerun after a bad-key test). The installer creates the new secret, then deletes every older `anthropic` secret for `api.anthropic.com`, on both the gateway seam and the legacy `auth` step; neither replaces on its own, and OneCLI keeps using an older duplicate. It also stops any running container that mounts this checkout's `groups/e2e-agent`, which would otherwise keep its old session for the ping. The driver forwards the remote gateway settings, display name, timezone, keep-agent and force-auth settings over stdin into a private environment file, which is removed after loading. `NANOCLAW_E2E_KEY_FILE` (or `--key-file`) selects the local key to upload; `NANOCLAW_E2E_ROOT` applies to standalone installer runs. The driver always uses `~/nanoclaw` on its VM. ## Compatibility evidence On **2026-09-12** a fresh exe.dev VM failed at the `onecli` step against NanoClaw `fd767d381fe5a1fea51373e4e8cb411da79880ec` with `Port 5432 is already