crabbox

# Crabbox ## Provider Selection Contract Provider selection is configuration resolution, not testing policy. Run `crabbox config show` from the exact worktree being tested; a session may have started in another checkout whose repository config or skill text is stale. A request to "use Crabbox" does not authorize a provider override. When the user does not name a backend, omit `--provider` and preserve the resolved provider, including values supplied by user config or `CRABBOX_PROVIDER`. Test scope, expected duration, and hydration failures do not select a provider. Override the resolved provider only when the user explicitly names a different backend. If that provider cannot perform the requested test, report the blocker instead of silently selecting another backend. Use the Crabbox wrapper for validation execution, environment lifecycle, sync, logs/results, cache inspection, and lease cleanup. Crabbox is the transport/orchestration surface. The actual backend can be: - local Docker isolation: `provider=local-container`, lease ids like `cbx_...` - brokered AWS Crabbox: direct provider, `provider=aws`, lease ids like `cbx_...`, `syncDelegated=false` - Blacksmith Testbox through Crabbox: delegated provider, `provider=blacksmith-testbox`, ids like `tbx_...`, `syncDelegated=true` The provider-specific sections below document capabilities and commands after configuration or an explicit user request has selected a backend. They are not provider-selection policy. Their validation commands intentionally omit `--provider`; add it only to carry out an explicit user override of the resolved configuration. Provider identity must remain attached to a lease after creation. A lease created through the resolved configuration can keep using provider-neutral commands while that configuration is unchanged. If the user explicitly overrode the provider, or the resolved configuration has changed since the lease was created, pass the provider reported by Crabbox to every later command that targets that lease, including hydrate, rerun, desktop/WebVNC, status, inspect, SSH, and stop. This preserves resource identity; it does not select a provider for unrelated work. When the resolved configuration selects Blacksmith Testbox, or the user explicitly requests it, do not describe the run as "AWS Crabbox". Report it as Testbox-through-Crabbox with the `tbx_...` id and Actions run. Direct AWS supports persistent leases, `--fresh-pr`, `--full-resync`, environment forwarding, capture/download, and provider-comparison workflows. Blacksmith Testbox provides a prepared CI environment and delegated broad/heavy `pnpm` gates. These capabilities do not override the resolved configuration. ## First Checks - Run from the repo root. Crabbox sync mirrors the current checkout. - Check the wrapper and providers before Crabbox work: ```sh command -v crabbox ../crabbox/bin/crabbox --version ../crabbox/bin/crabbox run --help | sed -n '1,120p' ../crabbox/bin/crabbox desktop launch --help ../crabbox/bin/crabbox webvnc --help ``` - OpenClaw scripts prefer `../crabbox/bin/crabbox` when present. The user PATH shim can be stale. - Read `.crabbox.yaml`, then run `crabbox config show` from the repository root before selecting a provider. The resolved provider can come from the user's Crabbox config or environment as well as the repository file. - Omitting `--provider` means "use the resolved configuration". Pass it only when the user explicitly requests a different backend, and report the provider from Crabbox output rather than inferring it from bootstrap wording. - Treat `CRABBOX_PROVIDER` as part of that resolved provider configuration. The provider reported by `crabbox config show` is authoritative; a hydration or test failure is not permission to switch backends. Override it only when the user explicitly names another provider. - Some package-manager wrappers insert a command delimiter before Crabbox options. If the printed command looks like `crabbox run -- --provider ...` or `crabbox run -- --no-hydrate ...`, call the trusted Crabbox binary directly so lifecycle flags are parsed by Crabbox. - The brokered AWS default is a Linux developer image in `eu-west-1`; the repo config pins hot `eu-west-1a/b/c` placement so Fast Snapshot Restore can apply. If warmup drifts well past the minute-scale path, verify image promotion, region/AZ placement, and FSR state before blaming OpenClaw. - Targeted and broad `pnpm` gates both use the resolved provider. Test size does not authorize a provider override. - Always report the provider field and id. Both local-container and direct providers can use `cbx_...`, so the id prefix alone does not prove AWS; `tbx_...` identifies Blacksmith Testbox through Crabbox. If the output only says `blacksmith testbox list`, use `blacksmith testbox list --all` before concluding no box exists. - If a warm direct-provider lease smells stale, retry with `--full-resync` (alias `--fresh-sync`) before replacing the lease. This resets the remote workdir, skips the fingerprint fast path, reseeds Git when possible, and uploads the checkout from scratch. - For live/provider bugs, use the configured secret workflow before downgrading to mocks. Copy only the exact needed key into the remote process environment for that one command. Do not print it, do not sync it as a repo file, and do not leave it in remote shell history or logs. If no secret-safe injection path is available, say true live provider auth is blocked instead of silently using a fake key. - Targeted versus broad describes test scope only; it does not imply local, remote, host, or container execution. - Do not treat unrelated inherited shell controls as operator intent. In particular, `OPENCLAW_LOCAL_CHECK_MODE=throttled` from the local shell is not permission to move broad `pnpm check:changed`, `pnpm test:changed`, full `pnpm test`, or lint/typecheck fan-out onto the laptop. This does not negate provider-selection variables such as `CRABBOX_PROVIDER`, which are resolved and reported by `crabbox config show`. - `OPENCLAW_LOCAL_CHECK_MODE=throttled|full` controls direct host execution; it is separate from Crabbox provider selection and must not override the provider reported by `crabbox config show`. ## macOS And Windows Targets Use these only when the task needs an existing non-Linux host. OpenClaw Linux validation uses the resolved Crabbox configuration unless the user explicitly requests another provider or target. Native Windows is available when the resolved provider supports that target. For providers other than AWS, keep placement and capacity in their resolved configuration: ```sh ../crabbox/bin/crabbox warmup \ --target windows \ --windows-mode normal \ --timing-json ``` AWS has a provider-specific placement requirement: the OpenClaw Windows developer image and Docker cache are available in `us-west-2`, and native Windows leases must use on-demand capacity. After `crabbox config show` confirms `provider=aws` (or the user explicitly selects AWS), use: ```sh CRABBOX_AWS_REGION=us-west-2 \ CRABBOX_CAPACITY_REGIONS=us-west-2 \ ../crabbox/bin/crabbox warmup \ --target windows \ --windows-mode normal \ --market on-demand \ --timing-json ``` The region variables pin both the AWS image lookup and the capacity candidate set because `warmup` does not expose a generic `--region` flag. These placement and market settings are AWS availability constraints applied after provider selection; they are not a reason to select AWS for other runs. If the user explicitly requests AWS while `crabbox config show` resolves another provider, add `--provider aws` to this command to carry out that explicit override. The hydrate workflow assumes Docker should already be baked into Linux images and only installs it as a fallback. Do not add per-run Docker installs to proof commands unless the image probe shows Docker is actually missing. When the user explicitly asks for brokered macOS runners, use Crabbox AWS macOS only after confirming the deployed coordinator supports EC2 Mac host lifecycle/image routes and the operator has AWS EC2 Mac Dedicated Host quota and IAM. Prefer `CRABBOX_HOST_ID` for a known Crabbox-managed Dedicated Host, or run the no-spend preflight first: These administration commands specify AWS because they inspect or allocate an AWS EC2 Dedicated Host. The explicit provider is a billing and resource-scope safety boundary, not a validation default. ```sh crabbox admin hosts quota --provider aws --target macos --region eu-west-1 --type mac2.metal --json crabbox admin hosts allocate --provider aws --target macos --region eu-west-1 --type mac2.metal --dry-run --json CRABBOX_MACOS_TYPES=all scripts/macos-host-region-preflight.sh ``` Do not silently substitute AWS macOS for normal OpenClaw Linux proof. Report paid-host blockers as quota, IAM, coordinator deployment, or host availability instead of falling back to local macOS. Crabbox supports static SSH targets: These commands specify `ssh` because the user-supplied static host selects that transport explicitly; they do not establish an SSH preference for other runs. ```sh ../crabbox/bin/crabbox run --provider ssh --target macos --static-host mac-studio.local -- xcodebuild test ../crabbox/bin/crabbox run --provider ssh --target windows --windows-mode normal --static-host win-dev.local -- pwsh -NoProfile -Command "dotnet test" ../crabbox/bin/crabbox run --provider ssh --target windows --windows-mode wsl2 --static-host win-dev.local -- pnpm test ``` - `target=macos` and `target=windows --windows-mode wsl2` use the POSIX SSH, bash, Git, rsync, and tar contract. - Native Windows uses OpenSSH, PowerShell, Git, and tar; sync is manifest tar archive transfer into `static.workRoot`. Direct native Windows runs support `--script*`, `--env-from-profile`, `--preflight`, and PowerShell `--shell`. - `crabbox actions hydrate/register` are Linux-only today; use plain `crabbox run` loops for static macOS and Windows hosts. - Live proof needs a reachable, operator-managed SSH host. Without one, verify with `../crabbox/bin/crabbox run --help`, config/flag tests, and the Crabbox Go test suite. ## Direct Brokered AWS Backend This section applies only after the resolved configuration or an explicit user request selects direct AWS Crabbox. Changed gate: ```sh ../crabbox/bin/crabbox run \ --idle-timeout 90m \ --ttl 240m \ --timing-json \ --shell -- \ "env CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test:changed" ``` Full suite: ```sh ../crabbox/bin/crabbox run \ --idle-timeout 90m \ --ttl 240m \ --timing-json \ --shell -- \ "env CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test" ``` Focused rerun: ```sh ../crabbox/bin/crabbox run \ --idle-timeout 90m \ --ttl 240m \ --timing-json \ --shell -- \ "env CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 pnpm test <path-or-filter>" ``` Read the JSON summary. Useful fields: - `provider`: `aws` - `leaseId`: `cbx_...` - `syncDelegated`: `false` - `commandPhases`: populated when the command prints `CRABBOX_PHASE:<name>` - `commandMs` / `totalMs` - `exitCode` Crabbox should stop one-shot AWS leases automatically after the run. Verify cleanup when a run fails, is interrupted, or the command output is unclear: ```sh ../crabbox/bin/crabbox list --provider aws ``` The explicit provider keeps cleanup attached to the AWS lease recorded by the run, including when AWS was a user-requested override of another configured default. ## Blacksmith Testbox Through Crabbox This section applies only after the resolved configuration or an explicit user request selects Blacksmith Testbox: A fresh Testbox requires a repository-specific workflow. Crabbox resolves it from `blacksmith.workflow`, falling back to `actions.workflow`; job and ref use the equivalent Blacksmith values with Actions fallbacks. The workflow must contain a `useblacksmith/testbox`, `useblacksmith/begin-testbox`, or `useblacksmith/run-testbox` step. Confirm both the effective config and the actual workflow before using the provider-neutral command below; nonempty fields alone do not prove that an ordinary Actions workflow is Testbox-capable. For a local workflow path, validate it with: ```sh rg -n 'useblacksmith/(testbox|begin-testbox|run-testbox)' <workflow-path> ``` For a workflow name or id, inspect its YAML through GitHub before dispatch. Do not guess a Testbox workflow from another repository. If the user explicitly supplies a Blacksmith override, carry those exact values on the command instead: ```sh ../crabbox/bin/crabbox run \ --provider blacksmith-testbox \ --blacksmith-workflow <repository-workflow> \ --timing-json -- <test-command> ``` Add `--blacksmith-org`, `--blacksmith-job`, and `--blacksmith-ref` only when the user supplies those overrides. The explicit provider and workflow flags in this form implement the user's repository-specific selection; they are not defaults for other runs. ```sh ../crabbox/bin/crabbox run \ --idle-timeout 90m \ --ttl 240m \ --timing-json \ -- \ CI=1 NODE_OPTIONS=--max-old-space-size=4096 OPENCLAW_TEST_PROJECTS_PARALLEL=6 OPENCLAW_VITEST_MAX_WORKERS=1 OPENCLAW_VITEST_NO_OUTPUT_TIMEOUT_MS=900000 OPENCLAW_TESTBOX=1 OPENCLAW_TESTBOX_REMOTE_RUN=1 pnpm check:changed ``` Read the JSON summary and the Testbox line. Useful fields: - `provider`: `blacksmith-testbox` - `leaseId`: `tbx_...` - `syncDelegated`: `true` - `syncPhases`: delegated/skipped because Blacksmith owns checkout/sync - Actions run URL/id from the Testbox output - `exitCode` `blacksmith testbox list` may hide hydrating or ready boxes. Use: ```sh blacksmith testbox list --all blacksmith testbox status <tbx_id> ``` ## Observability Flags Use these on debugging runs before inventing ad hoc logging: - `--preflight`: prints run context, workspace mode, SSH target, remote user/cwd, and target-specific tool probes. Defaults cover `git`, `tar`, `node`, `npm`, `corepack`, `pnpm`, `yarn`, `bun`, `docker`, plus POSIX `sudo`/`apt`/`bubblewrap` and native Windows `powershell`/`execution_policy`/`longpaths`/`temp`/`pwsh`. Add `--preflight-tools node,bun,docker`, `CRABBOX_PREFLIGHT_TOOLS`, or repo `run.preflightTools` to replace the list. `default` expands built-ins; `none` prints only the workspace summary. Preflight is diagnostic only; install toolchains through Actions hydration, images, devcontainer/Nix/mise/asdf, or the run script. On `blacksmith-testbox`, this prints a delegated-unsupported note because the workflow owns setup. - `CRABBOX_ENV_ALLOW=NAME,...`: forwards only listed local env vars for direct providers and prints `set len=N secret=true` style summaries. On `blacksmith-testbox`, env forwarding is unsupported; put secrets in the Testbox workflow instead. - `--env-from-profile <file>` plus `--allow-env NAME`: loads simple `export NAME=value` / `NAME=value` lines from a local profile without executing it, then forwards only allowlisted names. `--allow-env` is repeatable and comma-separated. Profile values override ambient allowlisted env values for that run. Direct POSIX, WSL2, and native Windows runs are supported; delegated providers are not. Crabbox probes the uploaded profile remotely and prints redacted presence/length metadata before the command. - `--env-helper <name>`: with `--env-from-profile` on POSIX SSH targets, persists `.crabbox/env/<name>` and `.crabbox/env/<name>.env` so follow-up commands on the same lease can run through `./.crabbox/env/<name> <command>`. Use only on leases you control; the profile stays until cleanup, lease reset, or `--full-resync`. - `--script <file>` / `--script-stdin`: upload a local script into `.crabbox/scripts/` and execute it on the remote box. Shebang scripts execute directly on POSIX; scripts without a shebang run through `bash`. Native Windows uploads run through Windows PowerShell, and Crabbox appends `.ps1` when needed. Arguments after `--` become script args. - `--fresh-pr owner/repo#123|URL|number`: skip dirty local sync and create a fresh remote checkout of the GitHub PR. Bare numbers use the current repo's GitHub origin. Add `--apply-local-patch` only when the current local `git diff --binary HEAD` should be applied on top of that PR checkout. - `--full-resync` / `--fresh-sync`: reset a stale direct-provider workdir before syncing. Use after sync fingerprints look wrong, SSH times out before sync, or rsync watchdog output suggests it. It is redundant with `--fresh-pr`, incompatible with `--no-sync`, and unsupported by delegated providers. - `--capture-stdout <path>` / `--capture-stderr <path>`: write remote streams to local files and keep binary/noisy output out of retained logs. Parent directories must already exist. These are direct-provider only. - `--capture-on-fail`: on non-zero direct-provider exits, downloads `.crabbox/captures/*.tar.gz` with `test-results`, `playwright-report`, `coverage`, JUnit XML, and nearby logs. Treat as secret-bearing until reviewed. - `--keep-on-failure`: leave a failed one-shot lease alive for live debugging until idle/TTL expiry. Useful on direct providers and delegated one-shots. - `--timing-json`: final machine-readable timing. Add `echo CRABBOX_PHASE:install`, `CRABBOX_PHASE:test`, etc. in long shell commands; direct providers and Blacksmith Testbox both report them as `commandPhases`. Live-provider debug template for the resolved direct-provider lease: ```sh mkdir -p .crabbox/logs ../crabbox/bin/crabbox run \ --preflight \ --allow-env OPENAI_API_KEY,OPENAI_BASE_URL \ --timing-json \ --capture-stdout .crabbox/logs/live-provider.stdout.log \ --capture-stderr .crabbox/logs/live-provider.stderr.log \ --capture-on-fail \