Run any Skill in Manus with one click

cwcli-e2e-testing

Stars4

Forks0

UpdatedJuly 18, 2026 at 12:21

End-to-end testing protocol for cwcli against real Frappe/ERPNext Docker instances - the isolation recipe (CWCLI_HOME or a temp HOME, the cwe2e- project prefix, dedicated volumes/network), building genuine v14/v15/v16 benches instead of fixtures, driving interactive prompts through a real pty (the ESC[?2004h raw-mode marker), driving non-interactive paths with flags, teardown, and the tests/e2e/harness.py contract. Use this whenever you write, run, or extend E2E tests (anything under tests/e2e/), touch tests/e2e/harness.py, or hand-validate a cwcli behavior change on a real instance - even if the task just says "test it on a real instance" without naming E2E.

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Run Skill in Manus

Source

karotkriss

karotkriss/caffeinated-whale-cli

View GitHub Repository View Creator Repositories

Download

Run Skill in Manus

Related occupationsSOC

Based on SOC occupation classification

Software Quality Assurance Analysts and TestersComputer and Mathematical Occupations·SOC 15-1253

SKILL.md

readonly

name	cwcli-e2e-testing
description	End-to-end testing protocol for cwcli against real Frappe/ERPNext Docker instances - the isolation recipe (CWCLI_HOME or a temp HOME, the cwe2e- project prefix, dedicated volumes/network), building genuine v14/v15/v16 benches instead of fixtures, driving interactive prompts through a real pty (the ESC[?2004h raw-mode marker), driving non-interactive paths with flags, teardown, and the tests/e2e/harness.py contract. Use this whenever you write, run, or extend E2E tests (anything under tests/e2e/), touch tests/e2e/harness.py, or hand-validate a cwcli behavior change on a real instance - even if the task just says "test it on a real instance" without naming E2E.
metadata	{"internal":true}

cwcli end-to-end testing

This is the detailed protocol for validating cwcli against genuine throwaway Frappe instances. The standing captain-standards that motivate it (support+test BOTH modes; re-run E2E after no-mistakes/review fixes; full-lifecycle validation of dangerous-delete work) live in the always-loaded AGENTS.md "Captain standards" section - read those too when the change is a prompting or destructive-delete command.

There is now an automated, CI-run E2E harness in tests/e2e/ (the e2e / e2e_p2p / e2e_pkg marker tier) that codifies the manual recipe below; it drives the real cwcli binary against genuine throwaway Frappe instances, on a v14/v15/v16 matrix (.github/workflows/e2e.yml). It is being filled in per command (see the open change openspec/changes/rebuild-e2e-test-suite); the harness contract - the cwe2e- name prefix, the CWCLI_HOME isolation seam plus temp HOME, the hard rails (enforce_isolation, name prefix) and the unconditional sweep_cwe2e backstop, a root-owned-path reclaim (reclaim_root_owned, run before a session's temp HOME is deleted so a Docker-daemon-created root-owned path can never survive teardown into shared /tmp), the port allocator (CWE2E_PORT_BASE override, since the harness also runs on operators' own machines), the CWCLI_BIN override to aim the harness at any binary (a runtime-deps-only uv tool install . build for the e2e_pkg leg below), real-readiness waits, and the pexpect/ESC[?2004h helper - lives in tests/e2e/harness.py; read tests/README.md before extending it. The manual docs/e2e/ worked runs remain the human-precedent reference the harness was built from (canonical examples: docs/e2e/restore-inspect-e2e-r6.md, docs/e2e/bench-ux-m7-multi-bench.md, docs/e2e/restore-noninteractive-h2.md). When validating a behavior change by hand, follow the same procedure and point back to those runs.

Isolate everything. Set CWCLI_HOME (preferred - relocates only cwcli's own state via config_utils.cwcli_home(), leaving HOME and HOME-derived tooling like git/ssh untouched) or a temporary HOME (isolates cwcli plus everything else that reads HOME) to a fresh throwaway directory so ~/.cwcli is never touched, use unique docker-compose project names (for example cwe2e-<something>), and dedicated volumes/network. NEVER touch the captain's real projects or real ~/.cwcli.
Build genuine benches. Use cwcli init + bench init to create real benches, not fixtures. Build BOTH Frappe version-14 and version-15 when the behavior is version-sensitive; some bugs only reproduce on v14 (for example the receive-mode bare-filename Invalid path bug, which v15's alternative-directory fallback masks).
Drive interactive prompts through a real pty (pexpect). Await prompt_toolkit's raw-mode readiness marker ESC[?2004h before each keystroke so nothing races the prompt. For a confirm-then-prompt flow, press y THEN Enter, the way a human types it, so the confirm consumes its own trailing Enter (see the auto_enter=False credential note in the cwcli-lifecycle skill (references/restore.md)).
Drive non-interactive paths with flags. Use --yes/-y, --mariadb-root-username/--mariadb-root-password, --site, and the backup selectors (--latest/--backup-file). A non-TTY without the needed flag must refuse with a non-zero exit, not hang or silently proceed.
Tear down all throwaway instances afterward (containers, volumes, network, temp HOME), and confirm the captain's real instances are untouched.
Add a runtime-deps-only clean-install smoke leg whenever the change touches imports or dependencies (a new import, a new module wired into main.py's command chain, or any pyproject.toml dependency edit). This is IN ADDITION to the editable-install behavior legs above, not a replacement - keep validating real behavior against the worktree's own editable install (uv run cwcli), never mocks and never a PyPI build. The reason it is its own leg: uv run cwcli runs in the dev env (uv sync --all-extras), where every dev/transitive dep is present, so it CANNOT catch a runtime dependency that is imported but not declared in [project.dependencies]. "In the user's shoes" is uv tool install, not uv run. cwcli shipped SHIPPED-BROKEN on develop exactly here: commands/config.py imported click directly while only typer was declared; typer 0.27 stopped supplying click transitively, so a real uv tool install had no click and EVERY command died at import with ModuleNotFoundError. Unit tests, no-mistakes, CI, and the 74-check docs/e2e/config-dx-rework.md E2E all ran in the dev env and stayed green. The leg (no Docker needed - purely an import/parse smoke):
```
# Isolated tool dir so the install resolves ONLY from [project.dependencies]
export UV_TOOL_DIR=$(mktemp -d) UV_TOOL_BIN_DIR=$(mktemp -d)
uv tool install .                                   # the shipped shape: runtime deps only, no --all-extras
uv pip list --python "$UV_TOOL_DIR"/caffeinated-whale-cli | grep -i pytest \
  && { echo "dev deps leaked - not a runtime-only install"; exit 1; } || true
BIN="$UV_TOOL_BIN_DIR/cwcli"
"$BIN" --help >/dev/null                            # exercises main.py's full `from .commands import ...` chain
"$BIN" config --help >/dev/null                     # and the specific group you touched
CWCLI_HOME=$(mktemp -d) "$BIN" config path >/dev/null   # one trivial no-Docker command that fully loads the tree
```
Any undeclared runtime import surfaces here as a ModuleNotFoundError at import time. This is the per-feature discipline; .github/workflows/test.yml's Clean install smoke job enforces the same leg mechanically on every push/PR (see AGENTS.md CI-gates). That job only exercises --help/config reads; e2e.yml's e2e-runtime-only job goes deeper, driving a genuine init -> apps list -> inspect -> backup -> rm lifecycle against the same runtime-only binary via CWCLI_BIN (the e2e_pkg marker, tests/e2e/test_pkg_lifecycle_e2e.py), catching a dependency imported lazily inside a command body. See docs/contributing/ci-cd.md for the full job description.

More from this repository

same repository

cwcli-apps-update

karotkriss/caffeinated-whale-cli

Incident-level internals of cwcli's apps command group (list/install/uninstall/update/checkout) and the update state machine - multi-site fan-out by default, --json stdout purity, post-mutation recache, git-URL app-name derivation, the deprecated `cwcli update` alias and its frappe special-case, list/install/uninstall on the logic core (core/apps.py: the untangled uninstall consent/auto_start, report.ok-driven exit codes, `cwcli axi apps list` and `axi apps checkout` shipping while install/uninstall stay deferred, and how checkout's absence assertion was reversed), and core/update.py's control flow plus maintenance-mode safety (why it is NOT a generator, why a lost stream is unknown-not-failed, per-site enable/disable, honest partial-failure exit codes, shlex-quoted interpolation). Use this whenever you edit or debug commands/apps.py, core/apps.py, commands/update.py, core/update.py, or `axi apps list`/`axi apps update`, or touch app install/uninstall/update/checkout or bench migration behavior. Each note gua

2026-07-204

cwcli-core-axi

karotkriss/caffeinated-whale-cli

The UI-pure logic core boundary and the cwcli axi agent-facing frontend - core purity (core/*.py imports NO rich/questionary/typer, returns a typed Result envelope or raises CwcliError, never prompts or exits), no live Docker object crossing a core.<verb> return boundary, the thin CLI wrappers over core resolvers, the backup pre-resolve-then-spinner pattern, axi's TOON-only structured output and exit-code mapping, the migrated read-only ls/list + where slices, and the AXI cross-cutting shell (the `cwcli axi setup` SessionStart-hook installer and the generated installable skill). Use this whenever you edit or debug src/caffeinated_whale_cli/core/, commands/axi.py, utils/toon.py, commands/list.py, commands/where.py, utils/agent_hooks.py, scripts/build_skill.py, skills/cwcli/SKILL.md, or migrate another command onto the logic core. Each note guards a locked design decision or a real shipped bug - do not silently re-break it.

2026-07-204

cwcli

karotkriss/caffeinated-whale-cli

Manage local Frappe/ERPNext Docker instances through the cwcli CLI - list instances, start/stop/restart them, check per-process health, back up and unlock sites, manage bench apps, and search cached apps and sites. Use whenever a task touches a local Frappe or ERPNext development instance, bench, or site: checking whether an instance is running, starting or restarting one, backing up or restoring a site, installing or updating bench apps, or finding which bench holds an app or site.

2026-07-204

cwcli-lifecycle

karotkriss/caffeinated-whale-cli

Incident-level root-cause internals of cwcli's lifecycle and destructive commands - init (Frappe/bench version gating, admin/db-root secret env-transport, existing-bench reuse flow), rm (what deletion actually removes, conf/-only archive, recache-under-spinner deadlock, the C1/H5/M11 backup+name+exit-code data-safety gates, multi-bench per-bench backup, the start-back-up-delete transient-start flow for a stopped project), and restore (receive-mode and normal-path data-safety, non-interactive selectors, secrets off the argv, streamed copies, shared site detection, default-site resolution, post-restore migrate+restart), and start/status/logs/restart (the supervisord per-process supervision substrate: PID-keyed idempotency, config/launcher generation, per-process logs, the stdout-token-only status contract, single-program restart + auto-heal). Use this whenever you edit or debug core/init.py, commands/init.py, core/rm.py, commands/rm.py, core/restore.py, commands/restore.py, commands/start.py, commands/status.py

2026-07-184

cwcli-inspect-benches

karotkriss/caffeinated-whale-cli

Incident-level internals of cwcli's inspect / cache / multi-bench layer - inspect's 3-tier freshness model (T1 serve-cache / T2 read-only partial drift-check / T3 full re-cache), the multi-bench addressing model (numeric vs durable user labels, the --bench selector, marker-file recovery), the shared --yes / auto-start / honest-exit-code contract, the cache secret-redaction whitelist that keeps secrets out of the SQLite cache, and the CWCLI_HOME on-disk footprint override. Use this whenever you edit or debug core/inspect.py, commands/inspect.py, commands/label.py, utils/bench_labels.py, utils/db_utils.py, utils/config_utils.py, the SQLite cache, or anything touching multi-bench selection / labels. Each note guards a real shipped bug - keep the root-cause "why".

2026-07-184

cwcli-release

karotkriss/caffeinated-whale-cli

How to cut a cwcli (caffeinated-whale-cli) release - the exactly-four version-bump files (pyproject.toml, __init__.py, uv.lock, CHANGELOG.md), the CHANGELOG Keep-a-Changelog format, the semver convention this repo practices, the develop -> vX.Y.Z tag publish path, the uv trusted-publishing PyPI flow in release.yml (OIDC, --check-url non-reproducible-build gotcha, why the old pypa action was replaced), and the one-time PyPI trusted-publisher prerequisite. Use this whenever you bump the cwcli version, edit CHANGELOG.md for a release, publish to PyPI, or touch .github/workflows/release.yml or build.yml.

2026-07-164

name	cwcli-e2e-testing
description	End-to-end testing protocol for cwcli against real Frappe/ERPNext Docker instances - the isolation recipe (CWCLI_HOME or a temp HOME, the cwe2e- project prefix, dedicated volumes/network), building genuine v14/v15/v16 benches instead of fixtures, driving interactive prompts through a real pty (the ESC[?2004h raw-mode marker), driving non-interactive paths with flags, teardown, and the tests/e2e/harness.py contract. Use this whenever you write, run, or extend E2E tests (anything under tests/e2e/), touch tests/e2e/harness.py, or hand-validate a cwcli behavior change on a real instance - even if the task just says "test it on a real instance" without naming E2E.
metadata	{"internal":true}

cwcli end-to-end testing

Isolate everything. Set CWCLI_HOME (preferred - relocates only cwcli's own state via config_utils.cwcli_home(), leaving HOME and HOME-derived tooling like git/ssh untouched) or a temporary HOME (isolates cwcli plus everything else that reads HOME) to a fresh throwaway directory so ~/.cwcli is never touched, use unique docker-compose project names (for example cwe2e-<something>), and dedicated volumes/network. NEVER touch the captain's real projects or real ~/.cwcli.
Build genuine benches. Use cwcli init + bench init to create real benches, not fixtures. Build BOTH Frappe version-14 and version-15 when the behavior is version-sensitive; some bugs only reproduce on v14 (for example the receive-mode bare-filename Invalid path bug, which v15's alternative-directory fallback masks).
Drive interactive prompts through a real pty (pexpect). Await prompt_toolkit's raw-mode readiness marker ESC[?2004h before each keystroke so nothing races the prompt. For a confirm-then-prompt flow, press y THEN Enter, the way a human types it, so the confirm consumes its own trailing Enter (see the auto_enter=False credential note in the cwcli-lifecycle skill (references/restore.md)).
Drive non-interactive paths with flags. Use --yes/-y, --mariadb-root-username/--mariadb-root-password, --site, and the backup selectors (--latest/--backup-file). A non-TTY without the needed flag must refuse with a non-zero exit, not hang or silently proceed.
Tear down all throwaway instances afterward (containers, volumes, network, temp HOME), and confirm the captain's real instances are untouched.
Add a runtime-deps-only clean-install smoke leg whenever the change touches imports or dependencies (a new import, a new module wired into main.py's command chain, or any pyproject.toml dependency edit). This is IN ADDITION to the editable-install behavior legs above, not a replacement - keep validating real behavior against the worktree's own editable install (uv run cwcli), never mocks and never a PyPI build. The reason it is its own leg: uv run cwcli runs in the dev env (uv sync --all-extras), where every dev/transitive dep is present, so it CANNOT catch a runtime dependency that is imported but not declared in [project.dependencies]. "In the user's shoes" is uv tool install, not uv run. cwcli shipped SHIPPED-BROKEN on develop exactly here: commands/config.py imported click directly while only typer was declared; typer 0.27 stopped supplying click transitively, so a real uv tool install had no click and EVERY command died at import with ModuleNotFoundError. Unit tests, no-mistakes, CI, and the 74-check docs/e2e/config-dx-rework.md E2E all ran in the dev env and stayed green. The leg (no Docker needed - purely an import/parse smoke):
```
# Isolated tool dir so the install resolves ONLY from [project.dependencies]
export UV_TOOL_DIR=$(mktemp -d) UV_TOOL_BIN_DIR=$(mktemp -d)
uv tool install .                                   # the shipped shape: runtime deps only, no --all-extras
uv pip list --python "$UV_TOOL_DIR"/caffeinated-whale-cli | grep -i pytest \
  && { echo "dev deps leaked - not a runtime-only install"; exit 1; } || true
BIN="$UV_TOOL_BIN_DIR/cwcli"
"$BIN" --help >/dev/null                            # exercises main.py's full `from .commands import ...` chain
"$BIN" config --help >/dev/null                     # and the specific group you touched
CWCLI_HOME=$(mktemp -d) "$BIN" config path >/dev/null   # one trivial no-Docker command that fully loads the tree
```
Any undeclared runtime import surfaces here as a ModuleNotFoundError at import time. This is the per-feature discipline; .github/workflows/test.yml's Clean install smoke job enforces the same leg mechanically on every push/PR (see AGENTS.md CI-gates). That job only exercises --help/config reads; e2e.yml's e2e-runtime-only job goes deeper, driving a genuine init -> apps list -> inspect -> backup -> rm lifecycle against the same runtime-only binary via CWCLI_BIN (the e2e_pkg marker, tests/e2e/test_pkg_lifecycle_e2e.py), catching a dependency imported lazily inside a command body. See docs/contributing/ci-cd.md for the full job description.