Run an integration test against real Docker (and, for the fixtures that own real AWS resources, a real CloudFormation stack deployed via the upstream cdk CLI). Which fixtures those are is DERIVED by tests/integration/_lib/aws-orphan-sweep.sh, never globbed from the name.
cdk-local is a local-execution CLI — it does NOT deploy resources itself. The only AWS-side activity in any integ test is when a fixture's verify.sh invokes the upstream cdk deploy (or, for local-invoke-agentcore-froms3, aws s3api create-bucket) to create a target stack for --from-cfn-stack to point at. Cleanup is always done by the fixture's own verify.sh.
-
Build first: Run vp run build to ensure dist/cli.js is up to date. The fixture's verify.sh resolves the binary via node ../../../dist/cli.js, so source changes without a build have no runtime effect.
-
Resolve the fixture path: tests/integration/<test-name>/. Confirm verify.sh exists; if not, the test does not have a Docker-driven flow yet and this skill exits with a clear error pointing the user at the missing script.
-
Pre-flight Docker sweep: docker ps --filter name=cdkl- -q | wc -l and docker network ls --filter name=cdkl-task- -q | wc -l should both return 0. If either is non-zero, abort and ask the user to run /cleanup first — running on top of orphans causes name collisions and confusing failures.
-
AWS pre-flight sweep — run it for EVERY fixture, unconditionally:
bash tests/integration/_lib/aws-orphan-sweep.sh <test-name>; rc=$?
Do NOT first decide whether the fixture is AWS-resource-owning. That
decision has silently excluded a resource-owning fixture twice
(*-from-cfn-stack missed three, and the widened *-from-cfn* still missed
local-invoke-assume-role, which deploys a stack containing an IAM role),
and the script now makes it internally, from one predicate, against the
fixture's own verify.sh. A fixture that owns nothing makes no AWS call and
exits 0. aws-orphan-sweep.sh --list-owners prints the derived set if you
want to see it.
Also confirm the deploy toolchain is present — the sweep checks that
AWS is reachable, not that the fixture can deploy, and this was dropped
when the recipe became a script:
which cdk
Only needed when the sweep reported the fixture as AWS-owning (rc 0 with a
fixture= line rather than the "owns no real AWS resource" line).
Gate on the exit code. Do not read the output and judge. That is the
whole point of go-to-k/cdk-local#601 — the previous recipe was prose whose
error branches nobody executed, and it carried four instances of one defect
class in a single PR (a bare stack name that could never match, a /cdkl
scope that listed every lane, a filter that degraded to contains(Name,'-')
over the whole ACCOUNT, and two stack scans with no guard at all, where a
wrong cwd produced a false clean on the PRIMARY resource):
| rc | meaning | what to do |
|---|
| 0 | clean | proceed to step 5 |
| 1 | usage / internal error | STOP. Nothing was concluded. Read the FATAL line. |
| 2 | orphan found | STOP. Remediate — see below. |
| 3 | indeterminate | STOP. A query could not be performed (no credentials, aws missing, an unrecognized error). This is NOT clean. |
| 4 | report-only | STOP and check by hand. An UNATTRIBUTABLE resource matched — see below. |
-
Run the test: bash tests/integration/<test-name>/verify.sh. Propagate the script's exit code — a non-zero exit must drive this skill into the failure path so step 7's cleanup verification fires. Do NOT swallow verify.sh failures.
-
Verify Docker cleanup (mandatory regardless of pass/fail):
docker ps --filter name=cdkl- -q | wc -l
docker network ls --filter name=cdkl-task- -q | wc -l
docker network ls --filter name=cdkl-svc- -q | wc -l
If any are non-zero, dispatch /cleanup (no --detect-only) and re-run the checks. Never end the run with orphan Docker resources still present.
-
Verify AWS cleanup — the SAME command as step 4, with the same
exit-code gate:
bash tests/integration/_lib/aws-orphan-sweep.sh <test-name>; rc=$?
This run matters more than step 4's: it comes after a long test where a
session token can expire (which is exactly the rc=3 indeterminate case,
NOT a clean one), and step 9 turns its verdict into a fresh integ marker.
Same table, same remediation, same live-peer check. Re-run the sweep after
any deletion and require rc=0 before continuing.
-
Report results: Show pass/fail for the test, plus a one-line cleanup summary — docker: 0 orphans, network: 0 orphans plus AWS sweep: rc=0 (clean) quoting the sweep's OWN verdict line and exit code. Quote what the script printed rather than paraphrasing it: a paraphrase is writable without having run anything, which is how a recipe reports clean while not having looked.
-
Set the integ markgate marker (only on full clean success):
When — and ONLY when — all of the following hold:
- the
verify.sh step finished with exit code 0,
- step 6 reports 0 docker orphans,
- step 7's
aws-orphan-sweep.sh exited 0 (any of 1 / 2 / 3 / 4 is a
stop, and rc=3 in particular means the sweep could not look — not that
nothing is there),
record the gate so subsequent gh pr merge calls are unblocked:
mise exec -- markgate set integ || echo "MARKER NOT RECORDED (rc=$?) — read the error above"
Check the exit code; do not assume the set succeeded. Under hash: files this command could not fail, so it was safe to fire and forget. Under hash: diff it CAN fail, and it reports the reason on stderr — an unchecked call looks silent and successful while nothing was recorded. The failure modes and their fixes:
no delta against merge-base(origin/main, HEAD) — you are on the base branch. Re-run from the PR's own worktree, on the PR branch.base ref "origin/main" does not resolve — run git fetch origin, then set again. Re-running the whole integ does NOT help; the set fails identically until the ref exists.hash=diff recorded an empty in-scope delta — this is a WARNING, not a failure: the marker WAS saved and the exit code is 0. It only means the branch changes nothing under src/** / tests/integration/**.
Confirm with mise exec -- markgate status integ (expect state: match) before reporting the run as complete. The expensive failure this prevents: a full Docker fixture run finishes, the marker is silently not recorded, the merge stays blocked, and the natural reaction is to run the integ AGAIN rather than fetch.
If any of the above failed, do NOT set the marker — that is the whole point of the gate. The integ gate (see .markgate.yml) blocks gh pr merge for any PR that touches src/** or tests/integration/** until this marker is fresh. Set the marker from the PR's own worktree, on the PR branch: the gate uses markgate's mode, whose digest is that branch's delta against .
