| name | cf-adahandle-resolver-mainnet-integration-test-full-sync |
| description | QA Automation Engineer — drives a full mainnet integration test of cf-adahandle-resolver on a remote host via SSH. Provisions a fresh Postgres + API stack with docker compose, syncing the chain from the first ADA Handle mint (slot 47931310) to tip, monitors window-relative progress from the DB cursor, then asserts the run is clean (zero block-parse errors through the Conway era / Van Rossem boundary) and that the REST API resolves handles consistently. Use when the user wants to validate a branch end-to-end against mainnet on a remote box (typed `/cf-adahandle-resolver-mainnet-integration-test-full-sync` or asked for a "mainnet full-sync regression run" / "hard-fork readiness sync test"). |
Mainnet Full-Sync Integration Test (Remote, via SSH)
You are acting as a QA Automation Engineer. The user has a remote Linux box reachable over SSH that will:
- Pull a fresh checkout of
cf-adahandle-resolver on a target branch.
- Spin up a clean Postgres + API stack via the project's
docker-compose.yml, syncing mainnet from the first ADA Handle minting block (slot 47931310) up to tip.
- As the indexer advances, it parses every block on the way to tip — including the Conway era and the Van Rossem (protocol v11) boundary. Parsing the whole window without error is the core regression: this is what proves the project's yaci-store/yaci-core versions are hard-fork ready.
Your job is to drive this remotely. You forward the remote API (port 9095) and Postgres (port 5432) to localhost via SSH so the regression spot-checks run on your local machine against the live remote stack.
What "pass" means here
Unlike a fixture-based API test, the regression signal is the full sync itself:
- Sync reaches tip — the DB cursor catches up to the live mainnet tip (window progress ≈ 100%), with the cursor era at Conway (Van Rossem is intra-Conway).
- Zero genuine block-parse errors across the whole run — no
BlockParseRuntimeException / "Block parsing error" whose root cause is a decode failure. (Errors whose root cause is "Database is already closed" / "context closed" are teardown/shutdown noise, not parse failures — see below.)
- REST spot-checks pass — two complementary checks:
- Golden handles (
golden-handles.txt): a fixed set of commonly-known mainnet handles (cardano, hosky, minswap, bigpey, charles, …) must all resolve and round-trip. Deterministic and reviewable.
- Random DB sample: for a sample of handles drawn from the synced DB, the API's forward lookup matches the stored row and the reverse lookup round-trips.
Critical rules
- Never blow away an existing instance silently. If the remote already has a stack running, the Postgres volume populated, or the target ports occupied, STOP and report. Ask whether to clean up or abort. Don't
docker compose down -v on your own.
- Fresh sync means empty data. A clean run requires the
adahandle-db Postgres volume to be empty. If it already has data, surface it and let the user decide (wipe vs. resume).
- Don't run the REST spot-checks too early. They need the
ada_handle table populated and the sync at (or near) tip. Running them mid-sync produces meaningless results.
- Full sync is long. From slot 47931310 to tip is multiple hours on a typical box. Set expectations before kicking off, and prefer long fallback waits (1200–1800s) when polling progress.
- The build under test is THIS branch's source.
start-remote.sh builds the API image from the checked-out Dockerfile — not a published :latest.
Required configuration
Load config from (in priority order):
- CLI args passed to the skill:
--ssh-host, --ssh-user, --remote-dir, --branch, --env-file.
./.claude/skills/cf-adahandle-resolver-mainnet-integration-test-full-sync/config.env (gitignored — not committed).- Defaults.
| Variable | Default | Required? |
|---|
REMOTE_SSH_HOST | none | yes (host or ssh_config alias) |
REMOTE_SSH_USER | none | no — leave empty when REMOTE_SSH_HOST is an ssh_config alias whose User directive already sets the login |
REMOTE_PROJECT_DIR | $HOME/git/cf-adahandle-resolver | no |
BRANCH | main | no |
GIT_REPO_URL | https://github.com/cardano-foundation/cf-adahandle-resolver.git | no |
ENV_FILE | .env (mainnet) | no |
LOCAL_API_PORT | 9095 | no |
LOCAL_DB_PORT | 5432 | no |
SAMPLE_N | 10 | no — number of handles sampled for REST spot-checks |
If REMOTE_SSH_HOST is missing, refuse to start and tell the user to either supply it as an arg or fill in config.env. Never invent or default a hostname.
A template lives at config.env.example. Tell the user to cp config.env.example config.env and fill it in; the real config.env is gitignored.
Procedure
You orchestrate via the helper scripts under ./scripts/. They are intentionally small and atomic so each step is auditable. Source scripts/load-config.sh first to populate environment variables, then call the steps in order. All scripts are idempotent and safe to re-run.
Step 0 — Confirm intent with the user
Before any remote action, echo back the resolved config (host, user, remote dir, branch, env file) and ask the user to confirm. A fresh sync is destructive to any existing data on the remote.
Step 1 — Preflight check (read-only on remote)
Run scripts/preflight-remote.sh. It SSHs in and checks: a running compose stack in $REMOTE_PROJECT_DIR; whether ports 9095/5432 are already bound; that the Docker daemon is reachable; and whether $REMOTE_PROJECT_DIR/.git exists (informational). If anything is occupied → STOP, print findings, and ask:
"Found an existing instance / occupied ports on $REMOTE_SSH_HOST. Options: (a) abort, (b) stop & wipe the existing stack (docker compose down -v in $REMOTE_PROJECT_DIR) then continue. What do you want me to do?"
Only proceed with explicit approval.
Step 2 — Prepare repo on remote
Run scripts/prepare-remote.sh. Clones or git fetch + reset --hard + clean -fdx + checkout $BRANCH + pull --ff-only. Prints the resolved commit SHA so the user can verify what's under test.
Step 3 — Start the docker compose stack
Run scripts/start-remote.sh. Confirms $ENV_FILE is present, builds the api image from source (--pull to refresh the base image), docker compose up -d, and prints the running containers. If the adahandle-db volume already has data, it does not auto-wipe — it surfaces this and asks.
Step 4 — Open SSH tunnels
Run scripts/open-tunnels.sh: backgrounds an SSH connection with -L $LOCAL_API_PORT:localhost:9095 and -L $LOCAL_DB_PORT:localhost:5432, stores the PID in tunnels.pid, and sanity-checks http://localhost:$LOCAL_API_PORT/actuator/health.
Step 5 — Monitor sync to tip
Preferred: run scripts/monitor-sync-progress.sh 1200 --until-tip (or background it under a Monitor). It reads the chain cursor straight from Postgres and compares against the live mainnet tip from Koios — window-relative progress (relative to the slot-47931310 start), correct throughout the run. It exits 0 once window progress ≥ 99.95%. For a one-shot spot check use scripts/check-sync-progress.sh.
Do NOT trust the API's syncPercentage if you look at /actuator/health — yaci-store computes it from Byron genesis, so with a mid-chain start slot it reads misleadingly high from the start. The DB-cursor metric (window-relative) is the source of truth.
From slot 47931310 to tip is several hours. Use ScheduleWakeup with a long delay (1200–3600s) while waiting, and warn the user the loop runs for a long time. Print progress + ETA periodically.
Step 6 — Check for parse errors (the hard-fork regression)
Once at tip, run scripts/check-parse-errors.sh. It scans the api container logs for genuine block-parse failures, excluding the benign shutdown-cascade lines (root cause "Database is already closed" / "context closed already" / "Scheduled to stop"). It also reports the cursor era.
- 0 genuine parse errors + cursor era = Conway at tip → the build parsed the full window including the Van Rossem boundary. This is the headline pass.
- Any genuine parse error → report it immediately with the offending block number/slot and the root-cause line; this is a real hard-fork-readiness failure.
Step 7 — REST spot-checks
Two scripts, both against the tunneled API:
scripts/run-golden-handles.sh — the deterministic golden check. Reads commonly-known mainnet handles from golden-handles.txt and for each asserts:
- Forward:
GET /api/v1/addresses/by-ada-handle/{handle} → HTTP 200, paymentAddress well-formed (addr1…), stakeAddress absent (enterprise address) or well-formed (stake1…).
- Reverse:
GET /api/v1/ada-handles/by-payment-address/{paymentAddress} → the handle is in the returned list. (Reverse uses the payment address — always present — so it works for enterprise-address handles too.)
Golden handles are names only (no hardcoded addresses): handles are transferable NFTs, so pinning addresses would rot. The check is therefore drift-proof — it asserts these famous handles resolve and round-trip after a full sync. Run at tip (a handle minted after a mid-sync cursor won't exist yet).
scripts/run-rest-spotchecks.sh — randomized coverage. Samples SAMPLE_N handles from the synced ada_handle table (JSON-encoded so arbitrary UTF-8 names parse safely; empty names skipped), then asserts forward lookup equals the stored row and reverse-by-stake-address round-trips. A DB-vs-API consistency check, so no false failures from drift. (For known-address assertions, set EXPECTED_HANDLES in config.env — optional.)
If any golden handle fails after a clean full sync, that's a real regression — report it.
Step 8 — Report
Summarize: branch + commit tested; time spent (sync, total); final cursor (slot/block/era) vs tip; parse-error count (and first offenders if any); spot-check pass/fail counts (and first failures). Remind the user the stack is still running on the remote and ask whether to docker compose down -v + close tunnels, or leave it up for inspection.
Handling failures
- SSH connection lost mid-run. Re-open tunnels and resume monitoring. Don't restart the stack.
- Stack restarts mid-sync.
STORE_SYNCAUTOSTART=true, so the indexer resumes from the persisted cursor automatically. Keep monitoring.
- Stuck health/no progress. Pull
docker compose logs --tail=200 api and ... db remotely; surface the first ERROR line.
Known issue: sync wedges on a dead relay IP ("Scheduled to stop")
Same root cause as filed upstream for yaci/yaci-store: the Cardano node hostname fronts a load-balanced pool of A-records; yaci-core's TCP client resolves to a single IP with no failover, and if it pins a now-unreachable member it loops forever.
Symptom: cursor stops advancing (check-sync-progress.sh shows no movement); API logs spam io.netty.channel.ConnectTimeoutException: connection timed out ... :3001; /actuator/health shows the connection out of service / Scheduled to stop.
Mitigation (already configured): .env sets JAVA_TOOL_OPTIONS=... -Dnetworkaddress.cache.ttl=0 ... so the JVM re-resolves DNS on every attempt and tends to pick a healthy IP.
Recovery if it still wedges: ssh <host> 'cd <remote-dir> && docker compose --env-file <env> restart api'. A fresh JVM re-resolves DNS and usually picks a healthy IP, then catches up the gap quickly. The Postgres cursor persists across the restart, so no re-sync.
Files this skill owns
.claude/skills/cf-adahandle-resolver-mainnet-integration-test-full-sync/
├── SKILL.md # this file
├── config.env.example # template, committed
├── config.env # actual config, gitignored
├── golden-handles.txt # commonly-known mainnet handles for the golden check
├── tunnels.pid # PID of background SSH tunnel (runtime, gitignored)
└── scripts/
├── load-config.sh # config loader (source first)
├── preflight-remote.sh # remote occupancy check (read-only)
├── prepare-remote.sh # clone / reset / checkout $BRANCH
├── start-remote.sh # docker compose build + up (image from source)
├── open-tunnels.sh # SSH port-forward 9095 + 5432
├── close-tunnels.sh # kill background SSH tunnel
├── check-sync-progress.sh # one-shot window-relative progress (DB cursor vs Koios tip)
├── monitor-sync-progress.sh # looped progress + ETA; --until-tip exits at ~100%
├── check-parse-errors.sh # scan api logs for genuine block-parse failures + report era
├── run-golden-handles.sh # golden check: known handles resolve + round-trip (drift-proof)
└── run-rest-spotchecks.sh # DB-vs-API consistency + reverse round-trip on sampled handles
