| name | validator-snapshot-recovery |
| description | Use when recovering validator or full nodes from snapshots with integrity, state, rollback, and sync verification. |
Validator snapshot recovery
Use when restoring a validator, sentry, or full node from a state snapshot. Do not use for publishing snapshots.
This public skill contains no provider list or infrastructure inventory. Resolve the authorized target_ref, network_ref, role, data directory, database backend, service, chain ID, and approved snapshot sources from private knowledge and live state.
1. Assess before stopping
- Confirm whether the workload signs, serves as a sentry, or is non-signing.
- Read the current network page and canonical recovery guide.
- Verify the current service, chain ID, version, database backend, data path ownership, sync state, and disk usage.
- Identify the failure mode and whether snapshot recovery is actually required.
- Calculate capacity for the archive, extracted data, retained rollback data, temporary files, and safety margin.
- Download and validate while the current node stays online when possible.
Do not delete or replace data merely because sync is slow. A destructive replacement requires clear operator intent after preflight.
2. Protect keys and signer state
Always back up key-bearing configuration before deleting node data. Verify the backup exists, is non-empty, access-restricted, and stored outside the deletion target without printing contents.
Treat consensus keys, remote-signer configuration, and anti-double-sign state as separate safety boundaries:
- never open, echo, diff, or copy key contents into chat or logs;
- never blindly reset validator signing state;
- determine the chain-specific relationship between snapshot height and signer state;
- ensure no second process or host can sign with the same key;
- use a non-signing recovery path when signer-state safety cannot be proven.
Database backups may be large; obtain operator approval unless the canonical procedure already makes one necessary.
3. Validate the snapshot
Establish provenance and check:
- correct chain ID and environment;
- snapshot height, age, block time, and trust source;
- compatibility with node version and database backend;
- archive type, compression, layout, and expected contents;
- checksum/signature when published;
- archive integrity with the native test command;
- free space and inode headroom;
- absence of path traversal, unsafe absolute paths, device nodes, or unexpected ownership.
Compare height and freshness against multiple independent healthy RPCs when available. A downloadable archive is not sufficient evidence.
4. Stage and inspect
Extract into a separate staging path when space permits. Inspect whether the archive is flat or nested and locate the actual database/state root. Do not extract over the live data directory.
Preserve configuration, keys, address book decisions, and chain-specific state exactly as the canonical guide requires. Keep the old data as a rollback target until recovery is verified, unless the operator explicitly chose deletion because capacity makes rollback impossible.
5. Controlled cutover
- Stop and verify the node process/container is actually stopped.
- Prevent automation from restarting it during the swap.
- Reconfirm key backup and signer uniqueness.
- Move current data to the rollback location or delete only the approved paths.
- Move staged data atomically when possible.
- Restore required permissions and chain-specific configuration/state.
- Start the expected workload.
Do not use broad glob deletion, follow symlinks outside the target, or remove configuration/key directories as a routine data reset.
6. Verify recovery
Completion requires:
- service/container active with stable restarts;
- no corruption, panic, schema, or wrong-network errors;
- expected chain ID and snapshot-derived starting height;
- height advancing toward external truth;
- acceptable peer and sync status;
- correct database backend and data ownership;
- for validators: active/not jailed state and recent signatures verified externally;
- monitoring and downstream consumers healthy.
If startup fails, preserve logs and use the defined rollback path rather than repeatedly mutating signer state. Report source/provenance, snapshot height, integrity checks, cutover method, rollback availability, and remaining catch-up.
Update private network knowledge and daily memory, but keep provider choices and environment facts out of this public skill.
