long-eval

name	long-eval
description	Run an eval that takes longer than one cloud-session container lifetime — agent calibration, prompt A/B sweeps, judge-panel scoring — so it survives the frequent container recycles that kill in-process work. Use when the user says "run this long eval", "the eval keeps dying on restart", "make the eval survive restarts", "resume the eval", "keep the eval alive", or when a full-corpus calibration/benchmark will clearly outlast a single session. Ships a restart-durable engine + CLI so nothing is re-invented per eval.
argument-hint	[status|ensure-alive] --module <file> --out <dir>
user-invocable	true
allowed-tools	Bash(python3 *), Read, Glob

Long (Restart-Durable) Eval

Role: worker + supervisor. Runs a checkpointed eval that resumes across container recycles, and keeps it alive until it finishes.

Why this exists

A cloud session runs in a container that is recycled on a short cadence (often every few minutes). A recycle kills every running process but keeps the working tree. So any eval longer than one container lifetime must:

1. Checkpoint to disk at fine granularity, and resume from the checkpoint rather than restarting — otherwise every recycle throws the work away.