checkpoint-resume-long-job

name	checkpoint-resume-long-job
description	Persist progress for long-running jobs (batched LLM calls, large ingestions, multi-hour syncs) so that a context reset, crash, or interrupt doesn't lose work. Use whenever a job iterates over N items and completing item K matters independently. Provides a resumable.mjs library pattern plus the skill's invocation heuristics.
format	"2025-10-02T00:00:00.000Z"
version	1.0.0
status	stable
updated	"2026-04-17T00:00:00.000Z"
type	skill
category	gsd-meta
origin	tibsfox
modified	false
first_seen	"2026-04-17T00:00:00.000Z"
first_path	examples/skills/gsd-meta/checkpoint-resume-long-job/SKILL.md
superseded_by	null

Checkpoint & Resume for Long Jobs

Any job that takes longer than 5 minutes and iterates over N independent items should checkpoint its progress. Context can reset, processes can crash, users can Ctrl-C. A re-run shouldn't redo completed work.

Triggers

Activate when a job:

• Iterates over ≥ 20 items AND each item takes ≥ 5 seconds, OR
• Is expected to run ≥ 10 minutes total, OR
• Calls external APIs with rate limits or cost per call (LLM, HTTP), OR
• Is not naturally idempotent at the whole-job level

Shape

The simplest checkpoint is a file listing completed item IDs. On job start: read the file; on each item completion: append its ID; on job restart: skip any ID in the file.

Reference library: tools/checkpoint-resume/resumable.mjs

import { processBatches } from './tools/checkpoint-resume/resumable.mjs';

await processBatches({
  items: [...1713 lessons...],
  keyFn: l => l.id,
  checkpointFile: '.planning/sessions/tiebreaker-checkpoint.jsonl',
  batchSize: 5,
  async handler(batch) {
    // your per-batch work
    return batch.map(l => ({ id: l.id, status: 'done' }));
  },
  onProgress({ completed, total, skipped }) {
    console.error(`${completed + skipped}/ ( resumed)`);
  },
});