| name | durable-execution |
| description | Conceptual introduction to durable execution — what it is, why you'd want it, and the tradeoffs between rolling your own (just your DB) versus using a framework like Resonate. Read this BEFORE picking an implementation. For the Resonate SDK's concrete Context API (ctx.run, ctx.sleep, ctx.promise, structured concurrency), use the per-SDK skill for your language — resonate-basic-durable-world-usage-{typescript,python,rust,go}. |
| license | Apache-2.0 |
Durable Execution
Your code will crash. Durable execution means: when it restarts, it picks up where it left off — not from the beginning.
Traditional: start → step1 → step2 → crash → lost
Queue-based: start → step1 → step2 → crash → retry from top → duplicates
Durable: start → step1 → step2 → crash → replay to step2 → continue from step3
Everything else — frameworks, protocols, infrastructure — is implementation detail.
Do You Need This?
Score 1–5 on each dimension:
| Dimension | 1 (low) | 5 (high) |
|---|
| Failure cost | Retry is inexpensive, no side effects | Retry causes duplicates, data loss, or revenue loss |
| Duration | Milliseconds, single request | Hours/days/weeks, spans process lifetimes |
| Coordination | Single service, single step | Multiple services, human gates, external callbacks |
| State complexity | Stateless or simple key-value | Branching workflows, conditional logic, fan-out/fan-in |
| Total | Recommendation |
|---|
| 4–8 | Use a task queue with idempotent handlers |
| 9–12 | Durabilize the critical path only |
| 13–20 | Durable execution is the right primitive |
Quick check — you need this if any are true:
- A crash mid-workflow means a customer gets charged but never receives the product
- Your workflow spans multiple HTTP requests or process lifetimes
- You need human approval gates that pause for hours or days
- Duplicate execution of a step causes real-world harm (double charges, duplicate emails)
- You have multi-step processes across multiple services that must all succeed or all roll back
Choose Your Path
Does your workflow need...
├─ Just crash recovery + idempotency?
│ └─ BAKED IN: add checkpointing to your existing DB. Zero infra cost.
│ See: references/BAKED-IN.md
│
├─ Fan-out parallelism, durable sleep, human-in-the-loop,
│ or cross-service coordination?
│ └─ RESONATE: single-binary server + tiny SDK. ~$5/mo on a VPS.
│ See: references/RESONATE-QUICKSTART.md
│
└─ Enterprise-scale with managed infrastructure?
└─ Temporal Cloud ($520/mo+ at scale) or AWS Step Functions.
But consider: do you actually need that complexity?
| Baked In | Resonate | Temporal |
|---|
| Infra cost | $0 (your existing DB) | ~$5/mo (VPS) to ~$170/mo (1M tasks/day) | ~$520/mo (1M tasks/day) |
| Dependencies | None | Single binary + SQLite | Cluster + multiple services |
| Serverless | Yes (any runtime) | Yes (Lambda, Edge Functions) | No (requires always-on cluster) |
| Setup time | Minutes | 5 minutes | Hours to days |
| Patterns | Checkpoint, idempotency, outbox | All 5 patterns + distributed coordination | All patterns + enterprise features |
| Learning curve | Low (just SQL + your code) | Low (sequential code — generators in TS/Py, async/await in Rust/Go) | High (proprietary DSL + concepts) |
| When it fits | Single-service, sequential workflows | Multi-service, any complexity | Large teams with dedicated infra staff |
Baked In — Framework-Free Durability
Three building blocks. No framework. Just your database.
1. Idempotency Keys
Every operation gets a deterministic ID. Before executing, check if it already ran.
async function runOnce<T>(db: Database, key: string, fn: () => Promise<T>): Promise<T> {
const existing = db.query("SELECT result FROM completed_steps WHERE key = ?").get(key);
if (existing) return JSON.parse(existing.result);
const result = await fn();
db.run("INSERT INTO completed_steps (key, result) VALUES (?, ?)", [key, JSON.stringify(result)]);
return result;
}
2. Step-Level Checkpointing
Wrap each step. On crash and restart, completed steps return cached results. Execution resumes from the first incomplete step.
async function durableCheckout(db: Database, orderId: string) {
const inventory = await runOnce(db, `${orderId}:reserve`, () => reserveInventory(orderId));
const payment = await runOnce(db, `${orderId}:charge`, () => chargeCard(orderId, inventory));
const shipment = await runOnce(db, `${orderId}:ship`, () => createShipment(orderId, payment));
const email = await runOnce(db, `${orderId}:notify`, () => sendConfirmation(orderId, shipment));
return { inventory, payment, shipment, email };
}
3. Outbox Pattern
Side effects (emails, webhooks, API calls) go to a table first. A separate process delivers them exactly once.
db.run("INSERT OR IGNORE INTO outbox (id, payload) VALUES (?, ?)",
[`${orderId}:confirmation-email`, JSON.stringify({ to: email, subject: "Order confirmed" })]);
const pending = db.query("SELECT * FROM outbox WHERE delivered_at IS NULL").all();
for (const msg of pending) {
await deliver(msg);
db.run("UPDATE outbox SET delivered_at = ? WHERE id = ?", [Date.now(), msg.id]);
}
When baked-in hits its limits:
- You need a workflow to sleep for days without a process staying alive → need durable timers
- You need to suspend and wait for a human to approve something → need external promise resolution
- You need to fan out work across multiple services in parallel → need distributed coordination
- You want the framework to handle retries, timeouts, and replay for you → use Resonate
Full implementation with templates: See references/BAKED-IN.md and assets/baked-in-checkpoint.ts.
With Resonate — Durable Execution Platform
Resonate's open-source server is a single binary (Rust + SQLite, zero external deps) paired with a tiny SDK. Runs anywhere — VPS, serverless, edge functions. Costs ~$5/mo on a small VPS.
Your code is an ordinary function with durable steps — a generator in TypeScript/Python, an async fn/func in Rust/Go. Each durable step (yield*/yield, .await, Future.Await) is a checkpoint. If the process crashes, the server re-dispatches the work to any available worker, which replays from the last checkpoint.
Language note. The examples below (and in this skill's references) are shown in TypeScript. The concepts are identical across all four Resonate SDKs; only the syntax differs. For concrete, idiomatic syntax in your language, see the per-SDK skills — resonate-basic-durable-world-usage-{typescript,python,rust,go} for the Context API, and the matching resonate-saga-pattern-* / resonate-recursive-fan-out-pattern-* / resonate-human-in-the-loop-pattern-* skills (and resonate-durable-sleep-scheduled-work-{typescript,rust,go}) for the patterns shown here.
import { Resonate, type Context } from "@resonatehq/sdk";
const resonate = new Resonate({ url: "http://localhost:8001" });
resonate.register("processOrder", function* (ctx: Context, orderId: string) {
const order = yield* ctx.run(fetchOrder, orderId);
const payment = yield* ctx.run(chargeCard, order);
const shipment = yield* ctx.run(createShipment, order);
yield* ctx.run(sendConfirmation, order.email);
return { payment, shipment };
});
await resonate.start();
await resonate.invoke("processOrder", ["order-42"], { id: "order-42" });
That's it. Each yield* ctx.run(...) is checkpointed. Crash recovery, retries, and replay are automatic.
Get running in 5 minutes: See references/RESONATE-QUICKSTART.md.
All patterns with full code: See references/RESONATE-PATTERNS.md.
SDK API reference (TypeScript): See references/RESONATE-SDK.md (for Python/Rust/Go, see resonate-basic-durable-world-usage-{python,rust,go}).
Starter template (TypeScript): Copy assets/resonate-worker.ts (the assets/ templates are TypeScript; for other languages start from the per-SDK skill).
The 5 Patterns
1. Saga — Multi-step with compensating rollbacks
Each step is checkpointed. On failure, compensate in reverse order.
function* orderSaga(ctx: Context, orderId: string) {
const completed: string[] = [];
try {
yield* ctx.run(reserveInventory, orderId); completed.push("inventory");
yield* ctx.run(chargePayment, orderId); completed.push("payment");
yield* ctx.run(createShipment, orderId); completed.push("shipment");
return { status: "success" };
} catch (error) {
for (const step of completed.reverse()) {
yield* ctx.run(compensate, step, orderId);
}
return { status: "rolled-back", compensated: completed };
}
}
Use when: Multiple services must all succeed or all roll back (payments, bookings, provisioning).
2. Fan-Out / Fan-In — Parallel work with aggregation
Dispatch work to parallel workers via RPC. Each branch is independently durable.
function* batchProcess(ctx: Context, items: string[]) {
const results: string[] = [];
for (const item of items) {
const result = yield* ctx.rpc("processItem", [item], {
target: "poll://any@item-workers"
});
results.push(result);
}
return results;
}
Use when: Batch processing, parallel API calls, map-reduce workloads.
3. Human-in-the-Loop — Suspend for external signal
Workflow suspends without holding resources. Resumes when a human (or webhook) resolves the promise.
function* approvalFlow(ctx: Context, orderId: string) {
const approvalId = `approval-${orderId}`;
yield* ctx.run(sendApprovalEmail, orderId, approvalId);
const decision = yield* ctx.promise(approvalId, {
timeoutAt: Date.now() + 48 * 60 * 60 * 1000
});
if (decision === "approved") yield* ctx.run(processOrder, orderId);
return decision;
}
Use when: Approval workflows, manual review gates, external callbacks, payment confirmation.
4. Scheduled / Cron — Durable timers
Sleep is durable. Process can die and restart — the timer still fires.
function* onboarding(ctx: Context, userId: string) {
yield* ctx.run(sendEmail, userId, "Welcome!");
yield* ctx.sleep(24 * 60 * 60 * 1000);
yield* ctx.run(sendEmail, userId, "Getting started tips");
yield* ctx.sleep(6 * 24 * 60 * 60 * 1000);
yield* ctx.run(sendEmail, userId, "How are we doing?");
}
Use when: Drip campaigns, SLA reminders, retry delays, polling loops, recurring jobs.
5. Entity — Long-lived mutable state
Each method call is a durable step on a persistent entity.
function* orderLifecycle(ctx: Context, orderId: string) {
const order = yield* ctx.run(createOrder, orderId);
yield* ctx.run(validateOrder, order);
const payment = yield* ctx.run(processPayment, order);
yield* ctx.run(fulfillOrder, order, payment);
yield* ctx.run(notifyCustomer, order);
return { orderId, status: "fulfilled" };
}
Use when: Lifecycle management, state machines, long-lived domain objects.
The Hard Problems
Versioning
You deploy new code. Old executions are mid-flight. The replay now hits different code paths than what was recorded.
Solutions: Version-tag your workflows. Drain in-flight executions before deploying breaking changes. Or design steps to be additive (new steps at the end, never remove or reorder existing ones).
Side Effects at the Boundary
You sent an email in step 3. Step 4 crashes. On replay, step 3 returns the stored result — but the email was already sent.
Solutions:
- Outbox pattern — write to a durable outbox; a separate process delivers exactly once
- Idempotency keys — pass a deterministic key to external APIs so duplicates are no-ops
- Accept-and-compensate — accept that duplicates happen; send a correction if needed
Testing Durability
How do you test that replay actually works? That your workflow survives a crash at every possible step?
Approaches:
- Kill-and-resume test — start workflow, kill the process mid-step, restart, verify it completes correctly
- Replay unit test — capture an execution log, replay against new code, assert same result
- Transition tests — enumerate all valid state transitions, verify each one produces correct output
- Deterministic simulation testing (DST) — inject controlled randomness across thousands of runs, verify invariants hold
Full testing guide: See references/TESTING.md.
Observability
You cannot console.log your way through replay. You need to see: what step am I on, what's pending, what failed, what's the state of each promise.
What to monitor: Execution list (ID, status, duration), step timeline/waterfall, promise state graph, worker health, queue depth, error rates, retry storms.
Efficiency — Why This Matters
Durable execution has a reputation for being expensive and complex. It doesn't have to be.
| Approach | Monthly cost at 1M tasks/day | Infrastructure |
|---|
| Baked in (your DB) | $0 incremental | Your existing database |
| Resonate (self-hosted) | ~$5 (small VPS) to ~$170 (dedicated) | Single binary + SQLite |
| Temporal Cloud | ~$520+ | Managed cluster |
| AWS Step Functions | ~$250 (standard) | AWS-locked |
Why Resonate is cost-efficient:
- Single binary, zero external dependencies — no Redis, no Kafka, no Kubernetes
- SQLite for storage — no database server to run or pay for
- Tiny SDK (~1300 lines, zero deps) — minimal memory footprint
- Runs on serverless (Lambda, Cloud Functions, Edge Functions) — pay only for execution time
- No cluster, no operator, no dedicated infra team
Why baked-in is free:
- Uses your existing database (Postgres, SQLite, MySQL)
- No additional processes, services, or infrastructure
- The "framework" is 50-80 lines of code in your app
The lowest-cost durable execution is the one that runs on what you already have.
Anti-Patterns
- Making everything durable — Not every function needs crash recovery. Only durabilize the critical path. The rest can use simple retries.
- Ignoring replay semantics — Code that works on first run but breaks on replay: random IDs, reading the wall clock in control flow (
Date.now() in TS, time.time() in Python, Instant::now() in Rust, time.Now() in Go), external reads that return different values. Wrap non-deterministic operations as durable steps.
- Treating it as a queue — Durable execution is not a task queue. If you just need "retry this job," use a queue. Durable execution is for multi-step workflows with state.
- Skipping idempotency on external calls — Durable execution guarantees at-least-once execution of each step. Without idempotency keys on external APIs, you get duplicate charges, emails, and API calls.
- Over-abstracting early — Pick one pattern (usually saga or checkpoint), prove it works in your codebase, then expand. Don't build a generic workflow engine before you have a concrete use case.
Quick Reference — Resonate Context API (TypeScript syntax)
The shapes below are TypeScript (yield*). Python uses bare yield, Rust marks the function #[resonate::function] and writes ctx.run(...).await?, Go uses ctx.Run/ctx.RPC/ctx.Sleep/ctx.Promise (PascalCase) then f.Await(&out). See resonate-basic-durable-world-usage-{typescript,python,rust,go} for each. Durations are milliseconds in TypeScript, seconds in Python, and native Duration in Rust/Go.
| Method | Purpose | Example |
|---|
yield* ctx.run(fn, ...args) | Local durable step (checkpoint) | yield* ctx.run(chargeCard, order) |
yield* ctx.rpc(name, args, opts) | Remote durable step (cross-service) | yield* ctx.rpc("process", [item], { target: "poll://any@workers" }) |
yield* ctx.sleep(ms) | Durable timer (survives crashes) | yield* ctx.sleep(86_400_000) |
yield* ctx.promise(id, opts) | Suspend until external resolution | yield* ctx.promise("approval-123", { timeoutAt: ... }) |
Asset Templates
The asset templates are TypeScript. For Python/Rust/Go, start from the per-SDK resonate-basic-durable-world-usage-* (and pattern) skills instead.
| Template | Use when... |
|---|
assets/baked-in-checkpoint.ts | Adding framework-free durability to an existing app |
assets/baked-in-outbox.ts | Exactly-once side effects without a framework |
assets/resonate-worker.ts | Starting a new Resonate worker from scratch |
assets/resonate-gateway.ts | Building an HTTP gateway that dispatches durable workflows |
assets/resonate-hitl-worker.ts | Workflow that suspends for human approval |
assets/resonate-saga-worker.ts | Multi-step transaction with compensation on failure |
References
| Reference | Load when... |
|---|
references/BAKED-IN.md | Implementing framework-free durability with just your DB |
references/RESONATE-QUICKSTART.md | Setting up Resonate from scratch |
references/RESONATE-PATTERNS.md | Implementing a specific pattern with Resonate |
references/RESONATE-SDK.md | Needing TypeScript SDK API details, configuration, or wire protocol info (other SDKs: see resonate-basic-durable-world-usage-{python,rust,go}) |
references/TESTING.md | Verifying that durability actually works under failure |
references/DEPLOYMENT.md | Deploying to production (VPS, serverless, Docker) |
references/TROUBLESHOOTING.md | Debugging workflow hangs, failures, or unexpected behavior |
Normative spec reference: For promise lifecycle, handler semantics, and state-transition ground truth, see the Resonate specification — an executable abstract machine in Lean 4: https://github.com/resonatehq/resonate-specification
