agentsop-llm-tool-idempotency

# LLM Tool Idempotency · SOP > One-liner: **The LM is at-least-once; the tool must be at-most-once.** > Every framework that promises "durable execution" still re-runs node bodies > on resume. Every HTTP client retries on timeout. Every model occasionally > emits the same tool_call twice. Idempotency belongs in the tool, not in a > wish. --- ## 1. 何时激活 (Activation Rules) Activate this skill when **any** of the following triggers fire: - You're defining a tool whose name contains `send_`, `create_`, `charge_`, `post_`, `write_`, `publish_`, `transfer_`, `delete_`, `update_`, or `notify_`. - The tool wraps a third-party API call (Stripe, SendGrid, Twilio, Slack, Discord webhook, S3 PUT, payment gateway, internal write API). - You're inside a LangGraph node that contains an `interrupt(...)` call AND a side effect in the same function body — the resume re-runs the body from the top `[langgraph/gotchas]`. - The tool is invoked through MCP, OpenAI tool-calling, Anthropic tool use, CrewAI delegation, or any layer where a transport timeout could be interpreted as "retry" even though the operation succeeded server-side. - The user reports "the agent sent it twice" / "charge appeared twice" / "duplicate row" / "got two emails". - Your test harness records the same `(tool_name, args_hash)` invoked more than once within a single user turn. **Do not activate** when the tool is read-only (GET-equivalent), or when the side effect is genuinely commutative AND verified safe under duplication. --- ## 2. 核心心智模型 (Core Mental Model) ### 2.1 The fundamental asymmetry ``` LM tool-call semantics Tool side-effect semantics ───────────────────── ───────────────────────── At-least-once delivery Must be at-most-once (network retry, framework (one charge, one email, resume, model dup-emit, one record) user re-prompt) │ │ └────── gap to bridge ───────────┘ ↓ IDEMPOTENCY KEY (a stable identifier the LM commits to BEFORE the call, which the tool dedupes on) ``` **The LM cannot promise it'll call exactly once.** Four independent retry sources stack here: 1. **Transport retry**: HTTP client (or MCP transport) sees a timeout, the server actually completed the operation, the client retries. Stripe's docs call this out as the canonical case `[stripe/idempotency]`. 2. **Framework resume**: LangGraph re-runs the *entire node body* on resume from an interrupt. Code before the interrupt re-executes on every resume `[langgraph/gotchas]`. Same applies to Temporal-style workflows on replay. 3. **Model duplicate emission**: Streaming sometimes yields the same `tool_call_id` twice (rare but documented in OpenAI tool-calling); model may re-emit on context-compaction round-trips. 4. **User-level retry**: The user clicks "send" twice, or re-runs the agent after timeout, with the same instructions. Any one of these turns a single-intent action into multiple side effects unless the tool itself dedupes. ### 2.2 The promise inversion Naive design says: "I'll make the LM call the tool exactly once." Mature design says: "I'll make the tool ignore the second call." The inversion matters because the LM is in the *control* path; the tool is in the *execution* path. Execution-path guarantees are the only ones that hold under failure. ### 2.3 The idempotency key has to come from the agent, not the tool A common bug: the tool generates a UUID *inside* itself, then dedupes on that UUID. This breaks because retry creates a *new* UUID. The key has to: - Be **chosen by the caller** (the agent / orchestrator). - Be **deterministic for a single logical operation** — same input → same key on retry. - Be **persisted into agent state** before the tool call, so a resume picks up the same key. Stripe's pattern (the industry reference): client generates an idempotency key (typically UUIDv4), sends it as `Idempotency-Key: <uuid>` HTTP header. Server caches the response for 24 hours keyed by that header `[stripe/idempotency]`. A retry with the same key returns the cached response — no second charge. ### 2.4 Three classes of side effect, three idempotency strategies | Side-effect class | Examples | Idempotency mechanism | |---|---|---| | **Non-replayable external API** | Stripe charge, Twilio SMS, SendGrid email | Caller-supplied idempotency key passed to API (header or body field) | | **Internal write to a store you own** | `INSERT` into your DB, `PUT` to your S3 bucket, append to your queue | Dedup table keyed on `(operation_id, args_hash)` with unique constraint, OR `INSERT … ON CONFLICT DO NOTHING`, OR content-addressed write | | **Compensatable operation** (no native idempotency, must undo on duplicate) | Bank wire to a counterparty, physical device actuation | Saga pattern: record intent, perform, compensate-on-duplicate-detection; or a reservation token (two-phase) | The SOP in §3 walks you through classifying first, then picking the mechanism. --- ## 3. SOP 工作流 (Agentic Protocol) ### Step 1 · Classify the side effect Ask three questions, in order: 1. **Does the downstream API accept an idempotency key?** - Yes (Stripe, Square, modern PayPal, AWS SDK with client tokens, Anthropic Files API): use the native key. Stop here. - No: continue. 2. **Do I own the store being written to?** - Yes (my Postgres, my S3 bucket, my Kafka topic): build a dedup layer. Go to Step 2. - No: continue. 3. **Is the effect compensatable?** - Yes (can issue a refund / send a correction): saga / compensation. - No (irreversible physical action, sent SMS): the only safe option is **block the second call** — fail-closed dedup table, no fallback. ### Step 2 · Choose the idempotency mechanism Decision tree: ``` Side effect classified above. │ ├─ Native idempotency-key API (Stripe et al.) │ → OP-1: pass key in HTTP header / SDK kwarg │ ├─ Internal write you own │ ├─ Operation has natural content identity (file hash, message dedup id) │ │ → OP-3: content-addressed write (PUT key = sha256(content)) │ ├─ Single-row insert │ │ → OP-4: INSERT ... ON CONFLICT (idempotency_key) DO NOTHING RETURNING * │ └─ Multi-step write │ → OP-2: dedup table + transaction at the tool boundary │ └─ Compensatable, non-idempotent external call → OP-5: saga (record intent → call → compensate on duplicate) ``` ### Step 3 · Decide where the key is generated and how it's named | Source of key | When to use | Pitfall | |---|---|---| | `uuid4()` in agent state, persisted before call | Default for one-shot operations | Lost if state isn't persisted before the side effect | | `hash(user_id, intent, day)` | Idempotency per-user-per-intent-per-day (e.g., daily digest email) | Too coarse → blocks legitimate second sends | | `hash(canonicalized_args)` | Content-determined (same email body to same address) | Blocks legitimate "send the same message again later" | | `hash(thread_id, node_id, run_id)` | LangGraph node-level dedup | Doesn't help across thread resumes that re-enter the same node | | **Composite**: `hash(thread_id, node_id, attempt_args)` | Recommended for LangGraph node bodies that re-run on resume | One more arg to wire | **Naming convention**: store the key in agent state as `{tool_name}_idempotency_key`, persist it **before** the tool call. On resume, check if state already has a key — if yes, reuse; if no, generate. ### Step 4 · Pass the key through every layer ``` Agent state ──→ Tool wrapper ──→ HTTP client ──→ External API (uuid) (header) (transport) (server) ``` Verify each layer: - Agent state stores the key in a serialized, checkpointed field (not in-memory only). - Tool wrapper reads from state, not from a fresh `uuid4()` call. - HTTP client uses the key as `Idempotency-Key` header *and* propagates it on client-side retries (don't generate a new key per HTTP retry). - Server caches the response keyed on that header for a documented TTL (Stripe: 24 h; AWS SQS dedup: 5 min default; design your own to ≥ 1 h). ### Step 5 · Handle the conflict response A successful idempotent retry returns the **original** response. A *conflict* (same key, different args) usually returns 4xx — Stripe returns `HTTP 409 Conflict` with code `idempotency_key_in_use` if the request payload mismatches `[stripe/idempotency]`. Decision rules: - **Same key, same args, cached response** → return to LM as if first call; surface a `was_replay: true` flag for observability. - **Same key, different args** → bug in the caller. Fail loudly, do not fall back to "just send again". This catches "agent re-prompted with new message content but reused old key" bugs. - **Key in flight** (server still processing first call) → either block (server-side wait) or return 409 / 425 Too Early. Caller must back off. ### Step 6 · Persist the dedup record on the *commit* side, atomically The classic race: tool calls API, API succeeds, tool crashes before recording the dedup row. Retry now duplicates because the dedup row is missing. Two safe patterns: - **Database transaction encloses both**: insert dedup row AND record of API call in the same transaction; commit. If external API is the side effect, the dedup row goes in *before* the external call, with a `status: pending` flag, then updated to `status: completed` after. On retry, `pending` means "wait or fail" — not "go ahead and call again". - **Use the external API's idempotency** as the source of truth (OP-1) — skip your own dedup table entirely; trust Stripe's. Recommended when available. ### Step 7 · Expose `was_replay` in the tool result The LM should know it didn't actually re-send. Return: ```json { "result": { /* original response payload */ }, "was_replay": true, "original_called_at": "2026-05-19T10:11:12Z" } ``` Why: prevents the agent from thinking "send failed, let me try a different phrasing" and producing a logical (not technical) duplicate. --- ## 4. 操作模型 (Operation Models) Format: **Trigger → Action → Output → Evidence**. ### OP-1 · Idempotency-key HTTP header (Stripe pattern) - **Trigger**: Wrapping a third-party API that supports idempotency keys (Stripe, Square, modern PayPal, Adyen, Anthropic Files, AWS with client tokens). - **Action**: Before the call, read or generate `key = state.get("charge_key") or uuid4()`; persist to state; pass as `Idempotency-Key: <key>` header (or SDK kwarg, e.g. `stripe.PaymentIntent.create(..., idempotency_key=key)`). - **Output**: At most one server-side execution per key; retries return cached response. TTL ~24 h for Stripe. - **Evidence**: `[stripe/idempotency]` "Stripe's idempotency works by saving the resulting status code and body of the first request made for any given idempotency key, regardless of whether it succeeded or failed." ### OP-2 · Dedup table at the tool-call layer - **Trigger**: Internal API with no native idempotency, multi-row write, or aggregation that doesn't fit content-addressing. - **Action**: Create table `tool_call_dedup(key TEXT PRIMARY KEY, tool_name TEXT, args_hash TEXT, result JSONB, status TEXT, created_at TIMESTAMPTZ)`. On call: `INSERT … ON CONFLICT(key) DO NOTHING RETURNING …`. If insert succeeds, perform side effect, then `UPDATE … SET result=…, status='completed'`. If conflict, read row, return its `result`. - **Output**: Single execution per key across all retries; replayable result. - **Evidence**: This is the generalised form of Stripe's server-side implementation `[stripe/idempotency]`; appears in PayPal, AWS SDK whitepapers as the standard "dedup table" pattern `[aws/idempotency-whitepaper]`. ### OP-3 · Content-addressed write - **Trigger**: Writing a file, blob, or message whose identity *is* its content (build artifact, immutable record, derived data). - **Action**: Compute `key = sha256(canonical_content)`; write to `PUT /bucket/{key}`. S3 conditional put (`If-None-Match: *`, available since 2024) returns 412 on second write. - **Output**: Repeated writes are no-ops (same key → same content). No dedup table needed. - **Evidence**: `[aws/s3-conditional]` S3 conditional writes documentation; this is the underlying primitive in Git, IPFS, content-addressable storage generally. ### OP-4 · Database `INSERT … ON CONFLICT … DO NOTHING` - **Trigger**: The side effect is a single row insert into a table you own (audit log, message record, user-generated record). - **Action**: Add a unique constraint on `(idempotency_key)` or `(user_id, intent, request_id)`. Use `INSERT … ON CONFLICT(idempotency_key) DO NOTHING RETURNING id`. - **Output**: Empty result set means "duplicate, no-op". Non-empty means "first call, inserted". - **Evidence**: PostgreSQL `ON CONFLICT` semantics `[pg/insert]`; MySQL `INSERT IGNORE`; SQLite `INSERT OR IGNORE`. The database does the dedup; no application race window. ### OP-5 · Saga / compensation for non-idempotent compensatable APIs - **Trigger**: The downstream system has no idempotency API but the effect can be undone (refund, retraction, correction email). - **Action**: 1. Record intent in your DB with `status='pending', key=<id>` in a transaction. 2. Call the external API. 3. Update to `status='completed', external_id=<their_id>`. 4. On retry detected (key already present, status='completed'): return the stored `external_id`, do not call again. 5. On retry detected (status='pending', call in flight): wait + poll, do not call again. 6. On reconciliation job finding a `pending` past TTL: investigate; possibly compensate the now-known external_id. - **Output**: At-most-once semantics with a recovery path. - **Evidence**: Saga pattern `[microservices/saga]`; Temporal workflow `[temporal/idempotency]` uses the same shape under the hood. ### OP-6 · LangGraph: pull key from state before `interrupt()` - **Trigger**: A LangGraph node both calls `interrupt()` and performs a side effect. - **Action**: ```python def send_email_node(state): # Step 1: derive a stable key BEFORE interrupt key = state.get("email_key") or str(uuid4()) state["email_key"] = key # persist via reducer decision = interrupt({"to": state["to"], "body": state["body"]}) if decision != "approve": return {"status": "rejected"} # Step 2: side effect uses the key — safe under resume result = send_email_api(to=state["to"], body=state["body"], idempotency_key=key) return {"email_sent": True, "email_key": key, "result": result} ``` - **Output**: Resume re-runs the node body but the key is stable, so the external send is a no-op on the second pass. - **Evidence**: `[langgraph/gotchas]` "Side effects after interrupt() will re-run on resume — wrap them with an idempotency key drawn from state." See LangGraph payment-double-charge Case Study 4 in `output/langgraph-sop-skill/SKILL.md`. ### OP-7 · MCP / tool-boundary dedup window - **Trigger**: MCP server exposing a side-effectful tool to a client that may retry on transport timeout. - **Action**: In the MCP tool implementation, accept an `idempotency_key` argument as part of the tool schema; require it for side-effectful tools. Maintain a server-side dedup window (e.g., 5-minute in-memory LRU + persistent backing for cross-restart safety). - **Output**: Transport-level retries (which MCP clients do) hit the dedup cache; only one execution. - **Evidence**: MCP spec does not mandate idempotency, so this is a *server* responsibility — analogous to Stripe's server-side dedup. Surface the requirement explicitly in the tool's input schema. ### OP-8 · Pre-call state checkpoint (works for any framework) - **Trigger**: Working in a framework without first-class durable execution (plain Python loop, CrewAI task, custom orchestrator). - **Action**: Before calling a side-effectful tool, persist `(intent_id, args_hash, key, status='in_progress')` to a local sqlite or durable store. After call: update to `status='completed'` with result. Wrap in a context manager so a crash leaves `in_progress`, and a retry reads that state and either polls or fails-closed.