webhook-design

name	webhook-design
description	Design inbound or outbound webhooks with authenticated origin, stable event contracts, fast acknowledgement, duplicate/reorder handling, retries, idempotent processing, observability, and replay recovery.

Webhook Design

Use when one system asynchronously notifies another about events over HTTP or a similar push callback.

Procedure

1. Define event ownership and semantics: event type, stable event ID, resource/tenant identity, occurrence time, schema/version, ordering guarantees, and whether payload is snapshot, delta, or reference requiring a follow-up fetch.
2. For inbound webhooks, authenticate origin using the provider's supported signature/mTLS/token mechanism and validate against the exact raw/body representation the scheme requires before trusting the payload.
3. Apply timestamp/replay-window checks when the signature protocol supports them, while keeping legitimate provider retries possible.
4. Return acknowledgement quickly after durable acceptance when processing may be slow. Do not hold the provider request open for unrelated downstream work if a queue/durable inbox is appropriate.
5. Assume duplicate delivery unless the provider explicitly proves otherwise. Deduplicate using stable event/business identifiers and make downstream effects idempotent.
6. Do not assume global order. Use event versions/sequence numbers or fetch authoritative current state when correctness depends on ordering and the provider supplies such mechanisms.
7. For outbound webhooks, define retryable status/network failures, bounded backoff, delivery timeout, signing/secret rotation, endpoint disablement, and dead-letter/manual replay policy.
8. Protect tenant/resource boundaries. A validly signed event still needs schema validation and correct mapping to the authorized internal tenant/resource.
9. Record delivery/receipt state, attempts, result, provider/request IDs, event ID/type, and non-sensitive diagnostics so a specific event can be traced or replayed.
10. Test valid, invalid signature, expired/replayed, duplicate, out-of-order, malformed, slow, failing endpoint, retry exhaustion, and manual replay behavior.