integration-patterns

name	integration-patterns
description	Apply Hohpe enterprise integration patterns updated for event-driven 2026 stacks. Use when wiring services with messaging, request/reply, claim check, outbox, or saga.

Integration Patterns

Pick a small, named pattern per integration and document it in AsyncAPI. Favor asynchronous messaging, idempotent receivers, and explicit transactional outboxes over distributed transactions.

Stack Baseline (2026)

Concern	Recommended
Message bus	Kafka 3.7, Pulsar 3.3, Redpanda, NATS JetStream
Cloud eventing	EventBridge Pipes, Google Eventarc, Azure Event Grid
Schema	Avro / Protobuf / JSON Schema in registry
Contracts	AsyncAPI 3.0, CloudEvents 1.0
Workflow / saga	Temporal 1.24, AWS Step Functions, Camunda 8
Outbox / CDC	Debezium 2.6, Estuary Flow
Sync RPC	gRPC, Connect-RPC, REST + Problem+JSON
Object payloads	S3 / R2 / GCS for claim check
Observability	OpenTelemetry traces with traceparent propagation

When to Use

• New service-to-service or system-to-system integration.
• Replacing brittle point-to-point or shared-DB integration.
• Designing async workflows that span multiple services.
• Bridging legacy and modern systems.

Prerequisites

• Bounded context map; clear ownership of producers and consumers.
• Schema registry and AsyncAPI repo per domain.
• Idempotency strategy for every consumer.
• DLQ + retry policy template.

Instructions

1. Pick the right pattern

flowchart TD
    Q{Integration intent?} --> A[Notify -> Event-carried state transfer]
    Q --> B[Need answer now -> Request/Reply over RPC]
    Q --> C[Need answer eventually -> Async req/reply with correlationId]
    Q --> D[Long workflow -> Saga / Process Manager via Temporal]
    Q --> E[Large payload -> Claim Check + object store]
    Q --> F[DB + bus consistency -> Transactional Outbox + CDC]

2. Define every message in AsyncAPI 3.0

asyncapi: 3.0.0
info: { title: Orders Events, version: 1.4.0 }
servers:
  prod:
    host: kafka.prod:9093
    protocol: kafka-secure
channels:
  orders.placed.v1:
    address: orders.placed.v1
    messages:
      OrderPlaced:
        $ref: '#/components/messages/OrderPlaced'
operations:
  publishOrderPlaced:
    action: send
    channel: { $ref: '#/channels/orders.placed.v1' }
components:
  messages:
    OrderPlaced:
      contentType: application/avro
      schemaFormat: application/vnd.apache.avro;version=1.11
      payload:
        $ref: 'schemas/order-placed.avsc'

3. Implement the transactional outbox

BEGIN;
INSERT INTO "order"(id, customer_id, total_cents, status)
VALUES ($1,$2,$3,'PLACED');

INSERT INTO outbox(id, aggregate_id, type, payload, headers)
VALUES (gen_random_uuid(), $1, 'OrderPlaced', $4::jsonb, $5::jsonb);
COMMIT;
-- Debezium reads outbox and publishes to Kafka with the outbox SMT

This avoids dual-write inconsistency without 2PC.

4. Apply Claim Check for large payloads

Producer writes the binary to S3, publishes a small message with the URI, checksum, and size. Consumer fetches on demand and verifies the checksum.

5. Use Saga / Process Manager for distributed workflows

sequenceDiagram
    participant Orders
    participant Payments
    participant Inventory
    participant Saga as Temporal
    Orders->>Saga: OrderPlaced
    Saga->>Payments: ChargeCard
    Payments-->>Saga: Charged
    Saga->>Inventory: Reserve
    Inventory-->>Saga: OutOfStock
    Saga->>Payments: Refund (compensation)
    Saga->>Orders: OrderCancelled

Compensations are first-class. No 2PC.

6. Make every consumer idempotent

• Dedupe by messageId in a TTL'd table or Redis SET.
• Use upserts keyed by domain id.
• Wrap side effects in the same transaction as the dedupe insert.

7. Operate retries and poison messages

• Exponential backoff with jitter; cap at N attempts.
• Per-topic DLQ with original headers, error class, and stack.
• Replay tool that supports filter by header and time window.

Common Pitfalls

Pitfall	Why it hurts	Mitigation
Shared DB integration	Couples release cycles	Replace with events or API
Synchronous chains across services	Cascading failures	Async + saga
Dual write to DB and bus	Silent inconsistency	Transactional outbox
Huge messages on the bus	Broker pressure, slow consumers	Claim Check
Unversioned topics	Breaks consumers on change	name.v1 + registry compat
No DLQ	Loops or data loss	DLQ + replay tooling
RPC for fire-and-forget	Tight coupling	Event-carried state transfer

Output Format

Deliver: integration matrix (producer x consumer x pattern), AsyncAPI 3.0 docs, outbox/CDC config, saga workflow code, idempotency strategy, DLQ + replay runbook, and OpenTelemetry trace plan.

Authoritative References

• Hohpe and Woolf, Enterprise Integration Patterns (eaipatterns.com)
• AsyncAPI 3.0 spec - https://www.asyncapi.com/docs/reference/specification/v3.0.0
• Transactional Outbox, microservices.io - https://microservices.io/patterns/data/transactional-outbox.html
• Temporal patterns - https://docs.temporal.io/encyclopedia
• CloudEvents 1.0 - https://cloudevents.io/