| name | cdc-and-source-to-stream-ingestion |
| description | Designs CDC and streaming ingestion from operational sources to event streams. Use when capturing database changes, replicating operational data, or building source-to-stream pipelines. |
CDC and Source-to-Stream Ingestion
When to use
Use when turning database/file/API changes into streams.
Objective
Produce a practical, concise, traceable architecture artefact that a coding agent can use to guide implementation or review.
Procedure
- Identify source and owner.
- Select capture method.
- Define snapshot and incremental flow.
- Define keys and ordering.
- Define delete/tombstone handling.
- Define schema evolution.
- Define backfill/replay.
- Define lag and reconciliation checks.
Required outputs
- Capture design
- Keys/order handling
- Schema evolution
- Delete handling
- Backfill/replay
- Monitoring
Best-practice alignment
Apply event-driven best practice: business event semantics, producer/consumer ownership, schema contracts, compatibility, ordering, partitioning, idempotency, replay, retention, lineage, observability and operational SLOs.
Quality checks
- Source impact is understood.
- Deletes and updates are handled.
- Reconciliation exists.
- Lineage is preserved.
Avoid
Do not treat raw CDC records as business events without translation.
Decision template
Use this template before implementation: source system and owner; capture mode; snapshot boundary; primary key and ordering field; update/delete semantics; schema evolution path; reconciliation rule; replay and backfill limit; lag SLI; lineage record. If downstream consumers need business facts, translate raw CDC records into governed events before publishing.
References
Verification
