メニュー
Guides agents through event streaming and real-time data pipeline design. Use when working with Kafka, Kinesis, Flink, stream processing, windowing, stateful consumers, or near-real-time publish flows.
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
SOC 職業分類に基づく
Forces replay-safe rollout plans, reconciliation gates, and rollback paths before executing any data backfill or historical reprocessing. Use when running /backfill, rerunning pipelines, repairing publish windows, or any work that risks double-counting or downstream corruption.
Helps agents classify data engineering work, choose the right preset and skill bundle, and pick the safest next command. Use when starting a session, triaging an ambiguous request, or deciding how to proceed.
Guides agents through resiliency testing for data platforms. Use when designing or running failure drills, recovery validation, failover tests, replay-safety checks, dependency outage exercises, or fault injection for pipelines and publishes.
Guides agents through production data incidents. Use when a pipeline fails, publishes bad data, misses an SLA, partially loads, corrupts state, or requires rollback, replay, or stakeholder communication.
Enforces production Kafka guardrails including non-breaking schema evolution, dead-letter queues for poison messages, and acks=all producer durability. Use when designing or changing Kafka topics, producers, consumers, schema registry policies, or streaming recovery paths.
Guides agents to wire Model Context Protocol servers for live data platform observability including Spark execution plans, OOM diagnosis, Kafka consumer lag, and orchestration run state. Use when agents need cluster metrics, streaming lag, or job diagnostics instead of blind code changes.
| name | streaming-and-messaging-systems |
| description | Guides agents through event streaming and real-time data pipeline design. Use when working with Kafka, Kinesis, Flink, stream processing, windowing, stateful consumers, or near-real-time publish flows. |
Use this skill for continuous data movement and real-time processing. It helps agents design safe event pipelines around brokers and stream processors such as Kafka, Kinesis, and Flink, with attention to ordering, state, replay, schema evolution, and delivery guarantees.
Kafka topics, consumers, or stream processorsKinesis for event ingestion or streaming deliveryFlink jobs or stateful stream transformationsDo not use this when the workload is purely batch and does not require streaming semantics.
Define the event contract. Include:
Choose the right platform role.
Kafka: durable event backbone and broad ecosystemKinesis: AWS-native streaming ingestion and transportFlink: stateful stream processing and event-time logicDesign for state and late data. Account for:
Make consumer behavior observable.
Lag, failed checkpoints, poison messages, schema drift, and dead-letter handling should be planned, not discovered in production.
For production Kafka hardening, load kafka-resilience-and-schema-evolution.
For live lag or broker metadata before changes, load mcp-data-observability-integration.
Treat replay as a first-class operation. Reprocessing events should not depend on manual guesswork.
| Rationalization | Reality |
|---|---|
| "Streaming just means faster batch." | Streaming introduces ordering, state, replay, and time semantics that batch systems do not have. |
| "We can ignore schema evolution because events are small." | Event size does not reduce contract risk; schema drift can break many downstream consumers. |
| "Exactly-once is automatic." | Delivery semantics depend on end-to-end design, state handling, sinks, and replay behavior. |