streaming-patterns
Provides Kafka producer and consumer best practices and dead letter queue patterns for streaming data; use when designing Kafka consumers or producers or implementing real-time pipelines.
Datos de origen
- Repositorio
- sawrus/agent-guides
- Última actividad en el origen
- 16 de octubre de 2026 a las 19:08
- Idioma detectado de SKILL.md
- inglés
- Estrellas
- 19
- Forks
- 4
Opciones de instalación
De forma predeterminada está seleccionado el prompt que primero revisa el origen. Puedes cambiar a un comando directo o descargar una copia local.
Revisa los archivos de origen
Lee SKILL.md y los archivos complementarios que muestra SkillsMP antes de decidir si quieres instalarlo.
Mostrando SKILL.md
SKILL.md
Instrucciones de origen · Vista previa de solo lectura- name
- streaming-patterns
- description
- Provides Kafka producer and consumer best practices and dead letter queue patterns for streaming data; use when designing Kafka consumers or producers or implementing real-time pipelines.
# Skill: Streaming Data Patterns
## When to load
When designing Kafka consumers/producers or implementing real-time pipelines.
## Producer Best Practices
```python
producer = Producer({
"bootstrap.servers": settings.KAFKA_BROKERS,
"acks": "all", # Wait for all replicas
"retries": 5,
"enable.idempotence": True, # Exactly-once delivery
"compression.type": "snappy",
})
```
## Consumer Pattern
```python
consumer = Consumer({
"group.id": "order-processor-v1", # Version when changing logic
"auto.offset.reset": "earliest",
"enable.auto.commit": False, # Manual commit: only after successful processing
})
msg = consumer.poll(timeout=1.0)
try:
process(deserialize(msg.value()))
consumer.commit(msg) # Commit only after success
except ProcessingError as e:
publish_to_dlq(msg, e) # Dead-letter, don't block partition
consumer.commit(msg)
```
## Dead Letter Queue
```json
{
"original_topic": "orders.created.v1",
"original_offset": 12345,
"original_payload": "...",
"error_message": "Deserialization failed",
"error_timestamp": "2026-02-16T10:30:00Z",
"retry_count": 3
}
```
Ver en GitHub