// Kafka topic management, consumer group monitoring, message production/consumption, and cluster health diagnostics. Use when working with Kafka brokers, topics, consumer lag, or debugging pipeline issues.
Enable, configure, and query Elasticsearch security audit logs. Use when the task involves audit logging setup, event filtering, or investigating security incidents like failed logins.
Authenticate to Elasticsearch using native, file-based, LDAP/AD, SAML, OIDC, Kerberos, JWT, or certificate realms. Use when connecting with credentials, choosing a realm, or managing API keys.
Manage Elasticsearch RBAC: native users, roles, role mappings, document- and field-level security. Use when creating users or roles, assigning privileges, or mapping external realms like LDAP/SAML.
Execute ES|QL (Elasticsearch Query Language) queries, use when the user wants to query Elasticsearch data, analyze logs, aggregate metrics, explore data, or create charts and dashboards from ES|QL results.
Ingest and transform data files (CSV/JSON/Parquet/Arrow IPC) into Elasticsearch with stream processing and custom transforms. Use when loading files or batch importing data.
Diagnose and resolve Elasticsearch security errors: 401/403 failures, TLS problems, expired API keys, role mapping mismatches, and Kibana login issues. Use when the user reports a security error.
| name | kafka |
| version | 1.0.0 |
| description | Kafka topic management, consumer group monitoring, message production/consumption, and cluster health diagnostics. Use when working with Kafka brokers, topics, consumer lag, or debugging pipeline issues. |
# Check broker health
kafka-broker-api-versions --bootstrap-server localhost:9092
# List topics with partition counts
kafka-topics --bootstrap-server localhost:9092 --list
# Describe a topic (partitions, replicas, ISR)
kafka-topics --bootstrap-server localhost:9092 --describe --topic <topic>
# Create topic
kafka-topics --bootstrap-server localhost:9092 --create --topic <name> --partitions 3 --replication-factor 1
# Delete topic
kafka-topics --bootstrap-server localhost:9092 --delete --topic <name>
# Increase partitions
kafka-topics --bootstrap-server localhost:9092 --alter --topic <name> --partitions <count>
# List all consumer groups
kafka-consumer-groups --bootstrap-server localhost:9092 --list
# Show consumer group lag (key debugging command)
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group <group>
# Reset consumer group offsets (use with caution)
kafka-consumer-groups --bootstrap-server localhost:9092 --group <group> --topic <topic> --reset-offsets --to-earliest --execute
# Produce a message with key
echo "key:value" | kafka-console-producer --bootstrap-server localhost:9092 --topic <topic> --property "parse.key=true" --property "key.separator=:"
# Consume messages from beginning
kafka-console-consumer --bootstrap-server localhost:9092 --topic <topic> --from-beginning --max-messages 10
# Consume with key and value
kafka-console-consumer --bootstrap-server localhost:9092 --topic <topic> --from-beginning --property print.key=true --property key.separator="|"
# Show earliest and latest offsets
kafka-run-class kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic <topic>
When debugging Kafka-dependent services in a deer sandbox with kafka_stub=true:
localhost:9092 inside the sandboxrpk topic list --brokers localhost:9092 for topic managementrpk topic produce <topic> --brokers localhost:9092 to send test messagesrpk topic consume <topic> --brokers localhost:9092 to read messagesrpk group list --brokers localhost:9092 for consumer groupskafka-consumer-groups --describe --group <group>systemctl status <consumer-service>GetOffsetShell --topic <topic>ss -tlnp | grep 9092kafka-broker-api-versions --bootstrap-server <host>:9092journalctl -u kafka --no-pager -n 100journalctl -u logstash --no-pager -n 100localhost:9092