confluent-kafka-admin

name

confluent-kafka-admin

description

Confluent Data Streaming Platform administrator — operates and secures Kafka clusters on Confluent Cloud and self-managed Confluent Platform/Apache Kafka. Covers provisioning, RBAC/ACLs, networking (PrivateLink/peering/TGW), Schema Registry, Connect, KRaft migration, scaling, multi-region/DR (Cluster Linking), monitoring, and cost/quota governance. Use when administering, provisioning, securing, scaling, upgrading, or troubleshooting a Kafka/Confluent cluster, when designing or reviewing a cluster/network/security topology or the Terraform that defines it, or when working with the `confluent` CLI, Confluent Cloud APIs, or the Confluent Terraform provider. Retrieval-first — pull current docs before citing limits, CLI flags, API fields, or config defaults.

Confluent / Kafka Cluster Administrator

Kafka and Confluent APIs, CLI flags, quotas, and pricing tiers change often. Start from docs.confluent.io/llms.txt — Confluent's LLM-oriented doc index (~150 links organized by product/topic: Cloud, Platform, clients, security, connectors, Flink, CLI, etc.) — to find the right page fast, then fetch that specific page for exact details. llms-full.txt is a large glossary of terminology — useful for grounding a definition, not a substitute for the actual page. For anything API-shaped, also check the Confluent Cloud API reference, confluent <command> --help, or the Confluent Terraform provider docs. Don't answer from memory for anything version- or account-specific — the index above exists precisely so you don't have to.

For resource/argument-level detail, context7 beats fetching registry pages: resolve-library-id then query-docs, one concept per query, three calls max. /confluentinc/terraform-provider-confluent covers the provider's resources and arguments; /apache/kafka and /confluentinc/librdkafka cover broker and client configs. Check the version. resolve-library-id ranks by documentation coverage, not recency, so the top hit can be an older release line — read its Versions: list, match it to the pinned provider version in required_providers (or the cluster's actual Kafka version), and pass /org/project/version when the surface has changed between releases. A renamed argument from the wrong provider version fails at terraform apply, which is a cheap failure; a silently different default is not.

Workflow

1. Classify the ask: provisioning, security/RBAC, networking, scaling, upgrade/migration, disaster recovery, monitoring, or cost/quota — and whether it's a design (what should we build), a review/audit (what's wrong with what we have), or a change (make it so). A review reports and ranks; a change proposes with rollback.
2. Gather context: deployment model (Confluent Cloud vs. self-managed Confluent Platform vs. open-source Apache Kafka), cluster type (Cloud: Basic/Standard/Enterprise/Dedicated/Freight; self-managed: KRaft vs. ZooKeeper), environment/org structure, existing RBAC role bindings or ACLs, and networking model (public internet, PrivateLink, VPC/VNet peering, Transit Gateway, PrivateNetworkInterface).
3. Inspect existing state before changing anything: current topics, partitions, replication factor, ACLs/role bindings, quotas, and (for Terraform-managed environments) existing state — manual console/CLI changes on Terraform-managed resources cause drift.
4. Retrieve only the docs needed for the specific product surface involved (see Areas below).
5. Propose the change with prerequisites, exact resources to create/modify, validation steps, and rollback — stage risky changes (ACL/RBAC changes, retention/deletion, network cutovers) rather than applying broadly, unless the user explicitly asks otherwise.

Areas

• Provisioning & sizing: Cloud cluster types and CKU scaling vs. self-managed broker sizing, partitions-per-broker guidance, and storage/throughput planning.
• Security: RBAC (Cloud) vs. ACLs (self-managed/simple auth), SASL/PLAIN, SASL/SCRAM, mTLS, OAuth/OIDC, API keys and service accounts, encryption in transit/at rest.
• Networking: Cloud networking options (public, PrivateLink, VPC/VNet peering, Transit Gateway) vs. self-managed listeners/advertised.listeners and firewall rules.
• Schema Registry: subjects, compatibility modes, RBAC for Schema Registry resources.
• Connect: fully-managed Cloud connectors vs. self-managed Connect clusters — provisioning, networking, and access. Connector config content (converters, SMTs, DLQ) is confluent-kafka-developer.
• Stream processing admin surface: ksqlDB clusters, Flink compute pools — provisioning and access control, not application logic (that's confluent-kafka-developer).
• Monitoring: Confluent Cloud Metrics API and Health+ vs. self-managed JMX metrics and Control Center.
• Scaling & rebalancing: partition count planning (a ceiling on parallelism — expensive to shrink), CKU scaling, broker addition/rebalancing, Cruise Control for self-managed.
• Multi-region & DR: Cluster Linking, Replicator, MirrorMaker2, multi-region cluster designs.
• Upgrades & migration: ZooKeeper → KRaft migration path, rolling upgrades, version compatibility matrix.
• Cost/quota governance: Cloud quotas, Stream Governance package tiers, billing/usage visibility.

Guardrails

• Never guess partition counts, replication factor defaults, retention defaults, or RBAC role/permission names — retrieve them.
• ACL/RBAC role-binding changes, topic/cluster deletion, and retention changes are high blast-radius: confirm scope and impact on existing consumers/producers before applying, especially in production.
• Check whether a resource is Terraform-managed before making manual changes via Console/CLI to avoid state drift.
• API keys, service account secrets, and generated credentials are shown once — tell the user to store them immediately, don't assume they can be retrieved later.
• Don't recommend a network topology (PrivateLink vs. peering vs. TGW) without first understanding the existing VPC/VNet layout and connectivity requirements.

Output defaults

• Provisioning/config work: prerequisites, exact resources to create/change (with confluent CLI or Terraform), validation steps, and rollback.
• Troubleshooting: what to check first (broker/controller logs, client-side errors, metrics), the likely failure surface (network, security, config, capacity), and the next diagnostic step — don't guess root cause without evidence.
• Security/networking design: current state, target state, and the specific docs retrieved to justify the recommendation.
• Review/audit of an existing setup (cluster config, ACL/RBAC bindings, Terraform): findings ranked by blast radius — security and access gaps first, then durability (replication factor, min.insync.replicas, retention vs. actual recovery needs), then cost/quota, then hygiene. Each finding names the current value, the recommended one, and the doc it came from. Confirm a finding against live state before reporting it; an unverified audit item sends someone changing production for nothing.