skills

Teaches how to build streaming data pipelines with Redpanda Connect (formerly Benthos) — declarative YAML config, the input/pipeline/output model, Bloblang mapping language, error handling, and canonical pipeline patterns. Use when: building or running a Redpanda Connect pipeline; wiring an input to an output with processors; writing Bloblang mappings or mutations; choosing connectors (kafka, redpanda, http_server, file, generate, sql_*, aws_*, gcp_*); running pipelines with rpk connect run or redpanda-connect; configuring buffers, caches, metrics, rate limits, or the global redpanda block; writing config.yaml for redpanda-connect; asking what Connect components exist and how to discover them; error handling with fallback outputs or catch processors; batching records before writing; understanding the difference between mapping and mutation processors; choosing between the bloblang and mapping processor names; connecting to Kafka or Redpanda with SASL/TLS from Connect; migrating Kafka clusters to Redpanda with

Provision and manage Redpanda Cloud BYOC (Bring Your Own Cloud) clusters via the Control Plane API (https://api.redpanda.com) and the rpk cloud byoc agent plugin. Covers OAuth2 client-credentials auth, creating Network resources per cloud provider (AWS VPC/subnets/IAM, GCP network/project/bucket, Azure VNet/subnets/identities), creating TYPE_BYOC clusters (network_id, throughput_tier, cloud_provider_tags, customer_managed_resources), polling long-running Operations until READY, and running the Terraform-backed rpk cloud byoc apply/destroy/validate flow with --redpanda-id. Use when: creating BYOC clusters in AWS/GCP/Azure via the public API; provisioning or tearing down BYOC networks; wiring customer-managed IAM roles/buckets/subnets into a Redpanda cluster; setting up private connectivity (AWS PrivateLink, GCP Private Service Connect, Azure Private Link, VPC/network peering via NetworkPeeringService, and AWS Transit Gateway centralized egress); registering cross-account AWS access (CloudProviderAccessService,

Configures and operates an Oxla cluster: YAML config (default_config.yml), OXLA__ environment-variable overrides, node ports and roles, leader election, storage backends (local/S3/GCS/Azure), memory limits, access control, TLS, OIDC, and the HTTP-based ConnectRPC admin service (port 9090) with its LoggingService (GetLogLevel/SetLogLevel). Also covers the Prometheus metrics endpoint (port 8080) and Docker/Compose deployment patterns. Covers Oxla's lakehouse and streaming differentiators: Apache Iceberg REST catalogs (CREATE ICEBERG CATALOG, oauth2/basic/aws_sigv4 auth, feature_flags.allow_iceberg_queries), transparent Redpanda/Kafka integration (CREATE REDPANDA CATALOG, topic-backed tables, schema_lookup_policy/error_handling_policy), object-storage connections (CREATE STORAGE TYPE S3/GCS/ABS), plus security: OIDC/JWT auth, SCRAM passwords, centralized access control, and AES-256-GCM secret encryption (OXLA_ENCRYPTION_KEY). Use when: configuring an Oxla cluster; setting ports or node names; enabling TLS or OID

Query external data from Oxla — Kafka topics via catalogs, Apache Iceberg tables, and S3/GCS/Azure parquet/ORC files — alongside native Oxla tables. Use when: querying Kafka topics with CREATE KAFKA CATALOG or CREATE REDPANDA CATALOG; reading Apache Iceberg tables using the catalog=>path.table syntax; loading or exporting parquet/ORC files from S3/GCS/Azure (COPY FROM/TO); joining external data with native tables; or inspecting external metadata via system.kafka_connections, system.kafka_sources, system.iceberg_catalogs, and system.iceberg_tables. Also covers the Redpanda Enterprise features that produce the data Oxla reads — Iceberg Topics (redpanda.iceberg.mode and the redpanda.iceberg.* topic properties; iceberg_enabled and iceberg_rest_catalog_* cluster properties), Tiered Storage (cloud_storage_enabled), and Server-Side Schema ID Validation (enable_schema_id_validation, redpanda.value.schema.id.validation) — all of which require a Redpanda Enterprise license. Trigger phrases: "query Kafka topic from Oxla

Write and run SQL against Oxla, a PostgreSQL-wire-compatible distributed columnar analytical database. Covers connecting via psql or any PostgreSQL driver (port 5432, password "oxla" by default), the full supported data-type set (INT/LONG/INT16/INT32/FLOAT/DOUBLE/CHAR/VARCHAR/STRING/TEXT/DATE/TIME/ TIMESTAMP/TIMESTAMPTZ/INTERVAL/BOOL/JSON/JSONB/BYTEA/NUMERIC/ARRAY/GEOMETRY/ GEOGRAPHY/POINT), DDL (CREATE/DROP TABLE, CREATE TABLE AS SELECT, CREATE/DROP VIEW, CREATE/DROP SCHEMA, TRUNCATE, CREATE ROLE/GRANT/REVOKE), DML (SELECT/INSERT/UPDATE/DELETE, SELECT INTO table or file), data loading (COPY FROM/TO with CSV/Parquet/ORC formats, S3 credentials), aggregate and window functions (SUM/AVG/COUNT/MIN/MAX/percentile_disc/percentile_cont/mode, ROW_NUMBER/RANK/DENSE_RANK/LAG/LEAD/FIRST_VALUE/LAST_VALUE/NTH_VALUE/NTILE/ CUME_DIST), CTEs (WITH), UNION/INTERSECT/EXCEPT (with optional ALL), PREPARE/EXECUTE, and analytic query patterns (GROUP BY, ORDER BY, LIMIT/OFFSET, multi-table joins, star-schema aggregations). Also co

Collects local and remote Redpanda diagnostics bundles and gathers broker process info using the `rpk debug` command group. Use when: collecting a debug bundle for Redpanda support or self-triage, generating a remote (cluster-wide) bundle via the Admin API, gathering local broker process or diagnostic info from the CLI, using rpk debug bundle or rpk debug remote-bundle, troubleshooting a Redpanda cluster and needing to capture logs/metrics/profiles, passing a debug bundle to the support team, or running rpk debug on Linux or Kubernetes. Also covers triaging enterprise (license-gated) features from a bundle — Tiered Storage, Cloud Topics, Iceberg Topics, Continuous Data Balancing, Shadow Linking (cross-cluster DR), Remote Read Replicas, Audit Logging, RBAC/GBAC, OIDC/OAuthBearer/ Kerberos auth, FIPS mode, Server-side Schema ID Validation, Schema Registry Authorization, and Leadership Pinning, plus checking license status and license violations. Applies to self-managed deployments only (Linux hosts and Kubernet

Manage Redpanda topics and produce/consume records from the CLI using the `rpk topic` command group. Covers creating topics with custom partition counts, replication factors, and configs; listing and describing topics (summary, configs, partitions); altering topic configs (set/delete/append/ subtract); adding partitions; trimming/deleting records; analyzing throughput and batch size; describing tiered-storage status; and deleting topics. Also covers `rpk topic produce` and `rpk topic consume` in depth including the --format percent-escape syntax, keys, headers, compression, acks, tombstones, schema-registry encoding/decoding, and all --offset forms. Also covers Redpanda enterprise topic properties (Enterprise license required): Tiered Storage (redpanda.remote.read/write/delete/recovery, redpanda.storage.mode, retention.local.target.*), Cloud Topics (redpanda.cloud_topic.enabled / redpanda.storage.mode=cloud), Iceberg Topics (redpanda.iceberg.mode/delete/partition.spec/target.lag.ms/ invalid.record.action), Re

Builds, deploys, and manages Redpanda Data Transforms — WebAssembly (Wasm) functions that run inside the broker to transform records in-flight from an input topic to one or more output topics. Covers the full lifecycle: enabling the feature, initializing a project, writing transform logic in Go (TinyGo), Rust, JavaScript, or TypeScript, building the Wasm binary, deploying with rpk transform deploy, listing and inspecting running transforms, viewing logs, pausing/resuming, and deleting transforms. Use when: writing or deploying a Redpanda data transform, in-broker Wasm transform, wiring an input topic to an output topic, filtering or transforming Kafka records inside the broker, using rpk transform init / build / deploy / list / logs / pause / resume / delete, enabling data_transforms_enabled cluster config, using the Go transform SDK, Rust transform SDK, or JavaScript/TypeScript transform SDK, from-offset reprocessing, or setting environment variables on a transform. Also covers wiring transform output topics

Guides setup and operation of the tigerbeetle_cdc Redpanda Connect input for streaming change data capture from a TigerBeetle financial transactions database into Redpanda or Kafka. Covers the cgo-enabled binary requirement, connecting to a TigerBeetle cluster (cluster_id, replica addresses), the progress_cache checkpointing model, resuming from a timestamp (timestamp_initial), the JSON change-event shape (transfer + debit/credit account snapshots), and per-event-type or per-ledger routing. Use when: capturing TigerBeetle ledger events (transfers, account updates) into Redpanda or Kafka, configuring tigerbeetle_cdc, choosing a persistent progress cache, filtering settled vs pending two-phase transfers with the event_type metadata, routing by ledger, archiving transfer events to S3, troubleshooting "component not available" (cgo builds) or duplicate events after restart, or asking whether tigerbeetle_cdc needs an Enterprise license (it does not — it is a certified community connector, unlike the other CDC inpu

Operates a Redpanda cluster from the command line using the `rpk cluster` command group — the CLI front-end to the Admin API. Covers health checks, cluster metadata, broker management, cluster configuration, partition balancing and movement, maintenance mode, client quotas, log directories, transactions, self-test benchmarks, and license management. Use when: checking cluster health or metadata, listing brokers, decommissioning or recommissioning a broker (via `rpk redpanda admin brokers` — covered here because it pairs with maintenance mode), getting or setting cluster configuration properties, balancing or moving partitions, enabling or disabling maintenance mode on a node, monitoring Kafka client connections, managing client quotas, viewing log dirs, running cluster self-tests, or managing the Redpanda license from the CLI. Also use when asked about rpk cluster health, rpk cluster info, rpk cluster config get/set/edit/import/export/lint/status, rpk cluster partitions list/balance/move/move-cancel/move-stat

Diagnoses, validates, and monitors Redpanda Connect pipelines. Use when a Redpanda Connect pipeline fails, stalls, drops messages, or won't start; when linting a config with rpk connect lint; tuning log level or format; reading Connect metrics (Prometheus, statsd, json_api) or traces (OpenTelemetry); checking /ready and /ping health endpoints; running a dry-run connection test; or diagnosing connector failures including auth errors, TLS errors, Enterprise license errors, backpressure, and checkpoint/cache issues. Covers Redpanda Connect enterprise features and their config keys: enterprise connectors including all CDC inputs (postgres_cdc, mysql_cdc, mongodb_cdc, oracledb_cdc logminer block), connector allow/deny lists, secrets management URNs, the redpanda: configuration service block, and FIPS compliance — all requiring an Enterprise license.

Operates a self-managed Redpanda broker process and node from the command line using the `rpk redpanda` command family and `rpk iotune` — node lifecycle (start/stop), production vs development mode, kernel autotuning, system checks, node configuration (redpanda.yaml), and per-node Admin API operations including broker decommission/recommission. SELF-MANAGED ONLY: these commands operate the broker process on the node itself and do not apply to Redpanda Cloud clusters. Use when: starting or stopping a Redpanda broker process, bootstrapping a node's redpanda.yaml before first start, setting node (not cluster) configuration, switching a node between production/development/recovery mode, running the autotuner or checking kernel tuning, benchmarking disk I/O with iotune, decommissioning or recommissioning a broker, monitoring decommission progress, listing brokers or a broker's partitions through the admin listener, temporarily changing a broker's log level, or printing a broker's effective configuration. Also use

Provision and manage Redpanda Cloud Dedicated clusters via the Control Plane API (https://api.redpanda.com). Covers OAuth2 client-credentials auth (Auth0, audience cloudv2-production.redpanda.cloud), creating a Network resource (cluster_type TYPE_DEDICATED, Redpanda-managed VPC, cidr_block), creating a TYPE_DEDICATED cluster (cloud_provider, region, zones, throughput_tier, connection_type, redpanda_version, tags), polling long-running Operations (STATE_IN_PROGRESS -> STATE_COMPLETED), and using the per-cluster Data Plane API URL returned by GetCluster for Topic/ACL/User/ Secret management. Use when: creating, listing, updating, or deleting Redpanda Cloud Dedicated clusters via the public API; choosing a region, zones, or throughput tier for a Dedicated cluster; configuring connectivity (CONNECTION_TYPE_PUBLIC, CONNECTION_TYPE_PRIVATE, PrivateLink on AWS/GCP/Azure, GCP Private Service Connect, or VPC/network peering via NetworkPeeringService); comparing Dedicated vs BYOC vs Serverless; scripting the full provi

Streams change data capture (CDC) from MySQL or MariaDB into Redpanda/Kafka using the mysql_cdc input in Redpanda Connect. Covers binlog (row) replication, initial snapshots, checkpoint caching, and operational tuning. Use when: capturing inserts/updates/deletes from MySQL or MariaDB into Redpanda or Kafka via Redpanda Connect's mysql_cdc input; configuring binlog replication; setting up snapshots with stream_snapshot; wiring a checkpoint_cache resource; tuning checkpoint_limit or max_parallel_snapshot_tables; using AWS RDS/Aurora with IAM auth; routing per-table CDC events to different Kafka topics; debugging a MySQL CDC pipeline that stalls, skips, or fails to resume after restart; understanding binlog_position metadata; distinguishing mysql vs mariadb flavor; landing CDC changes into Iceberg Topics (redpanda.iceberg.mode/delete/partition.spec/ target.lag.ms/invalid.record.action) or Tiered Storage (redpanda.remote.write/ read, cloud_storage_enabled) for a lakehouse; enabling server-side Schema ID Validatio

Streams change data capture (CDC) and platform events from Salesforce into Redpanda or Kafka using Redpanda Connect's salesforce_cdc input — the Salesforce Pub/Sub gRPC API (api.pubsub.salesforce.com:443), OAuth Client Credentials flow, optional REST snapshot of sObjects, and per-topic replay-ID checkpointing in a cache resource. Use when: capturing Salesforce change events (Account, Contact, Opportunity, or any CDC-enabled sObject) into Redpanda/Kafka; subscribing to the CDC firehose (/data/ChangeEvents); streaming custom or standard Platform Events (/event/Order__e, /event/LoginEventStream); configuring a Salesforce Connected App for OAuth client_credentials; enabling Change Data Capture for sObjects in Salesforce Setup; setting up a durable Connect cache_resource (e.g. Redis, Postgres, DynamoDB) for replay-ID persistence across restarts; tuning stream_snapshot, replay_preset, stream_batch_size, or snapshot_max_batch_size; understanding the topic/replay_id/operation/sobject/record_ids/event_uuid metadata em

Authenticate and manage Redpanda Cloud from the CLI using the `rpk cloud` command group. Covers login (SSO browser flow and client credentials), logout, cloud auth management (list/use/delete/token), cluster selection (rpk cloud cluster select), resource group management, BYOC plugin provisioning (install/apply/destroy/validate), and the MCP server (stdio/install/proxy) that exposes Redpanda Cloud to AI agents. Use when: logging in to Redpanda Cloud with SSO or --client-id/--client-secret; setting RPK_CLOUD_CLIENT_ID or RPK_CLOUD_CLIENT_SECRET; creating or switching an rpk profile that points at a Cloud cluster; running rpk cloud cluster select; managing resource groups in Redpanda Cloud; running rpk cloud byoc apply or rpk cloud byoc install; configuring the rpk MCP server for Claude Desktop or Claude Code; running rpk cloud mcp stdio or rpk cloud mcp install; printing the current auth token with rpk cloud auth token; listing or switching cloud auths across multiple Redpanda Cloud organizations. Also covers

Manages Redpanda authentication and authorization from the CLI using the `rpk security` command group (user, acl, role, secret subcommands). Use when: creating or deleting SASL/SCRAM users; changing user passwords; listing SASL users; granting or revoking Kafka ACLs (topics, consumer groups, cluster, transactional IDs, Schema Registry subjects); listing or deleting ACLs; creating or deleting RBAC roles (Enterprise); assigning or unassigning roles to principals; describing a role's members and ACL bindings; managing secrets for Redpanda Cloud clusters; setting up authentication on a self-hosted or cloud cluster; configuring SASL/SCRAM, SASL/PLAIN, OAUTHBEARER/OIDC (Enterprise), GSSAPI/Kerberos (Enterprise), mTLS principal mapping, or HTTP Basic/OIDC auth; configuring enterprise security features and their config keys — Audit Logging (`audit_enabled`), GBAC with OIDC `Group:` principals (`oidc_group_claim_path`), server-side Schema ID Validation (`enable_schema_id_validation`), Schema Registry Authorization (`s

Install and configure the rpk CLI — the single binary for all Redpanda operations. Covers installing rpk on macOS (Homebrew or curl/zip), Linux (amd64/arm64 curl/zip), and via Docker; creating and switching rpk profiles (rpk.yaml); the -X flag system and RPK_ environment variables for one-off overrides; connecting to self-hosted clusters and Redpanda Cloud clusters; and understanding the full command-group map. Also covers managing the Enterprise license (rpk cluster license) and operating Redpanda's Enterprise differentiators through rpk: Tiered Storage, Cloud Topics, Iceberg Topics, Continuous Data Balancing, Shadow Linking / cross-cluster disaster recovery (rpk shadow), Remote Read Replicas, Audit Logging, RBAC, OIDC/Kerberos/FIPS auth, server-side Schema ID Validation, and Leadership Pinning — with their nested config keys and topic properties. Use when: installing rpk; setting up or switching rpk profiles; configuring broker/admin/registry endpoints; adding SASL or TLS credentials to rpk; using -X flags

Operate a Redpanda cluster via its HTTP Admin API on port 9644 (base path /v1). Covers authentication (Basic, Bearer, mTLS), all major endpoint groups (brokers, partitions, cluster config, cluster health, features, licensing, transactions, cloud_storage, security/RBAC, debug, debug_bundle), and ConnectRPC endpoints added in v25.3. Also covers configuring and operating Redpanda's Enterprise features through the Admin API: Tiered Storage / shadow indexing, Cloud Topics, Iceberg Topics, Continuous Data Balancing (partition_autobalancing_mode=continuous), Shadow Linking cross-cluster disaster recovery (ShadowLinkService ConnectRPC), Remote Read Replicas, Topic Recovery / Whole Cluster Restore, Audit Logging, Role-Based Access Control (RBAC), OIDC/OAuthBearer/Kerberos authentication, FIPS mode, Server-Side Schema ID Validation, and Leadership Pinning (most require an Enterprise license). Use when: calling the Redpanda Admin API directly over HTTP, decommissioning or recommissioning brokers, reading or altering clu

Diagnoses a Redpanda broker or cluster using debug bundles, metrics endpoints, logs, CPU profiling, partition and raft health checks, and triage playbooks. Use when: a Redpanda broker is unhealthy, crashing, lagging, under-replicated, slow, or out of disk; when collecting a debug bundle for Redpanda support; when reading /public_metrics or /metrics; when triaging cluster health, partition movement, or raft recovery issues; when a broker won't start or keeps crashing; when consumer lag is growing unexpectedly; when disk pressure or a leadership imbalance is suspected; when running rpk debug bundle or rpk debug remote-bundle; when interpreting prometheus metrics from a Redpanda cluster; when debugging enterprise features and their health signals — Tiered Storage / shadow indexing, Continuous Data Balancing (partition_autobalancing_mode=continuous) and intra-broker core_balancing, Cloud Topics, Iceberg Topics (redpanda.iceberg.mode and the DLQ), Shadow Linking cross-cluster DR (rpk shadow), Remote Read Replicas,

Expert guide for Redpanda's Agentic Data Plane (ADP): Redpanda's governance infrastructure for building, running, and governing AI agents and MCP servers. ADP is its own product surface that runs on Redpanda; it provisions its own ADP environment. Use when: creating or managing ADP AI agents (managed or self-managed) via rpk ai agent or the AgentRegistryService API; configuring MCP servers on ADP (remote or managed catalog types, code mode, auth variants); setting up LLM providers or querying available models via rpk ai llm / rpk ai model or the LLMProviderService / ModelService API; working with the AI Gateway proxy (provider config, per-provider URLs, credential injection); configuring budgets, guardrails, or Cedar access-control policies via the governance APIs (BudgetService, GuardrailService, PolicyService); reading agent transcripts or spending insights via TranscriptsService or InsightsService; running rpk ai (install, upgrade, agent, auth, mcp, llm, model, oauth-client, oauth-provider, run); wiring OA

Streams change data capture from Oracle Database into Redpanda or Kafka using the oracledb_cdc input in Redpanda Connect (LogMiner-based, available since version 4.83.0). Covers ARCHIVELOG mode, supplemental logging, LogMiner privileges, SCN windows, snapshot mode, LOB handling, Oracle Wallet/SSL, pluggable database (CDB/PDB) monitoring, and checkpointing. Use when: setting up Oracle CDC with Redpanda Connect; configuring the oracledb_cdc input; enabling ARCHIVELOG mode or supplemental logging on Oracle; granting LogMiner privileges; tuning scn_window_size, backoff_interval, or mining_interval; capturing CLOBs or BLOBs (lob_enabled); configuring a transaction cache for large transactions; using Oracle Wallet for TLS/SSL; monitoring a pluggable database (pdb_name); troubleshooting missing redo logs or ORA-01291 errors; snapshotting existing rows (snapshot_mode: none/snapshot_only/snapshot_and_stream, replacing the deprecated stream_snapshot boolean); checkpointing SCN with an external cache or the built-in Ora

Streams change data capture (CDC) from PostgreSQL into Redpanda or Kafka using Redpanda Connect's postgres_cdc input — logical replication via the WAL (pgoutput), optional initial snapshot, replication slots, and publications. Use when: capturing inserts/updates/deletes from PostgreSQL into Redpanda or Kafka; configuring the postgres_cdc input (formerly pg_stream); setting up PostgreSQL logical replication (wal_level=logical); creating replication slots or publications; enabling stream_snapshot to back-fill existing rows before streaming changes; routing per-table CDC events to separate topics with Bloblang; using AWS IAM auth for RDS or Aurora PostgreSQL; tuning checkpoint_limit, heartbeat_interval, or max_parallel_snapshot_tables; understanding the lsn/operation/table/schema/commit_ts_ms/before message metadata emitted by the connector (before carries the pre-change row for updates/deletes, subject to REPLICA IDENTITY); troubleshooting slot growth or WAL accumulation; or asking about the Enterprise license

Guides setup and operation of the aws_dynamodb_cdc Redpanda Connect input for streaming change data capture from AWS DynamoDB into Redpanda or Kafka using DynamoDB Streams. Covers enabling DynamoDB Streams, IAM policy setup, checkpointing, snapshot modes, multi-table discovery, and the full pipeline YAML. Use when: capturing INSERT/MODIFY/REMOVE changes from DynamoDB into Redpanda or Kafka, configuring aws_dynamodb_cdc, enabling DynamoDB Streams, setting up the checkpoint table, using snapshot_and_cdc or snapshot_only, auto-discovering tables by tag, routing DynamoDB CDC events per table, troubleshooting DynamoDB stream retention or shard handling. Also covers the Redpanda Enterprise features the CDC destination topic and cluster can use: Iceberg Topics (redpanda.iceberg.mode/delete/partition.spec/target.lag.ms/ invalid.record.action), Tiered Storage (redpanda.remote.read/write) to outlive the 24h stream window, Remote Read Replicas, Shadow Linking (rpk shadow cross-cluster DR), server-side Schema ID Validati

Streams change data capture (CDC) from MongoDB into Redpanda or Kafka using Redpanda Connect's mongodb_cdc input — Change Streams over a replica set or sharded cluster, resume-token checkpointing, optional initial snapshot, and three document modes (update_lookup, pre_and_post_images, partial_update). Use when: capturing inserts/updates/deletes/replaces from MongoDB into Redpanda or Kafka; configuring the mongodb_cdc input; setting up MongoDB Change Streams (requires replica set or sharded cluster); enabling stream_snapshot to back-fill existing documents before streaming live changes; routing per-collection CDC events to separate Redpanda topics with Bloblang; configuring a cache resource for resume-token checkpointing; choosing between update_lookup and pre_and_post_images document modes; tuning snapshot_parallelism or snapshot_auto_bucket_sharding for Atlas environments; understanding the operation/collection/operation_time/schema message metadata emitted by the connector; resuming after restarts with resu

Streams change data capture (CDC) from Google Cloud Spanner into Redpanda or Kafka using Redpanda Connect's gcp_spanner_cdc input — Spanner change streams, partition-aware watermarked delivery, and metadata persistence. Use when: capturing INSERT/UPDATE/DELETE changes from a Google Cloud Spanner database into Redpanda or Kafka; configuring the gcp_spanner_cdc input; creating a Spanner change stream with CREATE CHANGE STREAM; setting up GCP service-account credentials or Application Default Credentials for the connector; configuring project_id, instance_id, database_id, and stream_id; using start_timestamp or end_timestamp to bound the stream window; understanding the metadata_table the connector creates in Spanner for partition watermarking; filtering mod types with allowed_mod_types (INSERT, UPDATE, DELETE); tuning heartbeat_interval or min_watermark_cache_ttl; understanding the message payload (Mod JSON with keys, new_values, old_values) and message metadata (table_name, mod_type, commit_timestamp, record_s

Stream change data capture from Microsoft SQL Server into Redpanda or Kafka using the Redpanda Connect `microsoft_sql_server_cdc` input — CDC capture instances, LSN-based change tables, initial snapshots, and LSN checkpointing. Covers every config field (connection_string, include/exclude regexp filters, stream_snapshot, max_parallel_snapshot_tables, snapshot_max_batch_size, checkpoint_cache, checkpoint_cache_table_name, checkpoint_cache_connection_string, checkpoint_cache_key, checkpoint_limit, stream_backoff_interval, batching), the emitted metadata fields (database_schema, schema, table, operation, lsn), operation types (read, insert, update_before, update_after, delete), the built-in SQL Server checkpoint cache (rpcn schema, auto-created table and stored procedure), and using an external Connect cache instead. Use when: capturing inserts/updates/deletes from Microsoft SQL Server into Redpanda or Kafka; enabling SQL Server CDC capture instances with sys.sp_cdc_enable_db and sys.sp_cdc_enable_table; configu

Diagnose and observe an Oxla distributed analytical database: query system catalog tables (system_nodes, system_queries, system_transactions, system_storage, system_execs), scrape Prometheus metrics from port 8080, change log levels at runtime via the admin gRPC service or config, monitor memory/OOM pressure, and follow troubleshooting workflows for slow queries and node health issues. Also covers debugging Oxla's external data sources: the Redpanda/Kafka ingestion path and the Apache Iceberg catalog Oxla reads (correlating stale-data or missing-row symptoms with the upstream Redpanda Iceberg Topics integration and its redpanda.iceberg.* / iceberg_* settings — an Enterprise-licensed Redpanda feature). Use when: troubleshooting Oxla, inspecting running queries or node health via system tables, scraping Prometheus metrics, changing log levels, diagnosing memory/OOM pressure, investigating slow or failed queries, checking cluster state, auditing storage connections and active transactions, or diagnosing Kafka/Re

Use rpk group to list and describe consumer groups, inspect lag and members, reset or seek consumer group offsets, delete groups, and delete committed offsets. Use when: inspecting consumer group lag and membership; resetting or seeking consumer group offsets (to earliest/latest/timestamp/specific offset or to another group's commits); stopping consumers and rewinding after a bad deploy; deleting a consumer group; deleting committed offsets for specific topic-partitions; reading CURRENT-OFFSET, LOG-END-OFFSET, and LAG columns; spotting stuck or over-lagged consumer groups; using rpk group list, describe, seek, offset-delete, or delete subcommands; verifying or repairing consumer group offsets after a Shadow Linking disaster-recovery failover (Enterprise offset-preserving replication, consumer_offset_sync_options, group_filters, offset clamping); and authorizing rpk group operations via GROUP-resource ACLs, Enterprise RBAC roles (rpk security role / --allow-role), or Enterprise GBAC OIDC Group: principals.

Manages schemas, subjects, compatibility levels, modes, and contexts in the Redpanda Schema Registry via the rpk registry CLI (alias: rpk sr). Covers registering Avro/Protobuf/JSON schemas, fetching schemas by version or ID, listing/deleting subjects and schemas, setting BACKWARD/FORWARD/FULL/NONE compatibility levels, controlling registry mode (READONLY/READWRITE/IMPORT), and using schema contexts for namespace isolation. Also covers the Enterprise-licensed registry features: Schema Registry Authorization (schema_registry_enable_authorization, registry/subject ACLs via rpk security acl with --registry-global/--registry-subject) and Server-Side Schema ID Validation (enable_schema_id_validation, redpanda.key/value.schema.id.validation, subject name strategies), plus registry authentication (HTTP Basic, OIDC/OAUTHBEARER, mTLS). Use when: registering or fetching schemas, managing schema subjects and versions, setting or checking compatibility levels, checking schema compatibility before registration, administeri

Teaches how to use the Redpanda broker through its Kafka-compatible API. Covers producing data, consuming data, consumer groups, idempotent producers, exactly-once semantics, transactions, topic creation and configuration, Kafka client compatibility, and tiered storage. Use when: producing to or consuming from Redpanda; creating or configuring topics via the Kafka API; working with consumer groups, offsets, or __consumer_offsets; enabling exactly-once semantics or transactions; choosing a Kafka client library (Java, Go franz-go, librdkafka, kafka-python-ng, Rust, Node.js KafkaJS); understanding Kafka protocol compatibility; enabling tiered/shadow storage on a topic; troubleshooting idempotent producers, acks, compression, or batching; configuring follower fetching or rack awareness; understanding how Redpanda differs from Apache Kafka (no ZooKeeper, no JVM, thread-per-core, Raft). Also covers Redpanda enterprise differentiators at the topic/broker level (license required): Iceberg Topics (redpanda.iceberg.mod

Provision and manage Redpanda Cloud Serverless clusters via the public Control Plane API (https://api.redpanda.com). Covers OAuth2 client-credentials authentication, ResourceGroup management, ServerlessRegion discovery, ServerlessCluster lifecycle (create/get/list/update/delete), the async Operation state machine (STATE_PLACING → STATE_CREATING → STATE_READY), and calling the per-cluster Data Plane API for topics, ACLs, users, secrets, and pipelines. Also covers the Enterprise differentiators configurable on Serverless via topic configs and roles: Iceberg Topics (redpanda.iceberg.mode/target.lag.ms/partition.spec/invalid.record.action/delete), Server-Side Schema ID Validation (redpanda.key|value.schema.id.validation, subject.name.strategy), Leadership Pinning (redpanda.leaders.preference), and Role-Based Access Control (/v1/roles) — all Enterprise-licensed (license included on Cloud). Also covers AWS PrivateLink for private connectivity to a Serverless cluster (ServerlessPrivateLinkService and the private_lin