data-engineering-skills

Docker Compose V2 and Compose Specification expertise for writing correct compose.yaml and docker-compose.yml files. Use when the user mentions Docker Compose, compose.yaml, docker-compose.yml, docker compose CLI, multi-container apps, profiles, healthchecks, depends_on, networks, volumes, secrets, build contexts, or avoiding deprecated V1 patterns.

Apache Flink stream processing expertise for stateful computations over bounded and unbounded streams. Use when the user mentions Flink, Flink SQL, Table API, DataStream API, CDC, Kafka-to-Flink pipelines, checkpoints, savepoints, watermarks, windows, Flink Kubernetes Operator, or Flink integrations with Iceberg, Paimon, or Fluss.

Apache Fluss Incubating streaming storage expertise for real-time analytics, streamhouse and lakehouse architectures. Use when the user mentions Fluss, streaming storage, columnar streams, sub-second ingestion, tiered storage, Flink Delta Join, Paimon/Iceberg/Lance tiering, Spark access to streams, or Fluss table design.

Apache Iceberg table format expertise for data lakehouses. Use when the user mentions Iceberg, lakehouse, table format v2 or v3, deletion vectors, partition evolution, time travel, snapshot isolation, Polaris, Nessie, REST catalogs, metadata.json, manifest lists, snapshots, Spark, Flink, Trino, Athena, Snowflake, or table maintenance.

Apache Iggy Incubating Rust-native message streaming platform expertise. Use when the user mentions Iggy, event streaming, Kafka/NATS alternatives, Iggy CLI or SDKs, Python apache-iggy, Docker apache/iggy images, Iggy MCP, QUIC/TCP/HTTP/WebSocket transports, streams, topics, partitions, retention, or consumer groups.

Lance columnar data format and LanceDB expertise for ML/AI workloads, vector search, embeddings, multimodal datasets, and .lance files. Use when the user mentions Lance, LanceDB, pylance, vector indexes, ANN search, IVF_PQ, IVF_HNSW_FLAT, HNSW, full-text search, dataset versioning, or migrating ML data from Parquet.

Apache Paimon streaming lake format expertise for real-time ingestion and lakehouse architectures. Use when the user mentions Paimon, streaming lakehouse, primary-key tables, changelog, Flink CDC into a lake, Paimon Materialized Tables, PyPaimon, deletion vectors, lookup joins, buckets, compaction, or Spark/Flink Paimon tables.