clickzetta-spark-flink-connector

스타7

포크3

업데이트2026년 6월 18일 07:29

Write data to ClickZetta Lakehouse using the Spark Connector or Flink Write Connector. Covers Spark DataFrame read/write configuration (Maven dependencies, connection parameters, read/write code), Flink Table API writes (CDC mode igs-dynamic-table, append-only mode igs-dynamic-table-append-only), checkpoint configuration, buffer/flush tuning, and key constraints such as primary-key table limitations. Trigger when the user says "Spark Connector", "Flink Connector", "Spark writes to Lakehouse", "Flink writes to Lakehouse", "spark-clickzetta", "igs-flink-connector", "Spark DataFrame write", "Flink CDC write", "Flink sink", or "spark.read.format clickzetta". Keywords: Spark, Flink, DataFrame, connector, read, write, CDC, igs-dynamic-table

설치

Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.

출처

clickzetta

clickzetta/clickzetta-skills

GitHub 저장소 열기 Creator 저장소 보기

다운로드

ClickZetta Spark & Flink Connector

Read references/spark.md for the Spark Connector and references/flink.md for the Flink Write Connector.

Key Constraints (Required Reading)

Constraint	Spark Connector	Flink Connector
Primary-key table writes	Not supported	Supported in `igs-dynamic-table` mode
Partial-column writes	Not supported; all columns must be written	Supported
CDC (UPDATE/DELETE)	Not supported; append only	Supported in `igs-dynamic-table` mode
Spark version	3.4.0+	N/A
Flink version	N/A	1.14, 1.15, 1.17, 1.18

Spark Connector Quick Example

// Write
df.write.format("clickzetta")
  .option("endpoint", "your_instance.cn-shanghai-alicloud.api.clickzetta.com")
  .option("username", sys.env("CZ_USERNAME"))
  .option("password", sys.env("CZ_PASSWORD"))
  .option("workspace", "your_workspace")
  .option("virtualCluster", "default")
  .option("schema", "public")
  .option("table", "orders")
  .mode("append")
  .save()

// Read
val df = spark.read.format("clickzetta")
  .option("endpoint", "your_instance.cn-shanghai-alicloud.api.clickzetta.com")
  .option("username", sys.env("CZ_USERNAME"))
  .option("password", sys.env("CZ_PASSWORD"))
  .option("workspace", "your_workspace")
  .option("virtualCluster", "default")
  .option("schema", "public")
  .option("table", "orders")
  .load()

Flink Connector Quick Example

-- CDC mode (supports INSERT/UPDATE/DELETE; the target table must have a primary key)
CREATE TABLE lakehouse_sink (
    order_id   INT,
    status     STRING,
    amount     DOUBLE,
    PRIMARY KEY (order_id) NOT ENFORCED
) WITH (
    'connector'       = 'igs-dynamic-table',
    'curl'            = 'jdbc:clickzetta://your_instance.cn-shanghai-alicloud.api.clickzetta.com/default?username=user&password=***&schema=public',
    'schema-name'     = 'public',
    'table-name'      = 'orders',
    'sink.parallelism' = '1',
    'properties'      = 'authentication:true'
);

INSERT INTO lakehouse_sink SELECT order_id, status, amount FROM source_table;

Selection Guide

Scenario	Recommended option
Spark ETL batch writes to a non-primary-key table	Spark Connector
Flink real-time stream writes to a non-primary-key table	Flink `igs-dynamic-table-append-only`
Flink CDC synchronization to a primary-key table, including UPDATE/DELETE	Flink `igs-dynamic-table`
High-frequency real-time writes from a Java application	Java SDK RealtimeStream

이 저장소의 다른 Skills

같은 저장소

clickzetta-analytics-agent

clickzetta/clickzetta-skills

Complete guide for managing ClickZetta Analytics Agent domains via cz-cli: domain CRUD, table registration, column semantics (types/descriptions/aliases), metrics (simple & compound), Answer Builders, Knowledge Bases, and QA testing. Wizard-driven workflows with intent classification, routing tables, and troubleshooting references. Trigger when the user says: "analytics agent", "分析域", "语义层", "指标", "answer builder", "知识库", "多别名", "column semantics", "domain prompt", "metric create", "AB create", "session run", "domain table add", "semantics set", "knowledge space". Keywords: analytics agent, domain, semantics, metric, answer builder, knowledge base, alias, cz-cli, data analysis, AI agent, semantic layer

2026-07-317

clickzetta-semantic-view

clickzetta/clickzetta-skills

Create, query, and manage ClickZetta Lakehouse Semantic Views — schema-level logical models that encapsulate multi-table JOINs and aggregations into a business-friendly layer of logical tables, dimensions, metrics, and facts. Query with the semantic_view() function without writing JOINs or GROUP BY manually. Triggered when user says "create semantic view", "semantic view", "semantic layer", "define metrics", "define dimensions", "unified metric definitions", "business semantic model", "semantic_view()", "CREATE OR REPLACE SEMANTIC VIEW", "FACTS", "PRIVATE metric", "conditional metric", "window metric", "SHOW SEMANTIC VIEWS", "GRANT SELECT ON SEMANTIC VIEW". Keywords: semantic view, dimension, metric, fact, logical model, unified metrics, semantic layer, grain, chasm trap, FILTER metric

2026-07-317

clickzetta-data-science

clickzetta/clickzetta-skills

End-to-end data science workflow guide for ClickZetta Lakehouse, covering environment setup, data discovery, feature engineering (SQL + ZettaPark), and model inference deployment. Details: Python 3.10+/Jupyter/ZettaPark setup, project structure, data quality assessment, and inference (BITMAP profiling, UDF batch inference, vector search). Trigger when the user wants to do data science, ML, or analytical work using ClickZetta Lakehouse — connecting Jupyter to Lakehouse, doing EDA, building features, running ML inference, user profiling, audience segmentation, or batch scoring. Keywords: data science, ML, ZettaPark, Jupyter, feature engineering, EDA, profiling, inference

2026-07-317

clickzetta-zettapark

clickzetta/clickzetta-skills

Use the ZettaPark Python library to work with ClickZetta Lakehouse data through a pandas-like DataFrame API that translates Python transformations into distributed SQL — no manual SQL needed. Covers Session creation, DataFrame construction and transformation (filter/select/join/groupBy), result collection (collect/to_pandas/show), writing tables (save_as_table), file operations (PUT/GET), and executing SQL. Trigger when the user says: "ZettaPark", "zettapark", "DataFrame API", "Python Lakehouse", "save_as_table", "session.table", "session.sql", "collect()", "to_pandas", "Python data engineering", "Python write to Lakehouse", "Python read from Lakehouse", "clickzetta_zettapark_python", "Python ETL", "Python ML on Lakehouse", "feature engineering Python", "Python DataFrame Lakehouse". Keywords: ZettaPark, DataFrame, pandas-like, Python, SQL translation, distributed compute

2026-07-317

singsight-install

clickzetta/clickzetta-skills

Guide users to connect their AI coding agents (Claude Code, Hermes Agent, OpenClaw, Opencode) to the Singsight AI Agent observability platform. Automatically reports every LLM call, tool execution, token usage, and session via OpenTelemetry. Triggered when the user says "connect Singsight", "install Singsight", "configure Singsight", "Agent observability", "OTel integration", "report token usage", "monitor AI Agent", "singsight install". Keywords: singsight, install, observability, opentelemetry, otel, agent telemetry, claude code, hermes, openclaw, opencode

2026-07-027

clickzetta-dbt-modeling

clickzetta/clickzetta-skills

dbt-clickzetta data modeling wizard: discovers Lakehouse data sources, infers modeling strategies, generates sources.yml and model files, and runs the full test suite. Presents choices rather than blank forms — explores data first, then proposes a justified plan for user confirmation. Trigger when the user wants to turn raw Lakehouse data into usable analytical tables — mentions dbt (dbt modeling, dbt model, sources.yml, incremental, dbt run, dbt test), wants ETL transformation, dimension/fact tables, or asks "how do I turn these tables into a queryable dataset". Keywords: dbt, dbt modeling, sources.yml, incremental, dbt run, dbt test, dimension table, fact table, ETL

2026-07-017

name

clickzetta-spark-flink-connector

description

ClickZetta Spark & Flink Connector

Read references/spark.md for the Spark Connector and references/flink.md for the Flink Write Connector.

Key Constraints (Required Reading)

Constraint	Spark Connector	Flink Connector
Primary-key table writes	Not supported	Supported in `igs-dynamic-table` mode
Partial-column writes	Not supported; all columns must be written	Supported
CDC (UPDATE/DELETE)	Not supported; append only	Supported in `igs-dynamic-table` mode
Spark version	3.4.0+	N/A
Flink version	N/A	1.14, 1.15, 1.17, 1.18

Spark Connector Quick Example

// Write
df.write.format("clickzetta")
  .option("endpoint", "your_instance.cn-shanghai-alicloud.api.clickzetta.com")
  .option("username", sys.env("CZ_USERNAME"))
  .option("password", sys.env("CZ_PASSWORD"))
  .option("workspace", "your_workspace")
  .option("virtualCluster", "default")
  .option("schema", "public")
  .option("table", "orders")
  .mode("append")
  .save()

// Read
val df = spark.read.format("clickzetta")
  .option("endpoint", "your_instance.cn-shanghai-alicloud.api.clickzetta.com")
  .option("username", sys.env("CZ_USERNAME"))
  .option("password", sys.env("CZ_PASSWORD"))
  .option("workspace", "your_workspace")
  .option("virtualCluster", "default")
  .option("schema", "public")
  .option("table", "orders")
  .load()

Flink Connector Quick Example

-- CDC mode (supports INSERT/UPDATE/DELETE; the target table must have a primary key)
CREATE TABLE lakehouse_sink (
    order_id   INT,
    status     STRING,
    amount     DOUBLE,
    PRIMARY KEY (order_id) NOT ENFORCED
) WITH (
    'connector'       = 'igs-dynamic-table',
    'curl'            = 'jdbc:clickzetta://your_instance.cn-shanghai-alicloud.api.clickzetta.com/default?username=user&password=***&schema=public',
    'schema-name'     = 'public',
    'table-name'      = 'orders',
    'sink.parallelism' = '1',
    'properties'      = 'authentication:true'
);

INSERT INTO lakehouse_sink SELECT order_id, status, amount FROM source_table;

Selection Guide

Scenario	Recommended option
Spark ETL batch writes to a non-primary-key table	Spark Connector
Flink real-time stream writes to a non-primary-key table	Flink `igs-dynamic-table-append-only`
Flink CDC synchronization to a primary-key table, including UPDATE/DELETE	Flink `igs-dynamic-table`
High-frequency real-time writes from a Java application	Java SDK RealtimeStream