clickzetta-zettapark

name

clickzetta-zettapark

description

Use the ZettaPark Python library to work with ClickZetta Lakehouse data through a pandas-like DataFrame API that translates Python transformations into distributed SQL — no manual SQL needed. Covers Session creation, DataFrame construction and transformation (filter/select/join/groupBy), result collection (collect/to_pandas/show), writing tables (save_as_table), file operations (PUT/GET), and executing SQL. Trigger when the user says: "ZettaPark", "zettapark", "DataFrame API", "Python Lakehouse", "save_as_table", "session.table", "session.sql", "collect()", "to_pandas", "Python data engineering", "Python write to Lakehouse", "Python read from Lakehouse", "clickzetta_zettapark_python", "Python ETL", "Python ML on Lakehouse", "feature engineering Python", "Python DataFrame Lakehouse". Keywords: ZettaPark, DataFrame, pandas-like, Python, SQL translation, distributed compute

ClickZetta ZettaPark

ZettaPark is ClickZetta Lakehouse's Python DataFrame framework. It translates Python operations into SQL for distributed execution in Lakehouse, giving you a pandas-like development experience without writing SQL manually. Use it when you need Python logic (ML, complex transformations, file processing) that operates on Lakehouse data at scale.

See references/zettapark-api.md for the complete API reference, all transformation methods, and worked examples (ETL, feature engineering, file import).

When the user is developing a Studio Python task (not local script), read references/studio-task-pattern.md — covers Studio session creation (get_active_lakehouse_engine), save-script deployment, task parameters, file I/O in Volume, watermark patterns, and common pitfalls.

When the user is migrating PySpark DataFrame code (spark.read, withColumn, groupBy, saveAsTable, UDFs, window functions), read references/migration-spark-dataframe.md — method name mapping (camelCase → snake_case), read/write differences, F functions, window API, complete side-by-side example.

When the user is migrating Snowpark (Snowflake Python) code (session.table, group_by, call_udf, @udf, Window.partition_by), read references/migration-snowpark.md — both use snake_case but differ in session config, function names (IFF→IF, flatten→explode), UDF patterns, and Snowflake SQL inside F.expr().

Installation

⚠️ Python version: Python 3.12 recommended (minimum 3.10; 3.9 and below not supported)

python3.12 -m venv .venv && source .venv/bin/activate
pip install clickzetta_zettapark_python

Create Session

from clickzetta.zettapark.session import Session

session = Session.builder.configs({
    "username": "your_username",
    "password": "your_password",
    "service": "cn-shanghai-alicloud.api.clickzetta.com",
    "instance": "your_instance_id",
    "workspace": "your_workspace",
    "schema": "public",
    "vcluster": "default",
}).create()

session.sql("SELECT current_user(), current_workspace()").show()

Core Pattern

from clickzetta.zettapark import functions as F

# Read → Transform → Write
result = (
    session.table("bronze.raw_orders")
    .filter(F.col("amount") > 0)
    .with_column("tax", F.col("amount") * 0.1)
    .group_by("category")
    .agg(F.sum("amount").as_("total"))
)

result.write.save_as_table("silver.orders_summary", mode="overwrite")
session.close()

Key methods: filter / select / with_column / join / group_by / agg / sort / limit

Collect results: show() (preview) · collect() (Row list) · to_pandas() (small data only) · count()

Troubleshooting

Problem	Cause	Solution
collect() timeout	Data too large or cluster too small	Increase hints.sdk.job.timeout, or test with limit() first
to_pandas() OOM	All data pulled to local memory	Aggregate/filter before converting, or process in batches
Column name conflict after JOIN	Both tables have same-named column	Use df_left["col"] to specify source explicitly
save_as_table error	Table exists with incompatible mode	Use mode="overwrite" or mode="append"