菜单
Build data ingestion pipelines with dlt (data load tool) -- extract from APIs, databases, and files, then load to any destination.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
| name | dlt |
| description | Build data ingestion pipelines with dlt (data load tool) -- extract from APIs, databases, and files, then load to any destination. |
| metadata | {"openclaw":{"emoji":"🔄","requires":{"bins":["dlt"]},"tags":["ingestion","dlt","etl","elt","pipeline","data"]}} |
You help build data ingestion pipelines using dlt.
Use this when the user wants to extract data from APIs, databases, or files and load it into a warehouse or lakehouse.
dlt init <source_name> <destination_name>
Example:
dlt init sql_database duckdb
python <pipeline_script>.py
dlt pipeline <pipeline_name> info
dlt pipeline <pipeline_name> show
import dlt
@dlt.source
def my_api_source(api_key=dlt.secrets.value):
@dlt.resource(write_disposition="replace")
def customers():
response = requests.get("https://api.example.com/customers",
headers={"Authorization": f"Bearer {api_key}"})
yield response.json()
return customers
pipeline = dlt.pipeline(
pipeline_name="my_api",
destination="duckdb",
dataset_name="raw",
)
load_info = pipeline.run(my_api_source())
print(load_info)
@dlt.resource(write_disposition="merge", primary_key="id")
def orders(updated_at=dlt.sources.incremental("updated_at")):
params = {"since": updated_at.last_value}
response = requests.get("https://api.example.com/orders", params=params)
yield response.json()
from dlt.sources.sql_database import sql_database
source = sql_database(
credentials="postgresql://user:pass@host:5432/db",
schema="public",
table_names=["customers", "orders"],
)
pipeline = dlt.pipeline(destination="bigquery", dataset_name="raw")
pipeline.run(source)
dlt.secrets and dlt.config for credentials -- never hardcodewrite_disposition carefully: replace, append, or mergeprimary_key for merge operationsdlt.sources.incremental for efficient incremental loadsManage Azure cloud resources -- resource groups, storage, databases, functions, and data services using the az CLI.
Work with Databricks -- run SQL via curl, manage clusters, warehouses, Unity Catalog, jobs, and pipelines.
Query and manage Google BigQuery datasets -- run SQL, list tables, inspect schemas, load data, and manage partitioning.
Query and manage PostgreSQL databases -- run SQL, inspect schemas, manage tables, and perform database administration.
Query and manage Snowflake data warehouses -- run SQL, manage warehouses, inspect schemas, and control access.
Generate Airflow DAGs from pipeline specifications using project templates.