القائمة
Extract data from database tables to CSV/Parquet files
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
استنادا إلى تصنيف SOC المهني
| name | extract-data |
| description | Extract data from database tables to CSV/Parquet files |
Extracts data from database tables into local files (CSV, Parquet). Supports full and incremental extraction, parallel processing, and schema/table filtering.
starlake extract-data [options]
--config <value>: Extract configuration name (required): references a file in metadata/extract/--outputDir <value>: Where to output data files (required)--limit <value>: Limit number of records extracted per table--numPartitions <value>: Parallelism level for partitioned table extraction--parallelism <value>: Parallelism level of the overall extraction process--ignoreExtractionFailure: Continue extraction even if individual tables fail--clean: Clean all files for a table before extracting it--incremental: Export only new data since last extraction (uses timestamp tracking)--ifExtractedBefore <value>: Only extract if last extraction was before this datetime--includeSchemas <value>: Comma-separated list of schemas/domains to include--excludeSchemas <value>: Comma-separated list of schemas/domains to exclude--includeTables <value>: Comma-separated list of tables to include--excludeTables <value>: Comma-separated list of tables to exclude--reportFormat <value>: Report output format: console, json, or html# metadata/extract/externals.sl.yml
version: 1
extract:
connectionRef: "source_postgres"
jdbcSchemas:
- schema: "sales"
tables:
- name: "orders"
fullExport: false
partitionColumn: "id"
numPartitions: 4
timestamp: "updated_at"
fetchSize: 1000
- name: "customers"
fullExport: true
- name: "*" # All remaining tables
starlake extract-data --config externals --outputDir /tmp/output
Extract only rows added/updated since last extraction:
starlake extract-data --config externals --outputDir /tmp/output --incremental
starlake extract-data --config externals --outputDir /tmp/output --limit 1000
starlake extract-data --config externals --outputDir /tmp/output --includeSchemas sales,hr
starlake extract-data --config externals --outputDir /tmp/output --excludeTables audit_log,temp_data
starlake extract-data --config externals --outputDir /tmp/output --parallelism 8 --ignoreExtractionFailure
starlake extract-data --config externals --outputDir /tmp/output --clean
Manage GizmoSQL processes: start, stop, list, and stop-all DuckLake-backed SQL servers
Create or modify database connections in application.sl.yml
Manage Quack DuckDB query servers exposing DuckLake over a thin remote protocol — serve (foreground), start/stop/list/stop-all (background)
Automatically infer schemas and load data from the incoming directory
Data quality expectations syntax, built-in macros, and validation patterns
Apply Row Level Security (RLS) and Column Level Security (CLS) policies