DESKILL

Use when working on any data engineering project, pipeline, ETL/ELT system, data platform, or data warehouse — from business problem to production-ready pipeline. Covers planning, source contracts, architecture, schema design, ingestion, transformation, testing, data quality, orchestration, serving, CI/CD, and documentation. Skip for pure data-analysis questions with no pipeline work.

Create a reproducible local development environment and produce `docker-compose.yml`, `.env.template`, and `requirements.txt`. Use this skill when the user asks 'how do I set up my environment', 'write a docker-compose for [tools]', 'configure my dev environment', 'what do I need to install', 'how do I containerize my pipeline', or has committed to a tool stack and needs it running locally.

Design the data pipeline architecture and choose exactly one tool per category. Use this skill when the user asks 'what architecture should I use', 'ETL or ELT?', 'which tools should I pick', 'design my data platform', 'how should I structure my pipeline', 'medallion or lakehouse?', 'Airflow or Prefect?', or has source contracts ready and needs to commit to a stack before writing code.

Validate actual pipeline output against source data contracts to detect schema drift, SLA violations, and field-level deviations. Use when the user asks 'validate my contracts', 'check schema drift', 'verify data matches contracts', 'contract compliance check', 'has my source schema changed', or 'detect breaking changes in my API'.

Build the orchestration DAG that wires ingestion, transformation, and quality tasks into a single scheduled workflow. Use when the user says 'build my DAG', 'orchestrate my pipeline', 'schedule my pipeline', 'connect all tasks', 'write Airflow DAG', 'set up Prefect/Dagster flow', or has individual tasks ready to wire into an end-to-end pipeline.

Implement runtime data quality monitoring on actual pipeline output — freshness, volume, schema drift, and distribution checks. Use when the user asks 'check my data quality', 'set up DQ monitoring', 'freshness checks', 'volume anomaly detection', 'schema drift alerts', or 'monitor my pipeline data'. For testing transformation logic at code-change time, use /test.

Implement the Bronze/raw ingestion layer. Trigger: 'write an ingestion script', 'fetch data from API', 'load raw data', 'implement Bronze layer', 'connect to [source]', 'how do I ingest from [API/database/file]', 'write a data loader'. Also use when existing ingestion code lacks retry logic, metadata tags, or validation.

Define the business problem, persona, and analytical questions for a data engineering project before choosing tools or writing code. Use when the user says 'I want to build a pipeline for...', 'where do I start', 'what should I build first', or jumps straight to tools without a clear problem statement.

Design the data warehouse schema — Fact tables, Dimension tables, grain definitions, and relationships — and produce `docs/dw_schema.md`. Use this skill when the user asks about DW schema, star schema, snowflake schema, galaxy schema, dimensional modeling, 'what tables should I create', 'how to model data for analytics', 'Fact vs Dim', or already has analytical questions and needs a table design.

Build the serving layer so stakeholders can consume the analytics output from the pipeline. Use when the user says 'build a dashboard', 'create a serving layer', 'visualize my data', 'expose my Gold tables', 'build a BI view', 'make a Streamlit app', 'create an API for my data', 'show my analytical results', or has Gold tables and needs to answer the business questions in a consumable format.

Evaluate data sources and produce machine-verifiable source contracts before writing ingestion code. Use when the user asks 'what data sources should I use', 'help me write a data contract', 'is this API reliable enough', or wants to start coding ingestion before documenting sources.

Write and run a data pipeline test suite covering schema and transformation logic. Trigger: 'write tests for my pipeline', 'test my transformations', 'add dbt tests', 'unit test my SQL', 'write schema tests', 'test my data models'. Note: this skill validates LOGIC at code-change time — for runtime data quality use /dq.

Implement Silver (cleaned) and Gold (analytics-ready) data models. Trigger: 'transform my data', 'write dbt models', 'write Spark jobs', 'clean raw data', 'build Silver layer', 'build Gold layer', 'implement transformation logic', 'deduplicate data', 'aggregate to grain', or when the user has Bronze data and needs analytics-ready tables.

Set up CI so the test suite runs automatically on every code change. Use when the user says 'set up CI', 'automate my tests', 'GitHub Actions for data pipeline', 'make tests run on PR', 'continuous integration', 'set up pre-commit hooks', or has a working test suite and wants it to run automatically without manual intervention.

Write project documentation including README, data lineage, and cost analysis for a data engineering project. Use when the user says 'write my README', 'document my pipeline', 'generate data lineage', 'what would this cost in production', 'write project docs', 'prepare for portfolio review', 'write a handoff document', or has a working pipeline ready to document for sharing, portfolio, or team handoff.