DKSang

Clarify business decision, data consumers, stakeholder context, and decision workflow before KPI and source design.

Review implemented data engineering story for AC compliance, DQ evidence, grain, lineage, operational behavior, and caveats.

Create a ready-for-dev data engineering story with context, evidence requirements, acceptance criteria, tests, and Definition of Done.

Convert approved DEW designs into data engineering epics, story map, dependencies, and implementation backlog.

Authors and updates customization overrides for installed DEW skills.

Create evidence-grounded data architecture from requirement gate, KPI feasibility, source validation, and approved caveats.

Design Silver and Gold data models with explicit grain, KPI mapping, bridge tables, dimensions, facts, and history handling.

Create, update, or validate a data product PRD with consumers, KPI hypotheses, grain, freshness, trust expectation, known limitations, and evidence requirements.

Use when designing the global data architecture, including storage, compute, integration patterns, layer strategy, environment promotion, security posture, observability, governance, cost, operational burden, and decision reversibility.

Use when creating the Bronze Layer Specification for raw/near-raw storage, source fidelity, ingestion metadata, schema drift, replay, quarantine, retention, access control, and lineage before Silver design.

Use when designing CI/CD, testing strategy, release gates, deployment promotion, rollback, release evidence, and environment validation for data products.

Use when designing cost, performance, scalability, storage, compute, query, serving, monitoring, and FinOps optimization strategies for data products.

Use when creating the Data Contract Specification for source feeds, Silver datasets, Gold datasets, serving outputs, dashboards, semantic models, APIs, ML/AI datasets, external exports, or downstream systems that need stable structure, meaning, freshness, quality, access, ownership, lineage, and change behavior.

Use when defining or designing data quality dimensions, rules, thresholds, severity, gates, failure handling, ownership, and validation expectations across Bronze, Silver, Gold, contracts, transformations, and serving outputs.

Use when creating the Gold Layer Specification for curated, consumer-ready marts, metric-ready datasets, reporting outputs, feature datasets, API/read-model outputs, or product-aligned datasets derived from trusted Silver data.

Use when creating the Ingestion Specification for any data engineering project, including source-specific ingestion pattern, frequency, access method, checkpointing, idempotency, replay, schema drift, security, error handling, landing expectations, and ingestion observability.

Use when working on any data engineering project, pipeline, ETL/ELT system, data platform, or data warehouse — from business problem to production-ready pipeline. Covers planning, source contracts, architecture, schema design, ingestion, transformation, testing, data quality, orchestration, serving, CI/CD, and documentation. Skip for pure data-analysis questions with no pipeline work.

Create a reproducible local development environment and produce `docker-compose.yml`, `.env.template`, and `requirements.txt`. Use this skill when the user asks 'how do I set up my environment', 'write a docker-compose for [tools]', 'configure my dev environment', 'what do I need to install', 'how do I containerize my pipeline', or has committed to a tool stack and needs it running locally.

Design the data pipeline architecture and choose exactly one tool per category. Use this skill when the user asks 'what architecture should I use', 'ETL or ELT?', 'which tools should I pick', 'design my data platform', 'how should I structure my pipeline', 'medallion or lakehouse?', 'Airflow or Prefect?', or has source contracts ready and needs to commit to a stack before writing code.

Validate actual pipeline output against source data contracts to detect schema drift, SLA violations, and field-level deviations. Use when the user asks 'validate my contracts', 'check schema drift', 'verify data matches contracts', 'contract compliance check', 'has my source schema changed', or 'detect breaking changes in my API'.

Build the orchestration DAG that wires ingestion, transformation, and quality tasks into a single scheduled workflow. Use when the user says 'build my DAG', 'orchestrate my pipeline', 'schedule my pipeline', 'connect all tasks', 'write Airflow DAG', 'set up Prefect/Dagster flow', or has individual tasks ready to wire into an end-to-end pipeline.

Implement runtime data quality monitoring on actual pipeline output — freshness, volume, schema drift, and distribution checks. Use when the user asks 'check my data quality', 'set up DQ monitoring', 'freshness checks', 'volume anomaly detection', 'schema drift alerts', or 'monitor my pipeline data'. For testing transformation logic at code-change time, use /test.

Implement the Bronze/raw ingestion layer. Trigger: 'write an ingestion script', 'fetch data from API', 'load raw data', 'implement Bronze layer', 'connect to [source]', 'how do I ingest from [API/database/file]', 'write a data loader'. Also use when existing ingestion code lacks retry logic, metadata tags, or validation.

Define the business problem, persona, and analytical questions for a data engineering project before choosing tools or writing code. Use when the user says 'I want to build a pipeline for...', 'where do I start', 'what should I build first', or jumps straight to tools without a clear problem statement.

Skill hỗ trợ đóng gói và chuẩn bị deploy: tạo manifest item, substitute env vars, và kiểm tra cấu hình trước khi deploy lên Fabric. Dùng khi cần chuẩn bị artifact để push hoặc validate Fabric deployment configs.

Skill hỗ trợ thu thập dữ liệu (ingestion): tạo sample, chạy fetch scripts, xuất mẫu sang OneLake. Dùng khi cần viết/khắc phục ingestion scripts, tạo sample dữ liệu để test local, hoặc xuất sample lên OneLake.

Skill hỗ trợ kiểm tra observability: smoke test notebooks, kiểm tra recent runs, và tạo báo cáo run history. Dùng khi cần xác minh các notebook đã chạy thành công hoặc tạo tổng hợp lỗi nhanh.

Skill hỗ trợ kiểm tra tính toàn vẹn dữ liệu: chạy dbt tests local như proxy cho Provero, kiểm tra schema, và tạo báo cáo nhanh. Dùng khi cần validate schema, chạy dbt tests, hoặc tái tạo rule-based checks local trước khi push.