dataproduct-builder-databricks

Edit an output-port ODCS file under src/output_ports/v<N>/, run the contract test against the live data, and classify any failures as breaking or non-breaking changes — with suggested fixes. Only edits output-port contracts (the spec this data product commits to); input-port contracts under src/input_ports/ are upstream's responsibility and refreshed by dataproduct-implement. Trigger when the user asks to "add/remove/change a column in the data contract", "update the data contract", or "test contract changes".

Run the Data Contract CLI (`datacontract test`) against ODCS contracts in the project to verify the live data still conforms — schema, quality rules, and freshness. Handles two kinds of contracts with different semantics: output-port contracts under `src/output_ports/**/*.odcs.yaml` (tested against this project's Databricks warehouse — "am I still producing what I promised?") and input-port contracts under `src/input_ports/*.odcs.yaml` (tested against the upstream warehouse — "is upstream still producing what I trusted?"). Trigger when the user asks to "test the data contracts", "verify the data product matches its contract", "are we still contract-conformant", "check upstream drift", or "run the contract tests".

Audit a Declarative Automation Bundle against the Entropy Data reference layout and add anything missing — Open Data Product Specification (ODPS), Open Data Contract Standard (ODCS), and the GitHub Actions publish workflow that deploys the bundle, runs the Lakeflow pipeline, runs the contract test, and publishes ODPS/ODCS back to Entropy Data. Trigger when the user asks to integrate a Databricks bundle with Entropy Data, set up Entropy Data publishing, or check whether a bundle follows the Entropy Data conventions.

Extract a small sample of rows from a Databricks output port via a non-production SQL warehouse, scrub anything classified as PII or sensitive in the data contract, and upload the scrubbed sample to Entropy Data via the entropy-data CLI. Trigger when the user asks to "upload example data", "publish sample rows for the data product", or "give consumers a preview of the data".

Initialize a brand-new Databricks data product from scratch — create databricks.yml (Declarative Automation Bundle with `dev` and `prod` targets), a serverless Lakeflow Spark Declarative Pipeline resource, a Lakeflow Job that schedules it, the src/{input_ports,transformations,output_ports/v1}/ layout, pyproject.toml, README, and .gitignore. After scaffolding, hands off to the entropy-data-publish skill to add the publishing layer (ODPS, ODCS, GitHub Actions). Trigger when the user asks to start a new data product, scaffold a new Databricks bundle, or "create a data product from scratch."

Validate, deploy, and run the Declarative Automation Bundle's Lakeflow pipeline against a chosen Databricks target. Wraps `databricks bundle validate`, `databricks bundle deploy`, and `databricks bundle run`, then polls `databricks pipelines get` for completion and surfaces any failed expectations or pipeline-level errors. Trigger when the user asks to "deploy the data product", "run the Lakeflow pipeline", "deploy and run the bundle", or "ship this to dev".

Given an Entropy Data data product URL or id, fetch its data contracts (output port ODCS files written next to the Python under src/output_ports/v<N>/, input port ODCS files cached next to their Spark reader under src/input_ports/), translate the schema into Lakeflow @dp.materialized_view / @dp.table Python pipelines, and ensure the project has the publishing layer (ODPS, GitHub Actions). Trigger when the user asks to "implement the data product <url>", "build the Lakeflow pipeline for this data product", or "scaffold output-port tables from a data contract".

Sync approved Entropy Data access agreements to Unity Catalog grants (internal consumers) or Delta Shares (external consumers). Reads `entropy-data access list --provider-dataproduct <id>`, filters to active agreements, and applies `GRANT SELECT` on the output port's table or creates a Delta Share with that table added. Trigger when the user asks to "grant access to consumers", "apply UC grants for approved access", "set up Delta Sharing for this product", or "sync access agreements to Databricks".

List the teams configured in Entropy Data so the user can pick one as the owner (`team.name` in ODPS) of a data product. Useful when initializing a new data product, integrating an existing Databricks bundle, or just exploring who owns what. Trigger when the user asks "what teams are there", "who can own this data product", "list teams in Entropy Data", or when another skill needs to fill in `TEAM_NAME` and the user does not already know it.