create-dq-rules

Stars4

Forks2

UpdatedMay 18, 2026 at 14:11

Reads Snowflake Metadata to construct a pipeline with DQ Rules

Installation

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

Source

orchestra-hq

orchestra-hq/orchestra-blueprints

View GitHub Repository View Creator Repositories

Download

Related occupationsSOC

Based on SOC occupation classification

Database ArchitectsComputer and Mathematical Occupations·SOC 15-1243

SKILL.md

readonly

name	create-dq-rules
description	Reads Snowflake Metadata to construct a pipeline with DQ Rules

Use the Snowflake credentials which are formatted dlt-style in environment variables to connect to snowflake and get the lists of databases, schema and tables
Query a subset of data for 3 tables
based on these, think of some SQL Queries you can run as dbt-style data quality rules, checking for things like nulls, irregular values, duplicates and so on
Use these to construct an Orchestra pipeline as specified below. Use the Orchestra-CLI to validate the pipeline. You should only be changing the SQL Statement field in the SNOWFLAKE_RUN_TEST Task and the matrix vlaues i.e. to parameterise the test

Example pipeline:

version: v1
name: 'Snowflake Data Quality Tests #snowflake #dataquality'
pipeline:
  3e058349-56be-4175-b835-e877e5ce12db:
    tasks:
      cf917cfd-7a64-4e88-83b0-5217409d92a8:
        integration: SNOWFLAKE
        integration_job: SNOWFLAKE_RUN_TEST
        parameters:
          statement: 'select * from BIGQUERY_SAMPLE.PUBLIC.${{ MATRIX.dq_inputs[''table'']
            }}

            where ${{ MATRIX.dq_inputs[''column''] }} is null

            limit 100'
          error_threshold_expression: '> 0'
          warn_threshold_expression: '> 0'
        depends_on: []
        name: Not Null test
        connection: ${{ ENV.SNOWFLAKE_CONNECTION }}
    depends_on:
    - 5c6e2fd1-7782-4a79-9850-0daf332a6f79
    name: ''
    matrix:
      inputs:
        dq_inputs:
        - table: issues
          column: _airbyte_raw_id
        - table: USERS
          column: COUNTRY
schedule:
- name: Daily at 8am
  cron: 0 8 ? * FRI *
  timezone: Europe/London
webhook:
  enabled: false
inputs:
  tablename:
    type: string
    default: hubspot_contacts_raw
alerts:
- name: Failures
  statuses:
  - FAILED
  destinations:
  - integration: SLACK
    destination: alert-testing
- name: Slack
  statuses:
  - FAILED
  destinations:
  - integration: SLACK
    destination: alert-testing

More from this repository

same repository

dlt-source-via-orchestra

orchestra-hq/orchestra-blueprints

Build a new dlt source that lands API data into MotherDuck, then ship it through Orchestra. Use this when the user wants to "pull X from <API> into MotherDuck" or "add a new dlt source and run it on Orchestra". Walks the full lifecycle on a feature branch — author the source, register an Orchestra pipeline that invokes it, trigger the pipeline against that branch, and poll until done.

2026-05-184

publish-motherduck-dive

orchestra-hq/orchestra-blueprints

Publish or update a MotherDuck Dive by inlining a local .tsx file into a `MD_CREATE_DIVE` / `MD_UPDATE_DIVE` SQL call sent through the MotherDuck MCP (`mcp__MotherDuck__query_rw`). Use this when the user says "publish the dive", "ship this dive to MotherDuck", or "update dive <id>". Reads the TSX from `.dive-preview/src/dive.tsx` (override-able), SQL-escapes it, executes via MCP, and records the returned dive UUID in `.dive-preview/dive_id.txt`.

2026-05-184

fix-and-rerun-pipeline

orchestra-hq/orchestra-blueprints

Given a failed Orchestra pipeline run ID, this skill automates the full workflow of diagnosing the failure, applying a code fix on a new GitHub branch, and re-running the pipeline on that branch.

2026-04-084

name	create-dq-rules
description	Reads Snowflake Metadata to construct a pipeline with DQ Rules

Use the Snowflake credentials which are formatted dlt-style in environment variables to connect to snowflake and get the lists of databases, schema and tables
Query a subset of data for 3 tables
based on these, think of some SQL Queries you can run as dbt-style data quality rules, checking for things like nulls, irregular values, duplicates and so on
Use these to construct an Orchestra pipeline as specified below. Use the Orchestra-CLI to validate the pipeline. You should only be changing the SQL Statement field in the SNOWFLAKE_RUN_TEST Task and the matrix vlaues i.e. to parameterise the test

Example pipeline:

version: v1
name: 'Snowflake Data Quality Tests #snowflake #dataquality'
pipeline:
  3e058349-56be-4175-b835-e877e5ce12db:
    tasks:
      cf917cfd-7a64-4e88-83b0-5217409d92a8:
        integration: SNOWFLAKE
        integration_job: SNOWFLAKE_RUN_TEST
        parameters:
          statement: 'select * from BIGQUERY_SAMPLE.PUBLIC.${{ MATRIX.dq_inputs[''table'']
            }}

            where ${{ MATRIX.dq_inputs[''column''] }} is null

            limit 100'
          error_threshold_expression: '> 0'
          warn_threshold_expression: '> 0'
        depends_on: []
        name: Not Null test
        connection: ${{ ENV.SNOWFLAKE_CONNECTION }}
    depends_on:
    - 5c6e2fd1-7782-4a79-9850-0daf332a6f79
    name: ''
    matrix:
      inputs:
        dq_inputs:
        - table: issues
          column: _airbyte_raw_id
        - table: USERS
          column: COUNTRY
schedule:
- name: Daily at 8am
  cron: 0 8 ? * FRI *
  timezone: Europe/London
webhook:
  enabled: false
inputs:
  tablename:
    type: string
    default: hubspot_contacts_raw
alerts:
- name: Failures
  statuses:
  - FAILED
  destinations:
  - integration: SLACK
    destination: alert-testing
- name: Slack
  statuses:
  - FAILED
  destinations:
  - integration: SLACK
    destination: alert-testing