create-dq-rules

النجوم٤

التفرعات٢

آخر تحديث١٨ مايو ٢٠٢٦ في ١٤:١١

Reads Snowflake Metadata to construct a pipeline with DQ Rules

التثبيت

التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.

المصدر

orchestra-hq

orchestra-hq/orchestra-blueprints

فتح مستودع GitHub عرض مستودعات المنشئ

تنزيل

المهن ذات الصلةSOC

استنادا إلى تصنيف SOC المهني

مصممو قواعد البياناتمهن الحاسوب والرياضيات·SOC 15-1243

SKILL.md

readonly

name	create-dq-rules
description	Reads Snowflake Metadata to construct a pipeline with DQ Rules

Use the Snowflake credentials which are formatted dlt-style in environment variables to connect to snowflake and get the lists of databases, schema and tables
Query a subset of data for 3 tables
based on these, think of some SQL Queries you can run as dbt-style data quality rules, checking for things like nulls, irregular values, duplicates and so on
Use these to construct an Orchestra pipeline as specified below. Use the Orchestra-CLI to validate the pipeline. You should only be changing the SQL Statement field in the SNOWFLAKE_RUN_TEST Task and the matrix vlaues i.e. to parameterise the test

Example pipeline:

version: v1
name: 'Snowflake Data Quality Tests #snowflake #dataquality'
pipeline:
  3e058349-56be-4175-b835-e877e5ce12db:
    tasks:
      cf917cfd-7a64-4e88-83b0-5217409d92a8:
        integration: SNOWFLAKE
        integration_job: SNOWFLAKE_RUN_TEST
        parameters:
          statement: 'select * from BIGQUERY_SAMPLE.PUBLIC.${{ MATRIX.dq_inputs[''table'']
            }}

            where ${{ MATRIX.dq_inputs[''column''] }} is null

            limit 100'
          error_threshold_expression: '> 0'
          warn_threshold_expression: '> 0'
        depends_on: []
        name: Not Null test
        connection: ${{ ENV.SNOWFLAKE_CONNECTION }}
    depends_on:
    - 5c6e2fd1-7782-4a79-9850-0daf332a6f79
    name: ''
    matrix:
      inputs:
        dq_inputs:
        - table: issues
          column: _airbyte_raw_id
        - table: USERS
          column: COUNTRY
schedule:
- name: Daily at 8am
  cron: 0 8 ? * FRI *
  timezone: Europe/London
webhook:
  enabled: false
inputs:
  tablename:
    type: string
    default: hubspot_contacts_raw
alerts:
- name: Failures
  statuses:
  - FAILED
  destinations:
  - integration: SLACK
    destination: alert-testing
- name: Slack
  statuses:
  - FAILED
  destinations:
  - integration: SLACK
    destination: alert-testing

المزيد من هذا المستودع

نفس المستودع

dlt-source-via-orchestra

orchestra-hq/orchestra-blueprints

Build a new dlt source that lands API data into MotherDuck, then ship it through Orchestra. Use this when the user wants to "pull X from <API> into MotherDuck" or "add a new dlt source and run it on Orchestra". Walks the full lifecycle on a feature branch — author the source, register an Orchestra pipeline that invokes it, trigger the pipeline against that branch, and poll until done.

2026-05-184

publish-motherduck-dive

orchestra-hq/orchestra-blueprints

Publish or update a MotherDuck Dive by inlining a local .tsx file into a `MD_CREATE_DIVE` / `MD_UPDATE_DIVE` SQL call sent through the MotherDuck MCP (`mcp__MotherDuck__query_rw`). Use this when the user says "publish the dive", "ship this dive to MotherDuck", or "update dive <id>". Reads the TSX from `.dive-preview/src/dive.tsx` (override-able), SQL-escapes it, executes via MCP, and records the returned dive UUID in `.dive-preview/dive_id.txt`.

2026-05-184

fix-and-rerun-pipeline

orchestra-hq/orchestra-blueprints

Given a failed Orchestra pipeline run ID, this skill automates the full workflow of diagnosing the failure, applying a code fix on a new GitHub branch, and re-running the pipeline on that branch.

2026-04-084

name	create-dq-rules
description	Reads Snowflake Metadata to construct a pipeline with DQ Rules

Use the Snowflake credentials which are formatted dlt-style in environment variables to connect to snowflake and get the lists of databases, schema and tables
Query a subset of data for 3 tables
based on these, think of some SQL Queries you can run as dbt-style data quality rules, checking for things like nulls, irregular values, duplicates and so on
Use these to construct an Orchestra pipeline as specified below. Use the Orchestra-CLI to validate the pipeline. You should only be changing the SQL Statement field in the SNOWFLAKE_RUN_TEST Task and the matrix vlaues i.e. to parameterise the test

Example pipeline:

version: v1
name: 'Snowflake Data Quality Tests #snowflake #dataquality'
pipeline:
  3e058349-56be-4175-b835-e877e5ce12db:
    tasks:
      cf917cfd-7a64-4e88-83b0-5217409d92a8:
        integration: SNOWFLAKE
        integration_job: SNOWFLAKE_RUN_TEST
        parameters:
          statement: 'select * from BIGQUERY_SAMPLE.PUBLIC.${{ MATRIX.dq_inputs[''table'']
            }}

            where ${{ MATRIX.dq_inputs[''column''] }} is null

            limit 100'
          error_threshold_expression: '> 0'
          warn_threshold_expression: '> 0'
        depends_on: []
        name: Not Null test
        connection: ${{ ENV.SNOWFLAKE_CONNECTION }}
    depends_on:
    - 5c6e2fd1-7782-4a79-9850-0daf332a6f79
    name: ''
    matrix:
      inputs:
        dq_inputs:
        - table: issues
          column: _airbyte_raw_id
        - table: USERS
          column: COUNTRY
schedule:
- name: Daily at 8am
  cron: 0 8 ? * FRI *
  timezone: Europe/London
webhook:
  enabled: false
inputs:
  tablename:
    type: string
    default: hubspot_contacts_raw
alerts:
- name: Failures
  statuses:
  - FAILED
  destinations:
  - integration: SLACK
    destination: alert-testing
- name: Slack
  statuses:
  - FAILED
  destinations:
  - integration: SLACK
    destination: alert-testing