| id | SKL-data-DATAQUALITYMONITORING |
| name | Data Quality Monitoring |
| description | Data Quality (DQ) Monitoring is the continuous process of validating data against predefined rules and expectations. In a modern data stack, monitoring must happen at every stage: **Ingestion**, **Tra |
| version | 1.0.0 |
| status | active |
| owner | @cerebra-team |
| last_updated | 2026-02-22 |
| category | Backend |
| tags | ["api","backend","server","database"] |
| stack | ["Python","Node.js","REST API","GraphQL"] |
| difficulty | Intermediate |
Data Quality Monitoring
Skill Profile
(Select at least one profile to enable specific modules)
Overview
Data Quality (DQ) Monitoring is the continuous process of validating data against predefined rules and expectations. In a modern data stack, monitoring must happen at every stage: Ingestion, Transformation, and Serving.
Core Principle: "Garbage in, garbage out. If data is wrong, analytics, AI, and decisions will be wrong too."
This skill provides comprehensive guidance on implementing data quality monitoring across the data pipeline, from automated testing to real-time anomaly detection.
Why This Matters
Core Concepts & Rules
1. Core Principles
- Follow established patterns and conventions
- Maintain consistency across codebase
- Document decisions and trade-offs
2. Implementation Guidelines
- Start with the simplest viable solution
- Iterate based on feedback and requirements
- Test thoroughly before deployment
Inputs / Outputs / Contracts
- Inputs:
- <e.g., env vars, request payload, file paths, schema>
- Entry Conditions:
- <Pre-requisites: e.g., Repo initialized, DB running, specific branch checked out>
- Outputs:
- <e.g., artifacts (PR diff, docs, tests, dashboard JSON)>
- Artifacts Required (Deliverables):
- <e.g., Code Diff, Unit Tests, Migration Script, API Docs>
- Acceptance Evidence:
- <e.g., Test Report (screenshot/log), Benchmark Result, Security Scan Report>
