| name | golden-dataset |
| license | MIT |
| compatibility | Claude Code 2.1.220+. |
| description | Golden dataset lifecycle patterns for curation, versioning, quality validation, and CI integration. Use when building evaluation datasets, managing dataset versions, validating quality scores, or integrating golden tests into pipelines. |
| tags | ["golden-dataset","evaluation","dataset-curation","dataset-validation","quality","llm-testing"] |
| context | fork |
| agent | data-pipeline-engineer |
| version | 2.0.0 |
| author | OrchestKit |
| user-invocable | false |
| disable-model-invocation | true |
| complexity | medium |
| persuasion-type | guidance |
| metadata | {"category":"document-asset-creation"} |
| allowed-tools | ["Read","Glob","Grep","WebFetch","WebSearch"] |
Golden Dataset
Comprehensive patterns for building, managing, and validating golden datasets for AI/ML evaluation. Each category has individual rule files in rules/ loaded on-demand.
Quick Reference
| Category | Rules | Impact | When to Use |
|---|
| Curation | 2 | HIGH | Content collection, annotation pipelines |
| Management | 2 | HIGH | Versioning, backup/restore |
| Validation | 1 | CRITICAL | Regression testing |
| Add Workflow | 1 | HIGH | 9-phase curation, quality scoring, bias detection, silver-to-gold |
Total: 6 rules across 4 categories. House thresholds and scars: references/ork-delta.md.
Curation
Content collection, multi-agent annotation, and diversity analysis for golden datasets.
| Rule | File | Key Pattern |
|---|
| Collection | rules/curation-collection.md | Content type classification, quality thresholds, duplicate prevention |
| Annotation | rules/curation-annotation.md | Multi-agent pipeline, consensus aggregation, Langfuse tracing |
Difficulty ladder, coverage floors, and duplicate thresholds: references/ork-delta.md.
Management
Versioning, storage, and CI/CD automation for golden datasets.
| Rule | File | Key Pattern |
|---|
| Versioning | rules/management-versioning.md | JSON backup format, embedding regeneration, disaster recovery |
| Storage | rules/management-storage.md | Backup strategies, URL contract, data integrity checks |
CI automation for backups is upstream's job; see "Upstream coverage" below.
Validation
Quality scoring, drift detection, and regression testing for golden datasets.
