| name | skill-navigator |
| description | Intelligent skill navigation system with tiered loading and orchestrator-first routing for context-efficient agent operation. Routes tasks to the correct domain skill based on keyword detection with orchestrator priority. Each skill uses progressive disclosure with references/, scripts/, and assets/ directories. Use this skill as the entry point for any Databricks-related task to determine which specialized skills to load. |
| clients | ["ide_cli","genie_code"] |
| bundle_resource | none |
| deploy_verb | none |
| deploy_note | Routing/navigator meta-skill — selects which domain skills to load; no deployed resource and no deploy verb. Client-agnostic routing. Client-awareness layer present (M06): `client_context` is detected/gated by `skills/vibecoding-state` and `skills/genie-code-environment` is loaded on demand — see the Tier-1-thin note under Context Budget Management. |
| coverage | full |
| metadata | {"author":"prashanth subrahmanyam","version":"3.0","domain":"meta","role":"navigator","last_verified":"2026-06-02","volatility":"low","upstream_sources":[]} |
Skill Navigator: Context-Aware Tiered Loading System
Purpose
This navigation skill implements tiered context loading with orchestrator-first routing to keep total context under Claude Opus's 200K token limit. After restructuring, ALL SKILL.md files are lightweight (< 2K tokens each). The heavy content lives in references/ files loaded on demand.
For detailed domain index summaries, see: references/domain-indexes.md
For visual learners:
For orchestrator deep dives:
For efficiency optimization:
Pipeline Progression Overview (Design-First)
This framework uses a Design-First pipeline: design the target Gold model from the customer's schema CSV, then build the data layers (Bronze → Silver) to feed it, then implement.
Entry point: Customer provides a source schema CSV in data_product_accelerator/context/. Start with Gold Design.
| Stage | Name | Orchestrator Skill | Key Deliverables |
|---|
| 1 | Gold Design | gold/00-gold-layer-design | Read schema CSV → ERDs, YAML schemas, documentation |
| 2 | Bronze | bronze/00-bronze-layer-setup | Bronze Delta tables matching source schema + Faker data |
| 3 | Silver | silver/00-silver-layer-setup | DLT pipelines with DQ rules |
| 4 | Gold Implementation | gold/01-gold-layer-setup | Tables, merge scripts, FK constraints |
| 5 | Planning | planning/00-project-planning | Plan semantic, observability, ML, GenAI phases |
| 6 | Semantic Layer | semantic-layer/00-semantic-layer-setup | Metric Views, TVFs, Genie Spaces |
| 7 | Observability | monitoring/00-observability-setup | Monitors, dashboards, alerts |
| 8 | ML | ml/00-ml-pipeline-setup | ML experiments, models, inference |
| 9 | GenAI Agents | genai-agents/00-course-orchestrator | Agents, evaluation, deployment |
data_product_accelerator/context/*.csv → Gold Design (1) → Bronze (2) → Silver (3) → Gold Impl (4) → Planning (5) → Semantic (6) → Observability (7) → ML (8) → GenAI (9)
Plan-as-Contract Pattern
The Planning orchestrator (stage 5) generates YAML manifest files that downstream orchestrators (stages 6-9) consume as implementation contracts. This follows the "Extract, Don't Generate" principle for the planning-to-implementation handoff.
Gold YAML ─► Planning (stage 5) ─► Manifests ─► Downstream Orchestrators (stages 6-9)
│ │
emits: consumes:
4 manifests 1 manifest each
| Manifest | Emitted By | Consumed By |
|---|
plans/manifests/semantic-layer-manifest.yaml | planning/00-* | semantic-layer/00-* (stage 6) |
plans/manifests/observability-manifest.yaml | planning/00-* | monitoring/00-* (stage 7) |
plans/manifests/ml-manifest.yaml | planning/00-* | ml/00-* (stage 8) |
plans/manifests/genai-agents-manifest.yaml | planning/00-* | genai-agents/00-* (stage 9) |
Each downstream orchestrator has a Phase 0: Read Plan step that reads its manifest. If the manifest doesn't exist (e.g., user skipped Planning), the orchestrator falls back to self-discovery from Gold tables.
Key metadata fields: emits (on Planning), consumes + consumes_fallback (on downstream orchestrators).
See also: Semantic Layer Orchestrator Walkthrough for a detailed example of how orchestrators consume manifests and fall back to self-discovery.
Input Convention: data_product_accelerator/context/ Directory
The data_product_accelerator/context/ directory is the standard location for customer-provided input metadata that drives the entire pipeline:
data_product_accelerator/
└── context/
├── {ProjectName}_Schema.csv # MANDATORY: Customer source schema (THE starting input)
├── business_requirements.md # Optional: Business context, use cases, stakeholders
└── prompts/ # Legacy prompt templates (reference only)
The schema CSV (e.g., data_product_accelerator/context/Wanderbricks_Schema.csv) contains table and column metadata exported from the customer's source system. Expected columns: table_catalog, table_schema, table_name, column_name, ordinal_position, full_data_type, data_type, is_nullable, comment.
Who reads it:
gold/00-gold-layer-design (Phase 0: Schema Intake) — parses into table inventory for dimensional modelingbronze/00-bronze-layer-setup (Approach A) — creates Bronze DDLs matching source schema
Output Convention: Generated Artifacts at Repository Root
All generated artifacts are created at the repository root (parent of data_product_accelerator/), not inside the framework module:
| Artifact | Created By | Path (from repo root) |
|---|
| Dimensional model YAMLs, ERDs | Gold Design (stage 1) | gold_layer_design/ |
| Notebooks and scripts | Bronze, Silver, Gold, Semantic, ML skills | src/ |
| Phase plans and manifests | Planning (stage 5) | plans/ |
| DAB job/pipeline YAML | Asset Bundle skills | resources/ |
| Bundle root config | Asset Bundle skills | databricks.yml |
This separation keeps the framework (data_product_accelerator/) read-only and portable, while generated code lives alongside it at the repo root.
CRITICAL: Skill Structure (Post-Restructuring)
Every skill now follows progressive disclosure and numbered ordering:
domain-folder/
├── 00-orchestrator-skill/ # Orchestrator (manages end-to-end workflow)
│ ├── SKILL.md # Overview, critical rules, mandatory deps
│ ├── references/ # Detailed patterns (loaded on demand)
│ ├── scripts/ # Validation utilities
│ └── assets/templates/ # YAML, SQL, JSON starters
├── 01-worker-skill/ # Worker (specific patterns)
│ ├── SKILL.md
│ └── references/
├── 02-worker-skill/ # Worker (more patterns)
│ └── SKILL.md
└── ...
Numbering Convention
00- = Orchestrator (start here for end-to-end workflows)01-, 02-, ... = Workers (specific patterns, loaded by orchestrator or standalone)- Gold domain exception: Workers are organized into
design-workers/ (Stage 1 design phase) and pipeline-workers/ (Stage 4 implementation phase) subdirectories instead of numbered prefixes
How to Navigate Within a Skill
- Read SKILL.md first (~1-2K tokens) — Contains overview, critical rules, and links
- Read specific references/ files only when you need detailed patterns
- Execute scripts/ as black-box utilities
- Copy assets/templates/ as starting points
Routing Algorithm
1. User request received
2. Detect domain keywords (see table below)
3. IF keyword matches an ORCHESTRATOR skill → Route to orchestrator
- Orchestrator will call worker skills via MANDATORY Read pattern
4. IF keyword matches a WORKER skill AND no orchestrator context → Route to worker directly
- Worker skills have standalone: true and work independently
5. IF keyword matches a WORKER skill AND orchestrator is active → Let orchestrator handle it
Context Budget Management
Claude Opus Context Limit: 200K tokens
Target Operating Budget: 40-60K tokens (for optimal performance)
Maximum Skill Budget: 80K tokens (leaves room for code/files)
| Tier | Purpose | Token Budget | Content |
|---|
| Tier 1: Core | Always loaded | ~4K tokens | 4 core SKILL.md files |
| Tier 2: Domain Index | Loaded on domain detection | ~2K per domain | Domain summaries |
| Tier 3: SKILL.md | Loaded on specific task | ~1-2K each | Individual skill overview |
| Tier 4: References | Loaded on demand | ~2-8K each | Detailed patterns & guides |
Client awareness (Tier-1-thin). This navigator routes the same way for every client. skills/vibecoding-state detects and gates client_context; load skills/genie-code-environment on demand (Tier 4) only when running on the in-workspace agent and a routed skill flags a client caveat (its deploy_note / coverage). Routed prompts open with a client-specific RULE_0 preamble — follow it. Keep this routing layer thin; behavior detail stays in genie-code-environment.
Task Detection & Skill Routing Table
Bootstrap / New Project Route (start here)
| Task Keywords | Route To |
|---|
| "new project", "schema CSV", "customer schema", "bootstrap", "start from scratch", "onboarding", "build data platform" | gold/00-gold-layer-design (Stage 1 — reads schema CSV from data_product_accelerator/context/) |
Orchestrator Routes (prefer these for end-to-end workflows)
| Task Keywords | Domain | Route To (Orchestrator) |
|---|
| "design Gold", "dimensional model", "Gold from scratch", "schema CSV" | Gold | gold/00-gold-layer-design (stage 1) |
| "Bronze setup", "Bronze tables", "test data", "demo data" | Bronze | bronze/00-bronze-layer-setup (stage 2) |
| "Silver layer", "create Silver", "Bronze to Silver", "Silver pipeline" | Silver | silver/00-silver-layer-setup (stage 3) |
| "implement Gold", "Gold tables", "Gold merge scripts" | Gold | gold/01-gold-layer-setup (stage 4) |
| "project plan", "architecture planning", "planning_mode: workshop" | Planning | planning/00-project-planning (stage 5) |
| "semantic layer", "build Genie", "Metric Views and TVFs" | Semantic | semantic-layer/00-semantic-layer-setup (stage 6) |
| "observability", "monitoring setup", "dashboards and alerts" | Monitoring | monitoring/00-observability-setup (stage 7) |
| "MLflow", "ML pipeline", "model training" | ML | ml/00-ml-pipeline-setup (stage 8) |
| "GenAI agent", "build agent", "ResponsesAgent" | GenAI | genai-agents/00-course-orchestrator (stage 9) |
Worker Routes (for standalone/specific tasks)
| Task Keywords | Domain | Route To (Worker) |
|---|
| "Faker", "synthetic", "corruption" | Bronze | bronze/01-faker-data-generation |
| "DLT", "expectations" | Silver | silver/01-dlt-expectations-patterns |
| "DQX", "validation" | Silver | silver/02-dqx-patterns |
| "Gold merge", "MERGE" | Gold | gold/pipeline-workers/02-merge-patterns |
| "duplicate key" | Gold | gold/pipeline-workers/03-deduplication |
| "Gold documentation" | Gold | gold/design-workers/06-table-documentation |
| "ERD", "diagram" | Gold | gold/design-workers/05-erd-diagrams |
| "dimension pattern", "role-playing", "junk dimension", "hierarchy" | Gold | gold/design-workers/02-dimension-patterns |
| "fact pattern", "factless", "accumulating snapshot", "measure additivity" | Gold | gold/design-workers/03-fact-table-patterns |
| "conformed dimension", "bus matrix", "drill-across" | Gold | gold/design-workers/04-conformed-dimensions |
| "design validation", "validate model" | Gold | gold/design-workers/07-design-validation |
| "industry alignment", "industry data model", "Vibe data model", "industry coverage" | Gold | gold/design-workers/08-industry-alignment |
| "schema validation" | Gold | gold/pipeline-workers/05-schema-validation |
| "fact grain" | Gold | gold/pipeline-workers/04-grain-validation |
| "YAML setup" | Gold | gold/pipeline-workers/01-yaml-table-setup |
| "metric view", "semantic" | Semantic | semantic-layer/01-metric-views-patterns |
