aboyalejandro

Configure manual-trigger judges in Opik — derive one evaluator dimension per promise, **match against existing evaluators on the account first**, generate `_local/create_evaluators.py` only for gaps, run it. Output is `.edd/evaluator-plan.md` + judges live in Opik with `enabled=False, sampling_rate=1.0` (disabled blocks auto-firing during experimentation; sampling_rate pre-staged so flipping to auto-sample is a one-field change). Invoke as `edd:scope-evals` or when the user says "set up judges", "create evaluators", "scope the evals". **Audit mode** (no promises.md needed): invoke when user says "audit my evaluators", "check existing evals", "audit evals against fundamentals", or invokes `edd:scope-evals audit` — reads live Opik evaluators and cross-checks against eval-fundamentals.md principles. For *firing* judges and reading scores, see `edd:run`.

Extract the agent's promise inventory and write baseline regression scenarios. Produces `.edd/promises.md` (one line per skill/tool/promise) and `regressions.txt` (5–8 baseline scenarios at aggression 1). Idempotent — skip if both exist and agent source is unchanged. Invoke as `edd:scope-agent` or when the user says "scope the agent", "extract promises", "build regressions".

Promote branch-tagged traces into a durable Opik dataset + experiment with a UI scorecard. Outer loop of the eval pipeline. Use when the score must outlive the branch — cross-time comparison, reviewer handoff. Invoke as `edd:experiment` or when the user says "build dataset", "run experiment", "scoreboard in Opik UI", "outer loop".

Fire scenarios at the agent. Mode 1 emits + tags traces for inline Claude analysis, with optional `edd score` to run judges and print the per-dimension table. Mode 2 emits + tags only — judges fire in `edd:experiment`, not here. A small smoke-judge-check (3–5 traces) verifies judges land before scaling. Generates `scenarios.txt` at session aggression level; branches on whether trace enrichment runs between emit and the smoke check. Invoke as `edd:run` or when the user says "run scenarios", "score traces", "fire judges", "edd inner loop". For *creating* judges, see `edd:scope-evals`.

Eval-Driven Development router — picks the mode and dispatches. Asks two questions (mode 1/2, aggression 1/2/3), writes `.edd/session.json`, hands off to one of the phase skills. Use this when the user wants an eval workflow but hasn't named a phase. Skill-specific work belongs in `edd:scope-agent`, `edd:scope-evals`, `edd:run`, `edd:experiment`, or `edd:expand`.

Audit OpenMetadata for missing or drifted descriptions across all dbt layers, generate AI descriptions, write confirmed descriptions back to dbt YAML (source of truth), then sync to OpenMetadata via patch_entity. Triggers include "enrich metadata", "missing descriptions", "which tables have no description", "fill metadata", "generate descriptions", "update catalog", "sync descriptions".

Audit and enrich dbt mart models for AI consumption. Checks schema quality, queries the database to discover edge cases, validates OpenMetadata catalog presence, and writes fixes back to dbt YAML. Triggers include "ai readiness", "is this model ready", "checklist for", "enrich", "pre-merge check".

Manage an OpenMetadata glossary derived from dbt models. Parses dbt YAML for column names and descriptions, groups them into business categories, and creates/syncs glossary terms via OpenMetadata MCP. Triggers include "metadata glossary", "create glossary", "business terms", "sync glossary".

Analyze downstream impact before making schema changes. Traces lineage through dbt models and dashboards to identify what breaks if a column is renamed, dropped, or its type changes. Triggers include "impact analysis", "what breaks if", "can I rename", "can I drop", "downstream impact".