rocky-codegen

name	rocky-codegen
description	Rocky CLI JSON-output schema cascade. Use when editing any `*Output` struct in `engine/crates/rocky-cli/src/output.rs` (or `commands/doctor.rs`), adding a new CLI command schema, or when a change could affect `schemas/*.schema.json`, `sdk/python/src/rocky_sdk/types_generated/`, or `editors/vscode/src/types/generated/`. Also use when CI reports `codegen-drift`.

Rocky codegen cascade

Rocky's CLI --output json is the interface contract with every consumer (Dagster, VS Code, shell scripts). Every command is backed by a typed Rust struct that derives JsonSchema, and the Pydantic + TypeScript bindings are autogenerated from those schemas. Drift is enforced in CI.

If you edit a Rust output struct and forget to regenerate, the codegen-drift workflow will fail the PR.

When to use this skill

• Editing a *Output struct in engine/crates/rocky-cli/src/output.rs or engine/crates/rocky-cli/src/commands/doctor.rs
• Adding a new field to any type that appears inside a CLI output
• Adding a brand new CLI command that needs a JSON schema
• Investigating a codegen-drift.yml CI failure

The three-step pipeline

engine/crates/rocky-cli/src/output.rs       (Rust source of truth)
        │
        ▼  cargo run -- export-schemas schemas/
schemas/*.schema.json                        (committed)
        │
        ├──▶  sdk/python/src/rocky_sdk/types_generated/   (Pydantic v2)
        └──▶  editors/vscode/src/types/generated/                       (TypeScript)

Canonical workflow

From the monorepo root:

# 1. Edit the Rust struct
$EDITOR engine/crates/rocky-cli/src/output.rs

# 2. Run the full codegen pipeline
just codegen

# 3. Review the diff — you should see changes in 3 places:
git status
#   engine/crates/rocky-cli/src/output.rs        ← your edit
#   schemas/<command>.schema.json                 ← regenerated
#   sdk/python/src/rocky_sdk/types_generated/…    ← regenerated
#   editors/vscode/src/types/generated/…          ← regenerated

# 4. Commit all of it in ONE commit
git add engine/ schemas/ sdk/ integrations/ editors/
git commit -m "feat(engine): add <field> to <command>Output"

Never commit a Rust output change without the regenerated bindings. The codegen-drift CI workflow (.github/workflows/codegen-drift.yml) runs just codegen and fails if git diff is non-empty.

What just codegen actually does

Three recipes in justfile, runnable individually:

Recipe	What it does
just codegen-rust	cargo run --release --bin rocky -- export-schemas schemas/ — rebuilds engine in release mode (shared with regen-fixtures), then writes the JSON schemas to schemas/ (run `ls schemas/*.schema.json
just codegen-sdk	Runs datamodel-codegen over schemas/*.schema.json into sdk/python/src/rocky_sdk/types_generated/. Self-heals the curated __init__.py barrel via git checkout.
just codegen-vscode	Runs json2ts per schema into editors/vscode/src/types/generated/. Self-heals the curated index.ts barrel via git checkout.

Both codegen-sdk and codegen-vscode intentionally overwrite the output directory and then git checkout HEAD -- <barrel> to restore the curated re-export files. Do not hand-edit files under types_generated/ or types/generated/ other than those two barrels — the next codegen run nukes them.

Adding a new CLI command schema

1. Add (or edit) the typed struct in engine/crates/rocky-cli/src/output.rs (or commands/<name>.rs) deriving JsonSchema:

   #[derive(Debug, Serialize, Deserialize, JsonSchema)]
   pub struct MyCommandOutput { ... }
   

2. Register it in engine/crates/rocky-cli/src/commands/export_schemas.rs::schemas().
3. Run just codegen.
4. For the dagster consumer: re-export the new type from integrations/dagster/src/dagster_rocky/types.py (in the re-export section near the bottom), then add a dispatch entry in parse_rocky_output(), add a RockyResource method in resource.py, and a fixture + test (see integrations/dagster/CLAUDE.md for the 9-step checklist).
5. For the vscode consumer: no extra wiring — src/types/rockyJson.ts is already a 100% type-alias shim over src/types/generated/index.ts.

Fixture drift (related but distinct)

scripts/regen_fixtures.sh (aka just regen-fixtures) captures live rocky --output json output against the playground POCs into integrations/dagster/tests/fixtures_generated/. This is a drift-detection corpus — test_generated_fixtures.py re-validates the captured JSON against the current Pydantic models. The dagster parsing tests themselves load hand-crafted Python dicts from integrations/dagster/tests/scenarios.py, exposed as *_json pytest fixtures via conftest.py.

When to regen fixtures:

• After a schema change that affects the shape of an output (new fields, renamed fields, changed types).
• Before committing, run just regen-fixtures and check that test_generated_fixtures.py still passes. If it doesn't, the captured shape no longer matches the Pydantic models — usually means the codegen step was skipped.

Prerequisite: the release binary at engine/target/release/rocky must exist (just codegen already builds it).

Common pitfalls

• Editing a generated file by hand — will be wiped on next codegen. Always edit the Rust source.
• Forgetting the JsonSchema derive — cargo build passes but export-schemas silently skips the type. Check the schemas/ diff.
• Adding a new type but not registering it in export_schemas.rs::schemas() — the schema never gets emitted.
• Committing partial regeneration — always run full just codegen, not just one of the sub-recipes.
• CI drift failure on a PR that did NOT touch output.rs — usually means someone bumped datamodel-code-generator or json2ts; re-run just codegen locally and commit the deterministic diff.

Local belt-and-braces: the pre-commit hook

There's a repo-root git hook at .git-hooks/pre-commit that runs the codegen check locally, so drift is caught before you push instead of after a failed CI run. Enable once:

just install-hooks
# or manually:
git config core.hooksPath .git-hooks

The hook only fires when the staged change includes one of:

• engine/crates/rocky-cli/src/output.rs
• engine/crates/rocky-cli/src/commands/doctor.rs
• engine/crates/rocky-cli/src/commands/export_schemas.rs

When triggered, it runs just codegen and fails the commit if the regenerated bindings produce a non-empty git diff. Skip with ROCKY_SKIP_CODEGEN_HOOK=1 git commit … in emergencies — but the CI workflow will still catch drift on the PR.

Note on core.hooksPath: it's a repo-wide setting (stored in .git/config, so it doesn't get shared across clones). If you've previously enabled hooks inside integrations/dagster/ via git config core.hooksPath .git-hooks, that points at the same root-level path (hooksPath is repo-relative) — running just install-hooks at the root is the same operation.

Reference files

• engine/CLAUDE.md — "JSON Output Schema" section lists all command output structs.
• integrations/dagster/CLAUDE.md — "Adding support for a new Rocky CLI command" 9-step checklist.
• .github/workflows/codegen-drift.yml — the CI check that enforces this.
• .git-hooks/pre-commit — the local mirror of the CI check.
• justfile — the codegen* recipes (lines ~63-126) + install-hooks.