add-cross-language-construct

name	add-cross-language-construct
description	Full round-trip checklist for adding or changing a construct that crosses the Python↔JS bridge as serialized IR — a new operator, mark, builder method, or coordinate transform. Use BEFORE implementing such a construct; missing a step fails late (at parity-render or CI, not at JS build).

Adding a cross-language construct (operator / mark / builder method)

A new thing that crosses the Python↔JS bridge as serialized IR touches more files than just the schema; missing one fails late (at parity-render or CI, not at JS build), so do all of these in one pass rather than discovering them a failure at a time:

1. JS factory — author it under packages/gofish-graphics/src/ast/ and export from src/lib.ts. Tag it with __serialize = { type, opts } so it round-trips.
2. JS reconstruction — TWO separate sites, both required. (a) The deserializer registry packages/gofish-graphics/src/serialize/registry.ts (OPERATOR_MAP / MARK_MAP) — used by Serialize.fromJSON / the widget. (b) The parity render harness tests/harness/main.ts — the Python visual-parity capture rebuilds a ChartBuilder from IR through its own switch on operator/mark type (it does NOT call the registry), so add a case there too. Miss (a) and fromJSON throws "unknown type"; miss (b) and the operator is silently dropped at parity-render (e.g. a resolve that never runs, so a downstream mark gets raw values). Anything end-users import by name in that harness (e.g. chart) must also exist — renaming/removing an export breaks window.__renderChart__ for every story at load. Both sites are still hand-maintained — they weren't generified as part of the descriptor-table work (see the "What shipped" / deferred list in apps/docs/docs/internals/design/python-wrapper-codegen.md); a construct-by-type-keyed-off-the-table version of both remains a candidate follow-up.
3. One descriptor entry in packages/gofish-ir/src/frontend/descriptors.ts — the single-source table of a construct's fields (operators, leaf marks, combinator marks, coord transforms), in the small t.*/ch.* type DSL. This is what validate.ts (generic interpreter over the table — operators reject on an unknown/mistyped field, leaf marks only warn during the current rollout), jsonSchema.ts (per-construct $defs, generated at call time), and the Python factory (next step) all read off. Envelope/structural types (ChartIR/LayerIR/DataIR/MarkIR union, ChannelValue, ConstraintIR, LabelIR, TranslateIR, AxesOptions) and cut/offset/ref are NOT in the table — those stay hand-authored directly in schema.ts/jsonSchema.ts, since they're recursive/structural shapes rather than flat field bags. After adding an entry: pnpm --filter gofish-ir build and pnpm --filter docs sync-ir-schema (CI runs check-ir-schema).
4. Python wrapper — run the generator, don't hand-write the factory. pnpm --filter gofish-python gen regenerates packages/gofish-python/gofish/_generated.py from the same descriptor table (closed-signature leaf marks, combinator-only marks incl. the compositing pyName renames, _opts() cores for dual-form constructs) and is CI-checked for freshness — do not hand-edit _generated.py. Hand-written Python work in gofish/ast.py is now only needed for genuinely non-mechanical constructs: the operator-vs-combinator dispatch that calls into a generated _opts() core, and anything not in the descriptor table at all (derive/resolve/join's bridge logic, palette/gradient/field/datum/normalize/repeat/ref/selectAll). Add the __init__.py export either way. A Python keyword like from becomes from_= via the descriptor's py key, mapped to the "from" wire key. A builder-level construct that lowers to a layer (like .layer()) ALSO needs tests/scripts/derive-server.py (serialize_chart / the _handle_load normalization) to recognize it, or its parity story loads with a 500.
5. Parity stories — port pure-spec stories to tests/python-stories/; add genuinely non-portable ones (e.g. a JS-side function-mark composing refs, which can't cross the derive RPC) to tests/.python-sync-exempt. Gates, and what each does/doesn't cover: pnpm --filter @gofish/tests validate-python-ir (Python IR → schema; exercises 2a/3/4 but NOT the harness 2b — a story can pass this and still fail to render) and python3 tests/scripts/check-story-ir.py <file> (fast single-file IR check). The harness (2b) is only exercised by the visual capture: pnpm --filter @gofish/tests capture-python — run it after a new operator/mark or it'll mask a dropped construct.
6. Docs (see the docs-sync note in CLAUDE.md) and the same datum.X→X / by migration in Python stories, not just JS — story by: strings flow to the same projection.

pytest gotcha: the gofish-python widget tests need pnpm --filter gofish-python build:widget first; without the bundle ~19 tests fail with FileNotFoundError: widget.esm.js — those are environmental, not your change.