| name | atlas-designer |
| description | Use when designing or writing a homeobox atlas schema. Produces a declarative schema YAML (the homeobox schema IR) that codegens into a schema.py — covering the obs/dataset/feature-registry/fk-registry/other table sections, enums, PointerField markers with feature_registry_schema, all informational field markers (stable_uid, registry_key, polymorphic_registry_key, ontology_aligned, cross_reference, summary), declarative constraints (require_any, equal_length), computed fields, and presence flags. Validates the YAML with scripts/validate_schema_ir.py. |
Atlas Designer
Design the schema for a new homeobox atlas. The output is a schema YAML — the
homeobox schema intermediate representation (IR), a compact, declarative
description of the atlas' table contracts. The YAML is the authored artifact; it
codegens into a deterministic schema.py. Authoring in YAML is safer (no
arbitrary Python, every key is validated at load) and more transportable than
hand-writing the Python file.
Do not write ingestion logic here — only the schema.
References:
- How to write the YAML (document structure, fields, markers, pointers, enums, constraints, computed fields):
references/schema_yaml_ir.md. Read this before authoring.
- Core mechanics (feature spaces, pointer types, stable UIDs, datasets table, FKs):
references/homeobox_concepts.md. Always read this file.
- Complete bundled example IR:
references/multimodal_perturbation_atlas_schema.yaml. The ceiling case — every section, enum, marker, constraint, and computed field. Use it as the template to adapt; do not copy its biological model unless the requested atlas matches it.
Do not author schema.py by hand — author the YAML. To see the Python a given IR codegens into, run scripts/validate_schema_ir.py <schema.yaml> --emit-to schema.py; how each marker maps to a *.declare(...) call is documented in references/schema_yaml_ir.md.
Official docs pages for homeobox:
Workflow
- Clarify the atlas shape:
- What is an obs row: cell, nucleus, spatial spot, image tile, perturbation condition, donor, sample, or something else?
- Which feature spaces are to be supported: gene expression, chromatin accessibility, protein abundance, image features, image tiles, etc.?
- Which feature spaces have a feature axis and therefore need feature registries?
- Which global entities need their own tables: publications, donors, perturbations, compounds, protocols, biosamples, cohorts? Think carefully about the benefits of maintaining separate tables versus a single denormalized table.
- Which identifiers should be durable stable UIDs?
- Write the YAML, working from the example IR and
references/schema_yaml_ir.md. Author in document/dependency order: schema block, enums, fk_registry_tables and feature_registry_tables (pointer/FK targets), other_tables, dataset_table, then obs_tables.
- Add an inline
doc: for every non-obvious biological meaning, and annotate relationships with markers.
- Add declarative
constraints only for real invariants (require_any, equal_length) and computed fields only for derived columns. Do not hand-write validators — the IR has no escape hatch for arbitrary Python.
- Validate with
scripts/validate_schema_ir.py (see Validation).
Ask the user when a table boundary or stable identity is ambiguous. Do not guess the canonical stable identifier for an atlas-specific entity. Atlas design is a collaborative process.
Top-level sections
A schema YAML is a single top-level mapping with fixed keys; unknown keys are a
hard error at load. A valid atlas needs at least an obs table and a dataset
table. See references/schema_yaml_ir.md for the full vocabulary.
schema — name (+ optional doc).enums — StrEnum definitions (optional).obs_tables — list → HoxBaseSchema (measured rows; each declares ≥1 pointer).dataset_table — single → DatasetSchema (the datasets inventory).feature_registry_tables — list → FeatureBaseSchema (feature axes).fk_registry_tables — list → RegistryBaseSchema (deduped global entities).other_tables — list → LanceModel (relationship/section/index tables).
Validation
After writing or editing the YAML, validate it with the bundled script. It parses
the YAML into the IR, checks every ontology_aligned / cross_reference value
against polycomb.registry (the same parse_ontology / parse_crossref the
resolution tooling uses), codegens schema.py (guaranteeing valid Python),
executes the generated classes (so pointer fields validate against registered
feature-space specs and enum references resolve), and builds a throwaway atlas
(exercising Arrow/Lance schema generation):
python scripts/validate_schema_ir.py path/to/schema.yaml
Useful flags:
--emit-to path/to/schema.py — also write the generated Python for inspection.--skip-atlas — stop after exec; skip the temporary-atlas step.
A clean run means the IR is well-formed and parses into a working atlas. Any
problem is a hard failure with an informative error pointing at the offending
key. Do not ingest data — schema validation is enough unless the user asks for
ingestion validation.
Quality checklist
- Every section is the right base for its tables (obs →
obs_tables, feature registries → feature_registry_tables, deduped entities → fk_registry_tables, etc.).
- There is at least one obs table and a
dataset_table.
- Every obs table declares ≥1
pointer field; pointer type matches the feature-space's pointer kind; multimodal pointers are | None with default: null.
- Pointers for spaces with a feature axis carry
feature_registry_schema; raw spaces (image tiles) omit it.
- Every table has at most one
stable_uid field; composite identities are a single derived field marked stable.
- Scalar references use
registry_key; polymorphic references use polymorphic_registry_key with a companion discriminator column; both are named *_uid/*_uids.
- Ontology terms use
ontology_aligned; external database IDs use cross_reference; values are the registry value strings (e.g. NCBITaxon, UniProt), which the validator checks against polycomb.registry.
- Dataset-level aggregates use
summary with a valid op (count, nunique, unique).
- Columns with multiple roles use a
markers: block (→ combine_markers).
- Enum-typed fields replace value-checking validators; only
require_any/equal_length constraints and computed fields are used — no hand-written validators.
default is present (incl. null) only where a default is wanted; its absence means required.scripts/validate_schema_ir.py succeeds on the final YAML.
