runtime-ui

# Runtime-authored UI ## Read first Before changing a runtime surface, read: 1. [`docs/architecture/runtime-authored-ui.md`](../../docs/architecture/runtime-authored-ui.md) 2. [`assets/ui/README.md`](../../assets/ui/README.md) 3. [`skills/lunco-ui/SKILL.md`](../lunco-ui/SKILL.md) when the surface overlaps egui, the workbench, or docking 4. [`skills/test-via-api/SKILL.md`](../test-via-api/SKILL.md) for live verification The generic implementation is `lunco-workbench-runtime-ui`; the `luncosim` windowed host in `crates/lunco-luncosim-ui/src/ui/` supplies app-specific gates, capture mode, and action handling. Do not assume that `lunica` or a headless server has this surface manifest. The committed dependency baseline is `bevy_hui 0.8.0`, `bevy_flair 0.9.0`, and Bevy `0.20.0`. Verify the lockfile and upstream release notes before changing it. The separate `bevy_hui_widgets 0.6.0` crate provides primitive text-input, slider, and select components, but is intentionally not a LunCoSim dependency: it does not define our clipboard, validation, keyboard-navigation, accessibility, or modal semantics. ## Choose the right layer Use runtime HTML/CSS for a small authored presentation surface that should be changed without recompiling Rust: HUDs, status cards, progress overlays, telemetry summaries, and simple buttons. Use `lunco-workbench`/egui for the application shell, docking, code editors, large inspectors, rich text input, complex forms, modal dialogs, and controls that need semantics not present in the runtime contract. Runtime UI uses the existing egui host and dock geometry; it does not replace the workbench or create a second hit-test/camera system. The standard Twin, file and library Twin browser is the optional `lunco-workbench-browser` feature layered on the shell; do not recreate its views in an authored runtime surface. `lunco-ui::modal` remains the owner for rich dialogs requiring focus, queued outcomes, editing, validation, or accessibility. The generic runtime surface also supports a deliberately small authored modal contract: a viewport surface may declare `modal: true` and an authored `dismiss_action`; visible controls own their computed input regions and Escape emits that semantic action. Generic keyed collection hosts provide dynamic rows, while Rhai owns the records, labels, ordering, visibility, and action meaning. Do not grow this into a second rich-dialog implementation without a separate typed contract and acceptance tests. Twin-authored actions are open-ended semantic identifiers. The reusable `program-browser` surface publishes typed arrays of records; its HUI row template is reconciled by the generic keyed collection host. A surface may also declare a generic dropdown: the manifest supplies the trigger, option source, key/label/action field names, and Rhai-owned width/max-height sources. The compact `camera-status` surface is one consumer of that primitive, not a camera-specific widget. `program_editor` owns selection, editor focus, atomic source switching, and creation for any authored program, regardless of whether its owner is a rover, lander, route, or another model. Rich source text entry remains in the existing Rhai editor/REPL until HUI gains tested typed input semantics. For dynamic semantic controls, use the HUI convention `on_press="runtime_ui_authored_action" tag:action="{action}"`. Additional `tag:*` values become a typed `HookValue` parameter map on the generic action event; the runtime does not concatenate names and values into a protocol string. This permits Twin-defined actions and dynamic controls without registering one Rust callback per item. Do not use JavaScript or encode action payloads as JSON. Collection hosts retain the Rhai-authored row order, clip their list, and consume wheel input at the host boundary. They own row lifecycle only; do not add a per-surface Rust list resource or a fixed row-count view model. Use stable `id` attributes and `#id` selectors for authored HUI nodes. Flair supports class selectors, but HUI 0.7 does not turn an HTML `class` attribute into a `ClassList`; it treats that attribute as an unknown style property. Project-owned visibility policy belongs in the active Twin manifest's generic `[settings]` table. A surface declares `setting` plus `setting_default` in `runtime_surfaces.json`; Rhai reads/writes the same scope with `get_twin_setting`/`set_twin_setting`. Keep user-global diagnostics, theme, input visualisation, and window preferences in `lunco-settings`. Missing Twin keys use the surface's authored default, so Rust does not grow a field for each new preference. ## Camera views and configurable mission cards Windowed LunCoSim supports up to eight live camera windows. Call `CreateCameraFeed` for an independent id, then `SetCameraFeed { feed, source: <exact camera gid>, label, width, height }`. Omitted `feed` selects primary view 1. `RemoveCameraFeed` releases a window; `InspectCameraFeeds` reports all selections. Native windows provide a camera picker, resolution menu, new-view button, move, resize and close. Use `SetCameraFeedResolution { feed, match_window, width, height }` for fixed pixels or live image-area matching. The menu also copies the app window's current physical dimensions. Omitted dimensions in `SetCameraFeed` preserve the view's resolution policy. Allocation limits and rejection details are reported by `InspectCameraFeeds`; an invalid request preserves the last valid target. Use two physically separated native sensor cameras for stereo, and verify both rendered views in the owned session. See the [render owner contract](../../docs/architecture/render-decoupling.md#live-camera-observations) and `camera_feed_live.rhai` production acceptance. For grouped navigation and live depth/noise controls, follow [camera windows](../lunco-ui/SKILL.md#camera-windows-and-image-controls). Run `scripts/api/test_camera_views.py --port <free-port>` for the windowed pixel and world-fixed-sky gate. Capture only after the view reports `ready`. Named guided cards accept `SetGuidedHudLayout { hud, width, text_scale, draggable }`. Layout is scene scoped and exposed in `InspectGuidedHud`; invalid bounds reject before mutation. A card's context menu changes the same settings. Twin Rhai owns which telemetry lines to publish, and can retain chosen groups through existing Twin settings. `guided_hud_layout.rhai` covers layout admission and rejection. HUI control surfaces remain configurable through their Twin manifest, template and stylesheet. ## Authoring workflow ### 1. Define a generic capability namespace Add or extend an engine-side producer only when the value is not already available. Publish authoritative, presentation-ready named values through `EngineExposures`: ```rust let mut ui = exposures.writer("mission-status"); ui.visible(has_mission); ui.property("title", mission_title); ui.property("state", state_label); ui.property("state_color", "var(--ok-color)"); ``` The namespace is a capability boundary shared by HTML, egui, API, telemetry, and remote consumers. Do not add `domain_to_view`, `vessel_exposure`, or a widget-specific Rust registry. Resolve source state in the engine producer; keep markup unaware of ECS/domain types. Initialize `EngineExposures` and scene-derived producer caches through `AppSceneExt::init_scene_resource`. Surfaces default to scene lifetime, including hidden properties and subject identities. Application-owned producers must explicitly call `retain_across_scenes(namespace)`; never retain a namespace that can contain Twin-owned values. Change-gated producers that survive a scene must republish once when their namespace is absent after reset. Remove an authored namespace with `remove_surface` when its source is deleted or renamed. The production Twin retirement gate seeds an unsaved authored surface through `ApplyUsdOp` and uses Rhai `ReadExposures` verdicts across replacement and same-path reopening. Engine health follows the same generic path. Read the typed `EngineHealthSnapshot`/`PhysicsHealthSnapshot` publication and expose named properties through the ordinary `engine-health` namespace; do not add a HUD reader for `DiagnosticsStore`, an Avian timing query, or another source-specific bridge. Native UI, HUI, API, telemetry, and recording consumers all read the common publication. For scalar participant state, use the shared `PortRegistry`; do not create a parallel port reader for a HUD. `SimulationProgress` owner and reason facts are projected into each authored runtime surface as typed `simulation_progress` data. Let the active Rhai policy turn those facts into user-facing labels such as `PHYSICS LOADING` or `WAITING FOR TERRAIN`; the engine does not hardcode HUD wording. Progress changes invalidate the existing surface projection, so a readiness label does not poll the simulation. During preparation the built-in Rhai visibility policy temporarily shows an authored surface before possession, then returns to its authored `possessed` or `always` mode when the holds clear. Use the optional typed `pause` fact (`condition`, `detail`) for a coupled subject's retained stop condition. Keep the wording in Rhai and qualify the paused HUD after its model clock has stopped; a model's transient failure output is not the retained pause owner. For recording, verify the required surface in an encoded trial with live data. The readiness owner tracks whether its retained tree has consumed an exposure snapshot; it must not wait for equality with a later global publication revision. Camera, layout and render extraction remain required before capture admission. Command and one-shot REPL timing is also generic presentation data. Read the `application-cadence` exposure for command and REPL sequence/interval/rate values; do not attach a HUD timer to `Time<Virtual>`, count API requests as completed script evaluations, or read the command/REPL owners directly. Producers must use change detection, revisions, or dirty flags. Continuous values are coalesced to the current bounded presentation cadence (20 Hz). `EngineExposures.revision` changes when a complete surface changes or retires; it is not a frame counter. Do not use JSON to detect internal changes. Runtime visibility/property policies prepare immutable facts through shared bounded async admission in the same headless and windowed owner. Use `ReadExposures` to wait for `state == "ready"` before asserting their properties. A new identity starts hidden and `pending`; `failed` exposes an owner error without partial properties and requires source/identity replacement to recover. Ready telemetry samples remain available while the next sample prepares. Never feed UI completion timing into simulation decisions or send live World/USD readers to the worker. See the [publication contract](../../docs/architecture/runtime-authored-ui.md). Run the held-physics preparation and negative-result gate with the built production binary in both host modes: ```sh python3 scripts/api/test_runtime_ui_preparation.py --port 4374 --log target/ui-preparation-headless.log python3 scripts/api/test_runtime_ui_preparation.py --port 4375 --windowed --log target/ui-preparation-windowed.log ``` When one producer owns several surfaces, keep invalidation domains separate so continuous motion does not rebuild static authored topology. Use the existing authoritative stage revision for USD-derived membership and cache that membership plus static authored metadata such as program source facts and declared public-output names. Do not reread those facts at publication cadence, rescan all prims, or add a second revision/source registry. Keep simulation status, telemetry, outputs, and Rhai policy results live. Authored runtime-surface fields belong on a prim with `LunCoUiSchemaAPI`. Their live edits refresh exposure discovery through the stage revision; they do not require recreating scene entities or resetting simulation state. For camera status, Rust publishes the current camera fact and compact label through the generic exposure namespace. The shared `lunco-usd-bevy::camera_switch::camera_display_labels` resolver is also used by the picker, Camera menu, USD/entity trees, and Inspector: unique leaves stand alone, duplicate leaves gain nearest-owner context and then ancestors, generated hexadecimal/UUID-like owner suffixes are hidden, and an unavoidable normalized collision gets an ordinal. The full USD path remains the typed selection value and hover/diagnostic text. Rhai owns selection policy (`set_camera(name)`) and can read the fact with `get_exposure(...)`; HUI/CSS owns rendering. Camera status emits `CameraSelectionStatusChanged` after its camera/viewport lifecycle projection changes, and the exposure observer consumes that event. The UI is revision-gated. Do not add a Rhai `on_tick` loop, a timer poll, or a per-frame camera scan for this HUD. Rejected camera commands may use the shared warning toast. Camera admission findings come from one camera-contract state and project to structured runtime diagnostics and Recent Events. A valid operator camera keeps the viewport ready; unresolved director tracks are warnings until director control resumes. Recent Events presents camera findings as warnings, emitted directly as telemetry rather than runtime errors, so the generic warning observer never dispatches a toast. Keep camera messages out of the Camera menu. Register open egui dropdown bounds with `ScenePickGate` so option clicks do not fall through to the 3D scene. ### 2. Add the template and stylesheet Place application files under `assets/ui/`, or Twin files under its own `ui/`. The stable contract is: - HUI `<template>`, `<property>`, `<node>`, `<text>`, and `<button>`; - stable `id` values and `{property}` interpolation; - `on_press="callback_name"` for semantic actions; - Flair CSS-like layout and visual properties, custom properties, and `var(...)`. Declare every property that the bridge should write: ```html <template> <property name="title">Status</property> <property name="state">offline</property> <node id="status-root"> <text id="status-title">{title}</text> <text id="status-state">{state}</text> </node> </template> ``` Only bindings with `"css": true` also project `--ui-<property-name>` into CSS. Use that flag for authored CSS values, never labels, descriptions, or error messages. An omitted bindings map projects text only. Keep the manifest-owned outer rectangle separate from CSS-owned internal layout: ```css @import "ui/runtime_fonts.css"; #status-root { display: flex; flex-direction: column; gap: 6px; padding: 12px; background-color: var(--panel-background); } #status-state { color: var(--ui-state-color); } ``` Use the bundled font import for telemetry/status glyphs. Do not rely on Bevy's minimal `default_font` or a host-installed fallback. ### 3. Register bindings, gates, actions, and placement Add a surface entry to the application manifest or a Twin-owned JSON document with `kind: "lunco.runtime-ui-manifest.v1"`. Normal Twin loading discovers it through the shared JSON asset lifecycle. Twin entries replace matching shared surface IDs and add new ones; paths resolve from that scope's root. Use explicit `lunco://` references for shared imports. Do not compose a launcher-local copy of the application assets. Verify direct `OpenTwin`, same-Twin reopen and replacement with another Twin, including UI retirement and real button input. Example surface entry: ```json { "id": "mission-status", "template": "ui/status.html", "stylesheet": "ui/status.css", "namespace": "mission-status", "bindings": { "title": { "source": "title" }, "state": { "source": "state" } }, "actions": [ { "callback": "runtime_status_focus_moon", "action": "view.body.moon" } ], "visible_in_perspective": "sandbox_view", "interactive": true, "placement": { "mode": "window", "anchor": "top_right", "offset": [-16.0, 16.0], "width": 260.0, "height": 84.0 } } ``` Surface `id` values and callback names must be unique within the manifest. The loader rejects unknown fields, unsafe asset paths, and invalid geometry. Authored semantic actions are accepted and delivered to Rhai; do not add a Rust enum arm for each Twin, route, rover, or lander. Binding target names must be declared template properties. `map` translates exact rendered strings, which is useful for `true`/`false`, body ids, `display`, or CSS colors. It does not perform arithmetic or general expressions; publish a formatted presentation value when that is what the surface needs. Placement modes: - `viewport` fills the window; - `dock_panel` uses the workbench's authoritative `PanelRects` plus an inset; - `window` uses logical-point width/height, a corner/center anchor, and an offset. Only a `window` surface may set `draggable: true`. The primary-button drag stores a finite logical top-left override by stable surface id, clamps it to the live target after resize/DPI changes, and persists it in the active Twin's existing workbench workspace state. A primary-button double click removes the override and restores the authored anchor. The shipped `celestial-view` switcher is a draggable window with a top-centre authored default; Settings ▸ HUD also exposes a surface-specific reset that removes its per-Twin override. Manifest reconciliation prunes unknown surface ids, and `TwinClosed` clears the in-memory layout scope. Script-driven hint/objective/action cards belong to `lunco-workbench-guided-ui`. Omitting `hud` uses the calling script's card; unscripted API/REPL input uses a distinct command-context card. Several scenario/tool producers share a scene card by supplying the same explicit `hud` id, for example `hint("mission-controls", text)` and `guided_hud_actions("mission-controls", tool, hook, actions)`. The Twin is part of the identity; never choose a card by last writer, hook name, or label. `GuidedHudVisibility` supplies the host default (View in LunCoSim). `hud_perspectives(ids)` or `hud_perspectives(hud, ids)` declares a per-card allowlist; `[]` explicitly allows all. Unknown ids reject atomically. Scope covers the whole card before rendering or input creation. Switching perspective preserves content. Coach, spotlight, and recovery presentation remains separate. `SetGuidedHudActions` validates the endpoint and action ids, creates an explicit `ToolHookBinding` entity, and retires the prior binding. The UI carries that entity into `RunRhaiToolHook.binding`; the shared scripting queue validates its exact endpoint, registered owner, Twin, and scene generation at admission and again at execution. The immutable action row's widget identity includes its binding entity, so replacement also retires in-progress input. Empty actions retire the binding. Script removal immediately clears its card, including while the driver is paused or preparing. Script-instance retirement (including recompilation) clears implicit script cards; scene replacement clears all cards; `TwinClosed` clears that Twin's cards. All three despawn their leases. Stop hooks may clear existing content/actions idempotently; clears never create cards. New content, perspective declarations, and bindings reject during stop. HUI, dropdown, Escape, and native menu actions capture `RuntimeUiActionScope` at emission; validate it before interpreting the action, then retain the shared queue's execution-time scene check. Never resolve the next active Twin for an old event. Named cards are scene-owned, so callback tools can update them independently of a scenario host. `ClearGuidedOverlay` clears the calling script's card and the active coach/spotlight presentation, preserving other cards. Inspect the admitted scope, content, and visibility through `InspectGuidedHud`. Its optional `tour` report exposes the active coach's exact Twin/script owner, step and anchor. Script-opened scene menus retain the same captured scope while open and clear on scene transition or owning Twin close; stop hooks cannot open menus. Each popup has its own widget identity and child callback leases. Replacement or dismissal retires unselected leases; selection transfers one `Invocation` binding to the queue, which releases it on execution or rejection. HUD rows use `Surface` retention. Scene transitions retire queued tools even while the interpreter is preparing. Never leave a selected popup endpoint unowned. Run `python3 scripts/api/test_guided_ui_callbacks.py --port PORT` for the production popup/menu input, replacement, dismissal, reset, and Twin-replacement gate. Installed workbench menus bind provider-lifetime endpoints beneath a fresh installation root and the current input Twin/scene; disable input during scene transition. Selection creates a scene-bound invocation child. This keeps application menus reusable after scene reset while replacement revokes both held input and selected work. Every terminal admission/queue path releases the exact invocation lease, including overload with an omitted request owner. Nested deferred calls retain their typed caller context and capabilities through `CommandOrigin::Rhai`/`WorldScriptExecution`. Never rebuild a stopping caller as UI/REPL evaluation. The gate stops `guided_hud_teardown.rhai`, then inspects the completed authored verdict from both deferred queues. The wrapper uses temporary application-source fixtures for the native asset watcher and removes them after its owned runtime exits. Run `assets/scripting/tests/test_guided_hud.rhai` with `scripts/api/run_rhai_test.sh PORT FILE` in an owned windowed production session. It checks View/Build/Editor, independent teaching scope, preservation, and invalid scope/endpoint rejection. Capture screenshots to verify actual placement/input; headless presentation acknowledgements do not prove visibility. `assets/scenarios/tests/guided_hud_teardown.rhai` reports `HUD_TEARDOWN` after its host is stopped; it also publishes a script-owned teaching card with an explicit all-perspectives scope. Register the generic `hud_binding_probe.rhai` test library only in an isolated test Twin when exercising real pointer callbacks and Twin replacement; never persist test tools into a user's Twin. `interactive: true` enables input ownership for visible HUI controls that carry an authored `on_press` action. The runtime adds `Pickable` to those visible controls and feeds each computed Bevy UI rectangle into the existing `ScenePickGate`; it never registers the surface root or a full-window `viewport` rectangle. The active presentation camera is enabled for UI picking with Bevy's marker filter, so decorative retained nodes without `Pickable` pass