rerun-data-model

name	rerun-data-model
description	How raw multimodal robot data maps onto the Rerun data model. Read FIRST, before modeling or converting a dataset. Resolves the entity-vs-component, property-vs-component-vs-layer, and static-vs-temporal decisions, then points at the mechanism skills (rerun-chunk-processing and the importer skills rerun-mcap, rerun-urdf, rerun-parquet, rerun-lerobot) for the how.
user_invocable	true
allowed-tools	Read, Grep, Bash, WebFetch

Rerun data model

The hard part of ingesting a dataset is the modeling decision, not the API call. Get the model right and any mechanism works; get it wrong and queries, views, and training all break. This skill is just the decisions. For mechanism details see rerun-chunk-processing (pipeline mechanics) and the importer skills it routes to: rerun-mcap,rerun-urdf,rerun-parquet, rerun-lerobot. For exact signatures, the docs at rerun.io/docs/concepts/logging-and-ingestion.

Before writing conversion code, fill in the mapping table below. It is the design, and a human can review it in seconds.

The model

Dataset → Segment → Layer → Recording → Entity → Component → Chunk

• Entity = a thing, named by a path (/robot/arm/camera). Component = one typed field on it (positions, image, a scalar). Archetype = a standard bundle of components that log and render together (Points3D, Image, Transform3D).
• Every value you log sits on two axes: where (entity path + component) and when (which timeline, or static = all time).
• Segment = one episode; on registration its recording_id becomes the segment_id. Layer = an extra .rrd on top of a segment. It attaches by matching the segment's recording_id and nothing else: that shared id is the whole layering mechanism.

The decisions

Entity vs component on an existing entity New entity if it has its own spatial frame (Transform3D/Pinhole), its own annotation context, or should be shown/cleared/shared independently (each robot link, each camera, each sensor). Same entity + extra component for auxiliary data at the same instances (per-point confidence). Use AnyValues for non-standard fields.

Property vs component vs layer (the most common confusion)

• Component: per-timestamp signal on the timeline (joint angle, image).
• Segment property: one-per-episode metadata for catalog filter/search (operator, robot, site, task, date). Not per timestamp.
• Layer: a whole derived .rrd over a base segment (FK transforms, point clouds, gripper state, labels, quality scores). Queryable as if part of base.

Static vs temporal Static belongs to all timelines and shadows any temporal value of the same component on the same entity. Use it for invariants (calibration, coordinate frames, robot meshes, annotation context, a video asset). Never make per-frame data static.

Which timeline A timestamp timeline (ns since epoch) for cross-sensor clock alignment, a sequence timeline for frame/ordinal alignment; stamp on both when useful. Do not resample to a common rate. Latest-at reconciles multi-rate streams at query time by holding each component's last sample (no interpolation).

Base vs layer Base = faithful conversion of the raw streams, nothing computed. Layer = anything derived (FK from URDF + joint states, clouds from depth + intrinsics). Keep them separate .rrds.

The mapping table (produce before coding)

Source (topic/column/key)	Entity path	Archetype	Component(s)	Timeline	Static/temporal	Base/layer	Property?
mcap /joint_states	/robot/<joint>	Scalars	scalars	sensor_time	temporal	base	no
cam0/color.mp4	/camera/cam0/video	AssetVideo+VideoFrameReference	asset+refs	video_time	asset static, refs temporal	base	no
calibration.json	/camera/cam0	Pinhole (+Transform3D extrinsics)	image_from_camera	—	static	base	no
URDF + joints (computed)	/robot/<link>	Transform3D	translation/rotation	sensor_time	temporal	layer	no
episode.json operator	segment	—	—	—	—	—	yes

Patterns worth knowing

• Transforms / FK trees: log a Transform3D per link entity; it relates to the parent path and composes down the tree. (Named CoordinateFrame + child_frame/parent_frame only when topology must be a flexible graph.)
• Cameras: extrinsics (Transform3D) + intrinsics (Pinhole) on the camera entity, image/depth as children so they inherit the projection.
• Video: VideoStream for raw H.264/H.265 samples.
• Columnar ingest: for existing datasets use send_columns, not a per-row log loop. It adds no automatic timelines, so pass every timeline you want.

Gotchas that cause real failures

1. Component columns come back as ListArray in queries: index [0]/[0][0] (0-based DataFrame, 1-based SQL). See rerun-catalog-queries.
2. A layer must share the segment's recording_id, or it won't attach. application_id is discarded on registration.
3. send_columns/send_chunks add no log_time/log_tick; only the timelines you pass exist.
4. Static shadows all temporal data of that component for all time; static is overwritten in the viewer but every write stays on disk until rerun rrd optimize.
5. One Transform3D/Pinhole relation per frame pair; logging the same relation on a second entity is rejected.
6. Entity paths are not file paths (.. is meaningless, __ is reserved).