Download and process lunar geo assets (DEMs, ortho/slope/shade maps, normal maps) with lunco-assets — Assets.toml entries, ROI cropping, terrain layer wiring in USD, quality presets, bake keys. Use when adding a terrain site to a twin, baking layer maps, or debugging the asset pipeline.
Download and process lunar geo assets (DEMs, ortho/slope/shade maps, normal maps) with lunco-assets — Assets.toml entries, ROI cropping, terrain layer wiring in USD, quality presets, bake keys. Use when adding a terrain site to a twin, baking layer maps, or debugging the asset pipeline.
Geo assets: download & process lunar terrain for a Twin
The pipeline is crates/lunco-assets in this repo. Pure Rust — no GDAL.
Sources may be GeoTIFF or PDS3 .IMG (attached or detached .LBL;
src/pds_img.rs); polar-stereographic products are refused loudly because the
crop affine is equirectangular-only. Use the target Twin's own Assets.toml as
the worked example and inspect its current scene before wiring outputs.
Quick commands (run from this repo's root)
cargo run -p lunco-assets -- list --twin <TWIN>
cargo run -p lunco-assets -- download --twin <TWIN> # ALL entries (can be GBs)
cargo run -p lunco-assets -- download --twin <TWIN> -a <key> # one entry
cargo run -p lunco-assets -- process --twin <TWIN> -a <key> --quality coarse|good
<TWIN> = a folder holding Assets.toml + twin.toml. -a <key> = the
[section] name in its Assets.toml. The same entries appear in-app under
Settings ▸ Downloadable data and the Twin inspector once the Twin is open
(scanned on open). Asset-consuming domains begin only after the asset owner
publishes TwinAssetMounted; they must not depend on observer registration
order. If a declared dataset is missing, the interactive app asks
which entries to download before terrain generation; nothing downloads until
the user confirms there or uses a CLI run. Closing a Twin retires its dataset
rows under the shared download/process commit barrier and cancels their
cooperative tasks before another Twin can reuse the authority. A failed mount
or poisoned lifecycle lock is surfaced as an error, never reported as a
successful asset postcondition. quarters
(floor 64) for a seconds-fast quick-start bake; re-run
with (default) for full res.
--quality coarse
target_resolution
good
Downloads use the shared download section in the user settings file
(lunco-settings::DownloadSettings): attempts include the first request,
delays are exponential and capped, and a failed body read resumes from the
received prefix with HTTP Range when the source supports it. The CLI and the
interactive window therefore have one policy and one cache/path resolver.
Where files live (cache resolution)
Shared cache — the OS-global cache (~/.cache/lunco on Linux,
~/Library/Caches/lunco on macOS, %LOCALAPPDATA%\\lunco on Windows).
LUNCOSIM_CACHE remains an explicit CI/custom-install override. Every
worktree and Twin therefore shares one pool of regenerable data (MSL,
textures, ephemeris, downloaded sources).
Twin cache — <TWIN>/.cache. A Twin's default-owned downloads land
beside the Twin. twin:// reads resolve <twin>/<rel> first, then
<twin>/.cache/<rel>, then the global cache <cache>/<rel>.
shared = true on an entry sends its write to the global pool instead
(<cache>/sources/<sha256(url)[..16]>/<basename>) — one download per URL,
reused by every twin and worktree. Use it when several Twins intentionally
share a multi-GB upstream product; a Twin that must be self-contained should
set shared = false.
Raw downloads: entries without dest land in
<owner cache>/sources/<url-hash>/<basename> — owner being the twin
(--twin) or the shared cache (crate manifest). Author dest only when a
file must sit at a specific path; it is then resolved against that same
owner cache.
Baked outputs (output_root = "twin"): inside the twin at output,
where the scene's demSource/layer attrs expect them. Per-twin, always.
Process kinds (in [key.process])
kind
Input
Output
dem
DTM (GeoTIFF/.IMG)
<output>/materials/textures/heightmap.tif — square float32, georef in tags. output is a FOLDER; scenes reference it as demSource = @terrain/<site>@
Find the product: https://data.lroc.im-ldi.com/lroc/view_rdr/NAC_DTM_<SITE>;
files under https://pds.lroc.im-ldi.com/data/LRO-L-LROC-5-RDR-V1.0/LROLRC_2001/DATA/SDP/NAC_DTM/<SITE>/.
Read its .LBL: MAP_PROJECTION_TYPE (EQUIRECTANGULAR → processable;
POLARSTEREOGRAPHIC → download-only entry, no [*.process]),
MAP_SCALE → pixel_scale_m, MIN/MAXIMUM_LATITUDE +
EASTERNMOST/WESTERNMOST_LONGITUDE → the four src_* fields.
Label longitudes are 0–360 °E — author center_lon in the same
convention. Never trust CENTER_LONGITUDE (body-frame quirk).
sha256 = "" on first download → the tool prints the hash; paste it in.
PDS3 .IMG sources declare extent/scale in their label — src_* may be
omitted (an authored manifest extent wins when all four are set).
Wiring maps as terrain layers (USD)
Maps bind through a stock UsdShade Material network — the only authoring
path. Bind the Terrain prim to a Material, whose surface output connects to a
Shader carrying one asset inputs:<role>_map + float inputs:weight_<role>
per layer. Inspect an existing terrain scene in the target Twin for its exact
prim paths before adding a new network:
The dem child layer also owns the physics collider-ring lattice. Author these
beside windowM, targetRes, lodViz, and colliderRing when a Twin needs a
non-default contract:
For an authored rocks layer, lunco:layer:regionM is an optional
half-extent in metres: omit it or leave it at 0 to cover the whole composed
terrain; author a positive value only when a near-field scope is intentional.
lunco:layer:density is per hectare, and the rendering-quality profile owns
the total instance cap.
int lunco:layer:colliderDepth = 8
int lunco:layer:colliderResolution = 49
These values are copied into the typed terrain-generation request and used by
native, worker, GUI, and headless physics. They are deliberately independent
of RenderingQualitySettings, camera-driven visual LOD, and targetRes; a
graphics preset must never change collider tile count or resolution.
Asset paths are scene-root-relative and resolve through twin://, so they
travel with the twin. The generic USD shader projection walks
material:binding → Material → outputs:surface.connect → Shader and publishes
one ShaderLook; the terrain source reconciler derives the typed roles from that
same look. Roles are albedo, mineral, surface (packed R=rough G=AO B=rockDens),
and normal.
Every inputs:* is a live-tunable, journaled knob (networked, undoable) and
the network is inspectable in usdview/Blender.
CONNECTED map inputs are skipped — a connected port is fed by a producer
node (doc 18 Tier B), not an authored file.
mineral composites UNLIT after lighting, so a slope/classification
drape stays readable inside shadow — its entire job.
The authored shader source and maps bind on both static and streamed terrain
through the same ShaderLook; streamed LOD tiles add only their CDLOD
geometry inputs, and the derived bake fills slots an authored map left empty.
The runtime derived bake is optional visual refinement after the DEM ground
is ready. Its effective resolution is bounded by a static terrain's authored
visual target, so a low-resolution static product does not pay for an
invisible high-resolution map. The task is cancelled at scene teardown or
when its liveness bound expires; the terrain remains usable and the status
bus reports the terminal warning. This refinement status is separate from
terrain tile streaming, so it cannot hide tile progress or make a presentable
ground scene wait for an optional map.
For multi-site scenes, author these inputs inside a terrain variant and
verify with
cargo run -p lunco-usd --example variant_probe -- <scene.usda>.
Node-graph authoring is outside this asset pipeline. Read the current
multi-domain architecture before introducing a new graph owner.
Caching & staleness
Downloads skip when the resolved file exists with matching sha256.
Bakes stamp a .bakekey =
sha256(source bytes ‖ effective config ‖ PIPELINE_VERSION) beside each
output; a matching stamp skips the bake before the expensive decode.
Changing the source, ROI, --quality, or bumping PIPELINE_VERSION
(src/process.rs) rebakes exactly what changed. Never time-based.
The dataset registry treats that stamp as the completion boundary: a DEM
output directory without .bakekey is partial and remains downloadable/
processable, even if the directory itself exists.
Processing roots are strict (cache, twin, or assets); an unknown root or
missing required owner fails visibly. Processing uses a unique staging output
and an atomic commit barrier, so cancellation cannot publish stale terrain.
The terrain derived bake keys through the oracle's canonical surface identity
and does not re-hash the full DEM for every request.
A queued DEM build is an indeterminate state until its owner admits a task;
do not show a numeric percentage for that scheduler hand-off. Phase changes
are discrete status events and active work uses the existing progress entry.
Baked artifacts and the twin cache are gitignored by policy
(terrain/*/materials/, .bakekey stamps, .cache/) — never commit them.
Gotchas
*_50CM/*_2M.IMG companions are ORTHOPHOTOS (brightness), never
elevation — kind = "map", never kind = "dem".
Confirm the DEM datum and the scene's celestial-body radius before combining
terrain elevations with orbital or body-fixed coordinates. Do not encode a
product-specific radius correction in the asset pipeline.
Heights are absolute body-datum metres: prims on a surface must be
authored at the DEM's own elevation.
The runtime DEM reader requires square rasters; keep the scene's
windowM/targetRes in step with the manifest ROI.
Optional QGIS/GDAL extras (custom-sun hillshade, slope ramps, contours) are
external to lunco-assets; verify that the Twin supplies and documents its
own tooling before invoking it.