geo-assets

name	geo-assets
description	Download and process lunar geo assets (DEMs, ortho/slope/shade maps, normal maps) with lunco-assets — Assets.toml entries, ROI cropping, terrain layer wiring in USD, quality presets, bake keys. Use when adding a terrain site to a twin, baking layer maps, or debugging the asset pipeline.

Geo assets: download & process lunar terrain for a Twin

The pipeline is crates/lunco-assets in this repo. Pure Rust — no GDAL. Sources may be GeoTIFF or PDS3 .IMG (attached or detached .LBL; src/pds_img.rs); polar-stereographic products are refused loudly because the crop affine is equirectangular-only. Use the target Twin's own Assets.toml as the worked example and inspect its current scene before wiring outputs.

Quick commands (run from this repo's root)

cargo run -p lunco-assets -- list     --twin <TWIN>
cargo run -p lunco-assets -- download --twin <TWIN>            # ALL entries (can be GBs)
cargo run -p lunco-assets -- download --twin <TWIN> -a <key>   # one entry
cargo run -p lunco-assets -- process  --twin <TWIN> -a <key> --quality coarse|good

<TWIN> = a folder holding Assets.toml + twin.toml. -a <key> = the [section] name in its Assets.toml. The same entries appear in-app under Settings ▸ Downloadable data and the Twin inspector once the Twin is open (scanned on open). Asset-consuming domains begin only after the asset owner publishes TwinAssetMounted; they must not depend on observer registration order. If a declared dataset is missing, the interactive app asks which entries to download before terrain generation; nothing downloads until the user confirms there or uses a CLI run. Closing a Twin retires its dataset rows under the shared download/process commit barrier and cancels their cooperative tasks before another Twin can reuse the authority. A failed mount or poisoned lifecycle lock is surfaced as an error, never reported as a successful asset postcondition. --quality coarse quarters target_resolution (floor 64) for a seconds-fast quick-start bake; re-run with good (default) for full res.

Downloads use the shared download section in the user settings file (lunco-settings::DownloadSettings): attempts include the first request, delays are exponential and capped, and a failed body read resumes from the received prefix with HTTP Range when the source supports it. The CLI and the interactive window therefore have one policy and one cache/path resolver.

Where files live (cache resolution)

• Shared cache — the OS-global cache (~/.cache/lunco on Linux, ~/Library/Caches/lunco on macOS, %LOCALAPPDATA%\\lunco on Windows). LUNCOSIM_CACHE remains an explicit CI/custom-install override. Every worktree and Twin therefore shares one pool of regenerable data (MSL, textures, ephemeris, downloaded sources).
• Twin cache — <TWIN>/.cache. A Twin's default-owned downloads land beside the Twin. twin:// reads resolve <twin>/<rel> first, then <twin>/.cache/<rel>, then the global cache <cache>/<rel>.
• shared = true on an entry sends its write to the global pool instead (<cache>/sources/<sha256(url)[..16]>/<basename>) — one download per URL, reused by every twin and worktree. Use it when several Twins intentionally share a multi-GB upstream product; a Twin that must be self-contained should set shared = false.
• Raw downloads: entries without dest land in <owner cache>/sources/<url-hash>/<basename> — owner being the twin (--twin) or the shared cache (crate manifest). Author dest only when a file must sit at a specific path; it is then resolved against that same owner cache.
• Baked outputs (output_root = "twin"): inside the twin at output, where the scene's demSource/layer attrs expect them. Per-twin, always.

Process kinds (in [key.process])

kind	Input	Output
dem	DTM (GeoTIFF/.IMG)	<output>/materials/textures/heightmap.tif — square float32, georef in tags. output is a FOLDER; scenes reference it as demSource = @terrain/<site>@
map	co-registered raster (ortho .IMG, _SHADE/_SLOPE/_CLRGRAD .TIF)	8-bit RGB PNG at output (a FILE). Gray sources get a 1–99 percentile stretch
normalmap	DTM	DEM-local ENU normal PNG (RGB = n*0.5+0.5, decoded by the shared terrain-surface shader kernel)
texture	any image	resized PNG (non-geo default)
gltf	.glb	Bevy-clean .glb (needs npx)

Shared ROI fields: center_lat, center_lon, window_m, target_resolution = [n, n], pixel_scale_m, src_min/max_lat, src_min/max_lon, frame = "MOON_ME", output_root = "twin".

Adding a new territory to a twin

1. Find the product: https://data.lroc.im-ldi.com/lroc/view_rdr/NAC_DTM_<SITE>; files under https://pds.lroc.im-ldi.com/data/LRO-L-LROC-5-RDR-V1.0/LROLRC_2001/DATA/SDP/NAC_DTM/<SITE>/.
2. Read its .LBL: MAP_PROJECTION_TYPE (EQUIRECTANGULAR → processable; POLARSTEREOGRAPHIC → download-only entry, no [*.process]), MAP_SCALE → pixel_scale_m, MIN/MAXIMUM_LATITUDE + EASTERNMOST/WESTERNMOST_LONGITUDE → the four src_* fields. Label longitudes are 0–360 °E — author center_lon in the same convention. Never trust CENTER_LONGITUDE (body-frame quirk).
3. Pick center_lat/lon (the POI), window_m (scene size), target_resolution ≈ window_m / native m-per-px (square).
4. sha256 = "" on first download → the tool prints the hash; paste it in.
5. PDS3 .IMG sources declare extent/scale in their label — src_* may be omitted (an authored manifest extent wins when all four are set).

Wiring maps as terrain layers (USD)

Maps bind through a stock UsdShade Material network — the only authoring path. Bind the Terrain prim to a Material, whose surface output connects to a Shader carrying one asset inputs:<role>_map + float inputs:weight_<role> per layer. Inspect an existing terrain scene in the target Twin for its exact prim paths before adding a new network:

def Xform "Terrain" ( prepend apiSchemas = ["LunCoTerrainAPI"] )
{
    string lunco:assetMode = "layered"
    rel material:binding = </Traverse/Looks/TerrainLook>
    # … dem/overzoom/rocks layers …
}

def Scope "Looks"
{
    def Material "TerrainLook"
    {
        token outputs:surface.connect = </Traverse/Looks/TerrainLook/Surface.outputs:surface>

        def Shader "Surface"
        {
            uniform asset info:wgsl:sourceAsset = @lunco://shaders/terrain_layered.wgsl@
            asset inputs:albedo_map  = @terrain/<site>/materials/textures/ortho.png@
            float inputs:weight_albedo = 1.0
            asset inputs:normal_map  = @terrain/<site>/materials/textures/normal.png@
            float inputs:weight_normal = 0.5
            asset inputs:mineral_map = @terrain/<site>/materials/textures/slope.png@
            float inputs:weight_mineral = 0.0   # raise for a classification drape
        }
    }
}

Physics parameters for a DEM generator

The dem child layer also owns the physics collider-ring lattice. Author these beside windowM, targetRes, lodViz, and colliderRing when a Twin needs a non-default contract:

For an authored rocks layer, lunco:layer:regionM is an optional half-extent in metres: omit it or leave it at 0 to cover the whole composed terrain; author a positive value only when a near-field scope is intentional. lunco:layer:density is per hectare, and the rendering-quality profile owns the total instance cap.

int lunco:layer:colliderDepth = 8
int lunco:layer:colliderResolution = 49

These values are copied into the typed terrain-generation request and used by native, worker, GUI, and headless physics. They are deliberately independent of RenderingQualitySettings, camera-driven visual LOD, and targetRes; a graphics preset must never change collider tile count or resolution.

Asset paths are scene-root-relative and resolve through twin://, so they travel with the twin. The generic USD shader projection walks material:binding → Material → outputs:surface.connect → Shader and publishes one ShaderLook; the terrain source reconciler derives the typed roles from that same look. Roles are albedo, mineral, surface (packed R=rough G=AO B=rockDens), and normal.

• Every inputs:* is a live-tunable, journaled knob (networked, undoable) and the network is inspectable in usdview/Blender.
• CONNECTED map inputs are skipped — a connected port is fed by a producer node (doc 18 Tier B), not an authored file.
• mineral composites UNLIT after lighting, so a slope/classification drape stays readable inside shadow — its entire job.
• The authored shader source and maps bind on both static and streamed terrain through the same ShaderLook; streamed LOD tiles add only their CDLOD geometry inputs, and the derived bake fills slots an authored map left empty.
• The runtime derived bake is optional visual refinement after the DEM ground is ready. Its effective resolution is bounded by a static terrain's authored visual target, so a low-resolution static product does not pay for an invisible high-resolution map. The task is cancelled at scene teardown or when its liveness bound expires; the terrain remains usable and the status bus reports the terminal warning. This refinement status is separate from terrain tile streaming, so it cannot hide tile progress or make a presentable ground scene wait for an optional map.
• For multi-site scenes, author these inputs inside a terrain variant and verify with cargo run -p lunco-usd --example variant_probe -- <scene.usda>.

Node-graph authoring is outside this asset pipeline. Read the current multi-domain architecture before introducing a new graph owner.

Caching & staleness

• Downloads skip when the resolved file exists with matching sha256.
• Bakes stamp a .bakekey = sha256(source bytes ‖ effective config ‖ PIPELINE_VERSION) beside each output; a matching stamp skips the bake before the expensive decode. Changing the source, ROI, --quality, or bumping PIPELINE_VERSION (src/process.rs) rebakes exactly what changed. Never time-based.
• The dataset registry treats that stamp as the completion boundary: a DEM output directory without .bakekey is partial and remains downloadable/ processable, even if the directory itself exists.
• Processing roots are strict (cache, twin, or assets); an unknown root or missing required owner fails visibly. Processing uses a unique staging output and an atomic commit barrier, so cancellation cannot publish stale terrain.
• The terrain derived bake keys through the oracle's canonical surface identity and does not re-hash the full DEM for every request.
• A queued DEM build is an indeterminate state until its owner admits a task; do not show a numeric percentage for that scheduler hand-off. Phase changes are discrete status events and active work uses the existing progress entry.
• Baked artifacts and the twin cache are gitignored by policy (terrain/*/materials/, .bakekey stamps, .cache/) — never commit them.

Gotchas

• *_50CM/*_2M .IMG companions are ORTHOPHOTOS (brightness), never elevation — kind = "map", never kind = "dem".
• Confirm the DEM datum and the scene's celestial-body radius before combining terrain elevations with orbital or body-fixed coordinates. Do not encode a product-specific radius correction in the asset pipeline.
• Heights are absolute body-datum metres: prims on a surface must be authored at the DEM's own elevation.
• The runtime DEM reader requires square rasters; keep the scene's windowM/targetRes in step with the manifest ROI.
• Optional QGIS/GDAL extras (custom-sun hillshade, slope ramps, contours) are external to lunco-assets; verify that the Twin supplies and documents its own tooling before invoking it.