cluster-labeling

name	cluster-labeling
description	Label unsupervised segmentation clusters (k-means over geospatial embeddings) into a land-cover/land-use hierarchy by looking at high-res imagery. Use when assigning semantic labels to cluster rasters, relabeling/auditing an existing land-cover map, or running a per-cluster vision-judgment pass over satellite crops. AOI-agnostic engine; pair with an AOI pack (e.g. cluster-labeling-auroville).

Cluster labeling (engine)

Turn a cluster raster + high-res RGB basemap + a label hierarchy into a labeled map, by rendering exemplar crops per cluster, judging them against the hierarchy (Claude reads the images directly in-harness — no API key needed), and voting one label per cluster. Designed for iterative rounds with human feedback.

This skill is the site-agnostic engine. Concrete paths, the label hierarchy, domain visual signatures and geography priors live in an AOI pack skill — for Auroville that's cluster-labeling-auroville. Always load the AOI pack first.

Inputs (contract — this skill consumes, does not produce)

input	what	produced by (out of scope)
cluster raster	kNN_s42.tif, int cluster ids, nodata < 0, EPSG:4326	embedding + k-means pipeline (e.g. sibling alpha-bhu)
basemap	high-res RGB COG, same CRS/bounds	imagery downloader/stitcher (e.g. bun run download-tiles/stitch-tiles)
hierarchy	dotted-path label tree JSON	AOI pack
(optional) prior labels	old label raster + pixel-mapping.json	previous labeling pass — used only for the review cross-tab

If inputs are missing, stop and point at the producer; do not reimplement downloading/embedding/clustering here.

Procedure

Let AOI, SEG (cluster raster), BASE (basemap), CENTER (lon lat landmark) come from the AOI pack. Pick a RUN_DIR per round (e.g. …/vlm_label_k88/).

The rendering scripts need an environment this project does not declare. pyproject.toml has rasterio/numpy/shapely but NOT pillow or scipy, so a plain uv run on any of steps 1–2c dies on ModuleNotFoundError: No module named 'PIL'. Use:

uv run --no-project --with rasterio,numpy,pillow,scipy python …

The steps below show bare python for readability; every one of them means that line. Written here because it used to live only in a run directory's HANDOFF.md — gitignored, in no repo, and discarded with the run.

Do not stash it in a shell variable and call $UV script.py. This harness's Bash tool runs zsh, which does not word-split an unquoted parameter expansion, so $UV is looked up as one long command name and the step dies with command not found: uv run --no-project …. It reads like a missing uv rather than a shell difference, and it fails identically on every step. Write the command out, or use ${=UV} if you must have the alias.

1. Render exemplar crops (light tint by default):

   python .claude/skills/cluster-labeling/scripts/gen_exemplars.py RUN_DIR \
     --seg SEG --base BASE --cluster-ids ID... --exemplars E --window-m 200
   

   → RUN_DIR/{crops/,results.jsonl}, each record carrying "result": {} for the in-harness reader to fill. More exemplars (5–6) for coverage; bump only as needed for uniform clusters. --clusters N takes the N largest instead of explicit ids; --window-m 100 + 4× upscale for fine crown/species.

   It will not overwrite an existing results.jsonl — that file is what the run's cards, ledger and verdicts are keyed to. Render an additional batch into a run with --results results_bN.jsonl.

1b. Render the patch crop — the view that is about the cell, once results.jsonl exists:

python .claude/skills/cluster-labeling/scripts/gen_patch_crops.py RUN_DIR \
  --seg SEG --base BASE

→ RUN_DIR/crops/cNNN_eN_patch.jpg, window sized to the patch, boundary burned in, no tint, captioned with the cell's share of the frame and any upscale factor.

Add --masks for cNNN_eN_patch_mask.png beside each crop — an L-mode layer, 255 inside the cell and 0 outside, on the crop's exact pixel grid (~1 KB each). A review page consumes it as a CSS mask-image to tint the interior on demand, answering which pixels am I being asked about without baking the wash into the frame the texture is judged from. Safe to add to a finished run: an existing .jpg is not rewritten unless --force.

Not optional, and not a nicety. The fixed 200 m crop is mostly not the thing being judged, and the patch crop is what fixes that:

exemplar set	200 m crop: median share, under-10%	patch crop: median share, under-10%
332, over 120 cells (2026-08-21)	5.2%, 65%	20.7%, 2%
488, over 172 cells (2026-08-22)	17.6%, 38%	23.5%, 1%

Re-measure per run; do not quote these. The 200 m column moved by a factor of three between two runs of the same segmentation, and why is not established — it is not exemplars-per-cell, which was the obvious candidate and is wrong: the 2026-08-21 set is already 3 per cell, and restricting it to the first three changes neither figure. What the two rows do agree on is the direction and the size of the fix: the patch view takes the share of exemplars whose cell is a sliver of its own frame from 65% to 2%, and from 38% to 1%. Readers worked this out on their own before it existed: three of them spent 25 resize, 21 crop and 6 composite calls per pass rebuilding it by hand, which inflated transcripts past the point where they could be resumed and left each reader judging different images. Render it once for everyone, then keep the reader off image tooling so concurrence means what it claims to.

2. Locator maps (where each cluster sits — essential for dispersed clusters):

   python .claude/skills/cluster-labeling/scripts/gen_locator.py RUN_DIR \
     --seg SEG --base BASE --center LON LAT [--window-m 800] [--per-exemplar]
   

   → crops/cXXX_locator.jpg: cluster in cyan, exemplars circled and numbered, and each one's context window drawn as a box. --per-exemplar writes crops/cXXX_eN_locator.jpg instead — one image per exemplar with that exemplar as the subject (yellow) and its siblings dimmed. A surface showing one candidate per card wants that: "which numbered box is mine?" is a question the card should not be asking. --window-m must match the --window-m gen_context.py was run with, or the box points at a crop of another size.

2b. Whole-area overview (macro context for judging + macro-QA after):

python .claude/skills/cluster-labeling/scripts/gen_overview.py RUN_DIR \
  --seg SEG --base BASE

→ RUN_DIR/overview_basemap.jpg (downsampled whole basemap — read it before judging to see the regional layout / belts) and, once cluster_to_label.json exists, RUN_DIR/overview_labels.jpg (label choropleth + legend — re-run after aggregate as a macro-QA surface).

2c. Mid-scale context crops (the zoom between exemplar and locator):

python .claude/skills/cluster-labeling/scripts/gen_context.py RUN_DIR \
  --seg SEG --base BASE [--window-m 800] [--masks] [--force]

→ RUN_DIR/crops/cXXX_eN_ctx.jpg — the same centre as each exemplar at a wider window, magenta outline, 100 m scale bar, nothing tinted baked in. --masks additionally writes cXXX_eN_ctx_mask.png, an RGBA layer a page overlays as a plain <img> to fill the cell on demand — at 800 m the outline around a small cell is a hairline, and the fill is the only way to see which pixels are being asked about (maintainer, 2026-08-23). Existing crops are skipped unless --force, so a --masks run over a rendered round only adds the layers. Rendered from results.jsonl centres so indices match the exemplars one-to-one. Any class defined by its surroundings needs this — a bare field reads as fallow at 200 m and as a harvested plantation block at 800 m; "matrix between the buildings", "opening inside forest", and "strip following a road" are all invisible in a close crop.

2d. Prior-label cross-tab (only when a prior label raster exists):

python .claude/skills/cluster-labeling/scripts/gen_prior_labels.py RUN_DIR \
  --seg SEG --old OLD.tif --mapping pixel-mapping.json \
  [--authoritative water --freeze-share 0.5]

→ RUN_DIR/prior_labels.json — per cell, the distribution of prior-map classes over its pixels, plus freeze candidates for the classes the AOI pack names as reliable — and RUN_DIR/frozen_water.json, which is the freeze actually taking effect: gen_round_cards.py skips the cells listed there, and until 2026-08-22 this script only printed the candidates while that file had no generator at all. A run whose operator did not know to hand-write it carded the very cells the AOI pack says must never be re-judged. Feed the distribution to the readers as evidence and hold the frozen cells out of judging entirely.

2e. Geography lines — geo.txt, the per-exemplar compass sector, distance from CENTER, and cell size that ride on every card:

python .claude/skills/cluster-labeling/scripts/gen_geo.py RUN_DIR \
  --center LON LAT --seg SEG [--mapping SEG_mapping.json]

Not optional. cluster-reader.md tells the reader outright that each exemplar comes with a direction and distance; without this the cards carry geo: "" and the reader is promised an input it does not have. --mapping is gen_intersection.py's parentage JSON when the segmentation is an intersection raster — it supplies the from k88 cN clause, which is the same ambiguity corrections.md has to name its segmentation for.

3. Judge. For each cluster read its exemplar crops, its context crops and its locator map, plus the AOI pack's reference example crops. Apply the AOI pack's signatures + geography priors. Hierarchy-aware fallback: most specific label you're confident in, else the parent; allow uncertain.

   The verdict fields are in references/verdict-record.md and nowhere else. Read them from there; this step deliberately does not list them, because the copy that used to stand here went stale (it predated changed, represents_cluster, no_class_fits and mixed). Validate before anything consumes the verdicts:

   uv run --no-project python .claude/skills/cluster-labeling/scripts/check_verdict_contract.py \
     RUN_DIR --contract .claude/skills/cluster-labeling/references/verdict-record.md
   

   (There is no prompt.txt and, since 2026-08-22, no round brief either. prompt.txt's generator went with the Gemini path in a20a78f; BRIEF.md was deleted under worklist T91. What readers read is their agent definition plus the task prompt round_workflow.js builds — run dir, cards file, and blind: true|false. Every brief this pipeline wrote restated the reader's standing guidance and drifted from it, and its genuinely per-round content was warnings about renderer defects that are now fixed. Do not reintroduce one: a rule written in a brief reaches a single round and is discarded with the run dir, which is gitignored.)

   corrections.md is FROZEN for the duration of a round (worklist T95). It is one of the files defs_version hashes and it is append-only, so a line added while readers are running gives the batches that started after it a different stamp from the batches that started before — one round, two contracts, and the split is invisible in the output because every record still carries a stamp that looks fine on its own. Queue expert feedback arriving mid-round and append it after the last batch lands. The same hazard is why nothing else in the --defs list may be edited mid-round.

3a. Pick the harness — and know what the choice forecloses. Workflow is the default path (ruled 2026-08-22). Agent is for the occasional debrief round, when the process itself is what is under test — not for ordinary labeling throughput. Do not re-litigate this per round; the cost of the choice is the debrief, and a debrief is worth a round of its own rather than a tax on every round.

Two paths, and the difference is not cosmetic:

	Workflow + scripts/round_workflow.js	Agent + SendMessage
readers	one-shot agent(), deterministic, resumable	live sessions you can come back to
debrief	impossible — no continuation to quiz	available (step 3b)
use when	the contract is settled and you want throughput	the process itself is under test

Workflow({scriptPath: '.claude/skills/cluster-labeling/scripts/round_workflow.js',
          args: {runDir: RUN_DIR, cards: 'cards.json', blind: true,
                 batchPrefix: 'batch', batches: [[...ids...], ...]}})

A first round in a fresh run dir takes its card inventory from results.jsonl, since there are no previous verdicts to derive it from. gen_round_cards.py does that on its own and says FIRST ROUND: when it does; before 2026-08-22 it returned zero cards and exited 0, a silent no-op that reads as success and that four re-judge rounds never hit.

Generate the cards with the same batches first — the workflow hands reader i cards_bN.json, not the whole file:

python .claude/skills/cluster-labeling/scripts/gen_round_cards.py RUN_DIR \
  --batches '2,78,103;35,90,109'

A reader has no shell (see its tool list), so it cannot narrow the full file down itself, and handing it every cell loads its context with the whole round rather than its batch — which is what puts a reader past the size where it can be reached for step 3b. Pass cardsPerBatch: false only for a run whose cards were never split. A reader's session ends the moment its agent() returns, so choosing Workflow decides step 3b before you get there and cannot be undone by noticing later. Round 4 lost its debrief exactly this way, with the rule written down — at the destination, not here.

3b. Debrief the readers — only on the Agent path, and only as a second turn after the verdict files are on disk. Contract, fields and the fixed question template: references/reader-debrief.md. Generate the prompts mechanically (never hand-word them per reader — frequency across readers is the filter, and it only measures the process if everyone was asked the same thing):

uv run --no-project python .claude/skills/cluster-labeling/scripts/gen_debrief_prompts.py \
  RUN_DIR --verdicts 'batch_*.json' --out-prefix debrief

SendMessage each filled prompt to the reader that produced that batch — the moment that reader returns, before the contract check and before waiting for the others. A reader goes unreachable a few minutes after it falls idle, and the larger its transcript the faster; a real reader's is 20-25 MB. The measured ladder and what it cost is in references/reader-debrief.md. Then mine the tally:

uv run --no-project python .claude/skills/cluster-labeling/scripts/mine_debrief.py RUN_DIR

Reader self-report is the weakest evidence class here — it produces signal to adjudicate, never findings to adopt. Same standing as no_class_fits.

4. Aggregate (confidence-weighted vote → one label + agreement per cluster):

   python .claude/skills/cluster-labeling/scripts/aggregate.py RUN_DIR \
     --judgments 'RUN_DIR/batch_[0-9]*.json'
   

   → RUN_DIR/cluster_to_label.json (+ fills results.jsonl).

   Point it at the round's own batch files. There is no judgments.json, and creating one is the bug, not the fix. Readers write one array per batch and nothing merges them; until 2026-08-22 this step named a single judgments.json that no producer in the pipeline ever wrote, so the only file that could satisfy it was one an earlier round had left in the run dir. That is gate.py's T5 verbatim — sessions 22 and 23 paired round-4 verify records with round-3 labels, nothing errored, and the summary looked right. aggregate.py now takes globs through the same loader the gate uses, which also refuses overlapping inputs rather than double-weighting an exemplar's vote.

5. Review page:

   python .claude/skills/cluster-labeling/scripts/gen_review_html.py RUN_DIR \
     --seg SEG [--old OLD.tif --mapping pixel-mapping.json] \
     [--nbr-labels PRIOR_RUN/cluster_to_label.json]
   

   → RUN_DIR/review.html (cards: exemplars + locator + voted label + agreement

   • spatial-neighbor labels with boundary shares + optional old-label cross-tab; filters for low-agreement / differs-from-neighbours / disagrees-with-old; click to zoom). --nbr-labels supplies fallback labels for neighbor clusters not judged in this run (e.g. the parent round of an intersection raster). Open it; this is the human-feedback surface.

5b. Neighbor-pair check (for the flagged differs-from-neighbour clusters — the VLM, not any clustering-side signal, decides "visually identical"):

python .claude/skills/cluster-labeling/scripts/gen_nbr_pairs.py RUN_DIR \
  --seg SEG --base BASE

→ raw boundary-straddling crops (crops/cXXX_nbrYYY.jpg, cell magenta / neighbor cyan) from nbr_flags.json. Judge each: same cover across the boundary? → RUN_DIR/nbr_verdicts.json ([{cluster, nbr, same_cover, cover, confidence, note, img}]), then re-run step 5 with --nbr-verdicts to fold verdicts + pair crops into the cards. NOTE adoption has no fixed direction: "same cover" as often means the big neighbor is wrong at that spot as the small cell — the verdict names which.

6. Corrections loop. Keep an append-only RUN_DIR/corrections.md (one section per cluster, geo direction in the header, - fb: lines). State at the top which segmentation's ids the file uses, and repeat it on every section header. A run at kA x kB shares an id range with its parent raster — a child cell keeps its parent's id where it is that parent's largest remainder — so a bare c34 is ambiguous by number and resolvable only from content. Two readers hit this and resolved it by inference; a wrong resolution applies another cell's ground truth silently, and nothing downstream can detect it. Record user feedback, revise the round's batch verdict files in place, re-run steps 4–5. Route durable learnings to the right sink: AOI-specific (signature/geography/reference/label policy) → the AOI pack; methodology that would hold for any AOI → this skill (see Self-improvement).

Methodology rules (hard-won)

• Layer separation: embeddings cluster, vision labels. The embeddings' job ends when the cluster raster is produced. In the labeling pass, every qualitative question — including "is this cell visually identical to its neighbor?" — is answered by the VLM looking at imagery, never by reaching back into embedding distances or other clustering-side signals. (Corollary: "k-means kept them separate" is NOT evidence the covers differ.) Imagery/embedding snapshot-date mismatches are acknowledged and worked modulo — do not reintroduce embedding signals to compensate.
• Consult the whole-area overview, and use the label choropleth as macro-QA. Read overview_basemap.jpg before judging — it shows large uniform regions and how cover types are arranged (the visual form of the geography priors, and the neighbor context the per-cluster crops can't give). After aggregating, scan overview_labels.jpg: a real land-cover class clusters spatially (coherent belts ⇒ trustworthy); a default/uncertainty artifact sprays as confetti across the map (suspect). This catches systematic errors that per-cluster review can't — they're only visible at whole-map scale (e.g. an over-applied "grassland" default showing as scattered specks while cashew forms a clean belt).
• The goal is a correct label per CELL, not per cluster. The cluster is a prior that propagates one label across visually-similar cells cheaply — it is not the unit of truth and is overridable. Where a cluster is pure, label it wholesale; where it isn't, drop to a finer unit (sub-cluster / tile / carve-out) and get each piece right. Better a correct label on a 10×10 m square than a tidy single label on a mixed cluster.
• Carve out confident minorities. When review (an exemplar, or a spotted sub-region like a small coconut grid inside a cashew cluster) shows a distinct cover different from the cluster majority, mask that sub-region out and relabel it — don't let the vote bury it. Mechanisms: intersect the cluster with a finer k, or carve the visually-similar connected patch around the exemplar. (This is the actionable form of "impurity ⇒ go finer".)
• Resolve low-confidence cells by spatial context, not in isolation. A ~0.4-confidence cell is frequently wrong on its own, but an adjacent cell/cluster of the same visual type is often confidently (and correctly) labeled — adopt the confident neighbor's label. Show neighbors in review and let confident regions propagate into ambiguous ones. Implemented: gen_review_html.py computes raster adjacency, prints each cluster's dominant neighbors + labels on its card, and flags/filters clusters whose dominant neighbor's label is unrelated and holds ≥ 25% of the boundary — the "cell visually identical to its surroundings" case.
• Don't relabel cells the old map gets correct (the AOI pack names which — e.g. water): freeze them and spend judgment only where the relabel adds value. A policy nobody can act on is not a policy: for two rounds this project said "inherit water" while no reader was ever shown what the prior map claimed, and dry-season tank beds were duly judged from the photo alone — four were guessed right at ~0.45 confidence and one became . Compute the cross-tab (step 2d), freeze the authoritative class, and pass the rest down as evidence.

Self-improvement (living skill)

This skill is a procedure, not a frozen product — every round should leave it better than it found it. Two grades of learning, two speeds:

• User-confirmed corrections fold in immediately. Feedback recorded in RUN_DIR/corrections.md is ground truth from the maintainer — route it before the round closes: AOI-specific (signature / geography / reference / label policy) → the AOI pack; AOI-agnostic methodology (a new rule, failure mode, QA surface) → this file's Methodology rules. A round that ends with an unrecorded surprise isn't done.
• Self-observed friction accumulates first. A one-session hunch (awkward step, suspected pattern, missing view) gets logged — a - friction: line in RUN_DIR/corrections.md, or the AOI pack's State section if cross-round — not an immediate edit here. Same friction in a second round ⇒ promote it. Log first, don't filter; a pattern only emerges from honestly recorded one-offs.
• Improvise views; promote on reuse. The scripted views (crops, locators, overview, review page) are a floor, not a ceiling. When a judgment stalls for want of evidence, compile a new task-shaped view on the spot (recrop, upscale, tint-free, composite, intersection — whatever the question needs); that's how every current script started. An ad-hoc view that earns a second use graduates into scripts/ and gets a line under Files and Procedure.

Files

• references/verdict-record.md — the verdict record's fields and their meaning, single-sourced. Step 3 and every harness read them from here; nothing restates them. scripts/check_verdict_contract.py --contract validates against it.

• references/reader-debrief.md — the reader debrief: its four channels, the two hard rules about when and who, and the fixed question template. Reachable only from the Agent + SendMessage harness (step 3a).

• references/convergence-loop.md — the multi-pass settle/split loop that wraps this procedure: per-cluster state machine, the settle gate and its criteria, disagreement triage, what is built vs not, and the open decisions. Load it when running a pass after the first, or tuning the gate.

• scripts/gate.py — ledger + settle gate: judgments (+ verify, prior, neighbour) → ledger.json with a state per cluster, stamped with defs_version and the --defs paths that stamp was computed over.

  --defs is these seven files, and the list lives here. It was eight until 2026-08-22, when T91 deleted the round brief; the reason it is written down in the skill is that the canonical list previously existed only in a run dir's gitignored HANDOFF.md, so retiring that run would have retired the list — the same trap that nearly took $UV with it.

  #	file
  1	AOI pack references/class-definitions.md
  2	AOI pack references/land-cover.json
  3	AOI pack SKILL.md
  4	.claude/agents/cluster-reader.md
  5	references/verdict-record.md
  6	RUN_DIR/corrections.md
  7	scripts/round_workflow.js

  Never quote a stamp value from prose — it is stale the next time anyone edits a reader-facing file, and the person it misleads is the one who trusted the document. Recompute with check_defs_drift.py.