nanodevice-flakedetect-align

name	nanodevice_flakedetect_align
description	Register source microscope images to the full_stack coordinate system using SIFT (same-substrate) or Chamfer+DE (cross-substrate) alignment. Use when aligning bottom_part, top_part, or other source images to the full_stack reference image for van der Waals stack detection.

nanodevice_flakedetect_align — Image Alignment

Register source microscope images to the full_stack target coordinate system.

• SIFT path: Same-substrate images (e.g., bottom_part → full_stack). Fast, automatic.
• Chamfer path: Cross-substrate images (e.g., top_part on PDMS → full_stack on SiO2). Requires agent rotation selection.

Prerequisites

• Conda env instrMCPdev with opencv, numpy, scipy, scikit-learn
• Source images and full_stack reference image
• All scripts: ${PYTHON_PATH:-conda run -n instrMCPdev python} <script>

---

Agent Workflow

Runs fully autonomously except for one mandatory pause: rotation selection after the coarse sweep.

Step-by-Step

1. Determine alignment type
   ├─ Same-substrate? → Run sift_align.py → DONE (if ≥20 inliers)
   └─ Cross-substrate? → Continue to step 2

2. Run source_contour.py [--mirror]
   → View 01_source_contour.png: contour must trace the full flake boundary.

3. Run footprint.py [--mirror] [--bottom <bottom_part>]
   → Use --bottom when bottom_part image is available (diff mode, preferred).
   → *** CRITICAL: Verify footprint before proceeding ***
   → View 03_footprint_candidates.png — it shows multiple candidates side by side.
   → Compare EACH candidate against the source contour shape from step 2.
   → The default candidate (#1) is often WRONG — it may grab debris/satellite
     flakes instead of the PDMS stamp. Candidates #2 or #3 are often better.
   → If 04_footprint_grabcut.png does NOT match the source flake shape:
     Re-run with --candidate-rank 2 (or 3). Do NOT proceed with a bad footprint.
   → **Do NOT rely on shape_distance alone to pick candidates.** A candidate with
     slightly worse shape_distance may produce much better IoU after sweep+refine
     because the sweep optimizes position, rotation, AND scale. When all candidates
     have shape_distance > 0.5 (none clearly good), run sweep+refine on at least
     the top 2 candidates and compare final IoU.

4. Run sweep.py
   → Produces candidate overlay images

5. *** PAUSE: Select rotation ***
   View 05_sweep_grid.png and individual candidate_NN.png files.
   Pick the candidate where the contour best matches the flake.
   IGNORE cost ranking — the lowest cost is often wrong.

6. Run refine.py --rot-hint <degrees>
   → **Runtime**: refine.py takes 10-15 minutes on 2-CPU machines (differential
     evolution optimizer).
     **MANDATORY EXECUTION METHOD**: Run refine.py as a FOREGROUND BLOCKING
     command with a long timeout. Use the Bash tool with timeout=1200000
     (20 minutes). Example:
       Bash(command="${PYTHON_PATH:-conda run -n instrMCPdev python} .../refine.py ...", timeout=1200000)
     Do NOT use run_in_background=true. Do NOT launch it as a background
     process with &. Do NOT poll with sleep loops. Do NOT check for output
     files in a loop. Just run the single blocking command and wait for it
     to return. The Bash tool will hold until the process exits or the
     timeout is reached.
   → Check metrics against acceptance thresholds (see below)
   → If accepted: DONE. warp_top.npy is ready.
   → If FAILED: Go to step 7.

7. *** RETRY LOOP (max 2 retries) ***
   NEVER retry refine.py with the same footprint. Fix the INPUT first.

   Retry 1: Re-run footprint.py with --candidate-rank 2
            → then sweep.py → select rotation → refine.py

   Retry 2: Re-run footprint.py with --candidate-rank 3 or --n-clusters 24
            → then sweep.py → select rotation → refine.py

   If still failing after 2 retries → STOP. Report failure.

IMPORTANT: Never retry refine.py more than once with the same footprint. If refine fails, the problem is the footprint or rotation selection, not refine's optimizer. Go back to step 3 and try a different --candidate-rank.

---

Acceptance Thresholds (refine.py)

Auto-accept when ALL pass:

Metric	Pass	Borderline	Fail
fwd_chamfer_mean	< 2.5 um	2.5-4.0 um	> 4.0 um
IoU	> 0.70	0.50-0.70	< 0.50
top_containment	> 0.90	0.80-0.90	< 0.80
outside_fraction	< 0.10	0.10-0.20	> 0.20

Borderline: Accept but log a warning. Check diagnostic images. Fail on any metric: Do NOT accept. Adjust parameters and retry.

---

Adjusting Parameters: Feedback → Action

This is the core skill — reading diagnostic outputs and knowing which knob to turn.

After source_contour.py

Goal: The contour must capture the entire largest bright region — the full flake outline, including any very bright sub-regions (reflections, thin areas). A contour that misses the bright center but traces only the dim edges is wrong.

Common failure: Otsu auto-threshold can split the flake into "bright" and "very bright" regions, discarding the very bright part. In 01_source_contour.png, look for holes or missing chunks in the center of the flake — that means the threshold excluded the brightest pixels.

What you see in 01_source_contour.png	What's wrong	Action
Contour traces the full flake boundary	Nothing	Proceed
Contour has a hole or missing center (very bright area excluded)	Otsu split the flake — bright part was thresholded out	Re-run with --gray-only to skip saturation threshold
Contour is too small / misses edges	Threshold too aggressive	Check if the image is very dark or low-contrast
Contour includes substrate/debris	Threshold too loose	Usually means the flake isn't the largest bright region — check source image quality
No contour found (area=0)	Flake not detected	Image may need manual inspection; verify it's the right file

After footprint.py

What you see in diagnostics	What's wrong	Action
04_footprint_grabcut.png matches source flake shape	Nothing	Proceed
Footprint grabs entire flake assembly + debris/satellite flakes	Default candidate (#1) picked up too much	Re-run with --candidate-rank 2 (or 3). Always check 03_footprint_candidates.png first — a better candidate likely exists
Footprint too large (includes bottom hBN)	Wrong clusters selected	Re-run with --n-clusters 20, --n-clusters 24 for finer segmentation
Footprint too small (misses edges)	GrabCut too aggressive	Re-run with --candidate-rank 2 or --candidate-rank 3
Footprint is completely wrong shape	Shape matching failed	The source and target may look too different; check if --mirror is correct
shape_distance > 0.5 in stdout	Poor shape match	Continue anyway — GrabCut may still produce a usable footprint

After sweep.py — Choosing Rotation

What you see in candidates	Guidance
One candidate clearly matches	Use its rotation as --rot-hint
Two candidates look similar	Try the one where long edges align with visible flake edges
No candidate looks right	Footprint is likely wrong — go back to step 3
Contour is right shape but shifted	Rotation is correct but translation is off — refine.py will fix this

Key judgment: Look for edge alignment, not just overlap. The contour's straight edges should line up with the flake's crystallographic edges in the target image.

After refine.py — When Metrics Fail

Failed Metric	What it means	Adjustment
outside_fraction > 0.20	Warped flake extends beyond footprint	Wrong rotation. Try the next-best sweep candidate.
IoU < 0.4	Poor overlap between masks	Scale is wrong. Add --scale-hint with a value from the sweep candidate, ±0.1.
fwd_chamfer > 5 um	Contour edges don't align	Rotation off by a few degrees. Widen: re-run with --rot-hint ±5° from current.
top_containment < 0.80	Much of warped flake is outside footprint	Footprint too small or rotation wrong. Check 21_mask_overlap.png: blue regions = warped-only = problem areas.
All metrics fail badly	Fundamentally wrong alignment	Start over. Re-examine footprint, try different rotation candidate, or check if --mirror is correct.

Retry Strategy

Rule: NEVER retry refine.py with the same footprint. If refine fails, fix the footprint first. Time budget: Each refine.py attempt takes 10-15 min. Budget max 2 full attempts (footprint→sweep→refine). If 2 attempts fail and the best IoU is above 0.5, accept it and proceed — an imperfect alignment that lets you complete the pipeline is better than a perfect alignment that times out. Execution reminder: ALWAYS run refine.py as a foreground blocking Bash command with timeout=1200000. NEVER use run_in_background or sleep/poll loops.

Attempt 1: footprint (default) → sweep → select rotation → refine
  → If refine FAILS (IoU < 0.50):
Attempt 2: footprint --candidate-rank 2 → sweep → select rotation → refine
  → If still FAILS (IoU < 0.50):
Accept the best result from attempts 1-2 and proceed. Do NOT run a 3rd refine.
Max refine.py invocations: 2. Each takes 10-15 min — 3 would consume 45 min.

---

Scripts Reference

sift_align.py

${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/sift_align.py \
    --source <image> --target <image> --pixel-size <um/px> --output-dir <path> \
    [--min-inliers 20] [--scalebar-bottom 0.08] [--scalebar-right 0.20]

Optional:

• --min-inliers N — minimum RANSAC inliers for "sufficient" quality (default: 20). Thresholds: good ≥ max(50, 2N), warning ≥ N, insufficient < N. Lower to 10 for images with few substrate features.
• --scalebar-bottom F — fraction of image height to mask from bottom to exclude scalebar (default: 0.08). Set to 0 to disable.
• --scalebar-right F — fraction of image width to mask from right to exclude scalebar (default: 0.20). Set to 0 to disable.

Exit code	Meaning	Agent action
0, ≥50 inliers	Good alignment	Done. Use warp_sift_bottom.npy
0, ≥min-inliers	Marginal alignment	Accept with warning. Check 01_sift_matches.png
2	Too few matches (<min-inliers)	Try --min-inliers 10. If still fails, switch to Chamfer pipeline
1	Error	Check stderr

Outputs: warp_sift_bottom.npy, 01_sift_matches.png, 01_sift_overlay.png (magenta-tinted warped source on desaturated target), updates alignment_report.json

source_contour.py

${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/source_contour.py \
    --image <image> [--mirror] [--gray-only] --output-dir <path>

Optional: --gray-only — use grayscale Otsu only, skip saturation intersection. Use this when the flake has very bright/overexposed areas that appear white (low saturation). Without this flag, bright areas are excluded by the saturation threshold.

Outputs: source_contour.npy, source_mask.png, 01_source_contour.png, updates alignment_report.json

footprint.py

SIFT-aligns bottom_part to target, computes LAB diff image, K-means on diff intensity. Isolates the top-placed flake from substrate. Splits disconnected blobs within clusters into sub-clusters before enumeration, so spatially separate flakes sharing the same intensity are treated independently.

${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/footprint.py \
    --source <top_part> --target <full_stack_raw> \
    --bottom <bottom_part> [--mirror] \
    [--source-contour <out>/align/source_contour.npy] \
    [--source-mask <out>/align/source_mask.png] \
    --pixel-size <um/px> --output-dir <path>

Optional:

• --source-contour + --source-mask — use pre-computed contour/mask from source_contour.py instead of re-segmenting internally. Recommended: ensures footprint uses the same source shape as sweep/refine.
• --n-clusters N — number of K-means clusters (default: 12; increase for finer segmentation on retry)
• --candidate-rank N — use the Nth-ranked candidate instead of the default (#1). Always check 03_footprint_candidates.png — candidate #1 is often wrong. Try --candidate-rank 2 or --candidate-rank 3 on retry.
• --warp <path-to-warp_sift_bottom.npy> — reuse the SIFT warp produced by sift_align.py instead of re-running SIFT internally (issue #31). If omitted, auto-resolves <output-dir>/warp_sift_bottom.npy then <source-parent>/../align/warp_sift_bottom.npy. Internal SIFT only runs when neither path exists. Sibling scripts (sweep.py, refine.py, source_contour.py) do not run SIFT internally and need no equivalent flag.

Outputs: footprint_mask.png, footprint_contour.npy, 02_diff_image.png, 02_cluster_map.png, 03_footprint_candidates.png, 04_footprint_grabcut.png, updates alignment_report.json

sweep.py

${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/sweep.py \
    --source-contour <.npy> --source-mask <.png> \
    --footprint-contour <.npy> --footprint-mask <.png> \
    --target-image <image> --pixel-size <um/px> --output-dir <path>

Outputs: candidate_01.png ... candidate_NN.png, 05_sweep_grid.png, updates alignment_report.json with "status": "needs_rotation_selection"

Auto re-sweep: If all top-8 candidates have scale < 0.75 (degenerate small-scale minimum), sweep.py automatically re-runs with scale floor raised to 0.75. This adds ~50s but avoids passing degenerate scales to refine.

refine.py

${PYTHON_PATH:-conda run -n instrMCPdev python} skills/nanodevice_flakedetect_align/scripts/refine.py \
    --source-contour <.npy> --source-mask <.png> \
    --footprint-contour <.npy> --footprint-mask <.png> \
    --target-image <image> \
    --rot-hint <degrees> [--scale-hint <value>] \
    --pixel-size <um/px> --output-dir <path>

Auto scale hint: When --scale-hint is omitted, refine.py reads alignment_report.json and uses the scale from the sweep candidate closest to --rot-hint. This constrains the search to ±0.1 around the sweep's estimate, avoiding the degenerate small-scale minimum.

Outputs: warp_top.npy, 20_best_overlay_raw.png, 21_mask_overlap.png, 22_chamfer_heatmap.png, updates alignment_report.json with "status": "complete"

---

Warp Matrix Convention

• warp_sift_bottom.npy: full_stack → bottom_part direction. Use cv2.invertAffineTransform() to go bottom_part → full_stack.
• warp_top.npy: source (top_part, possibly mirrored) → full_stack direction. Apply directly.