| name | validation-diff-gifs |
| description | Generate pixel-difference GIF animations comparing benchmark validation captures against golden images. Use when the user asks to visualize validation failures, compare captured vs golden images, generate diff GIFs, or debug image tolerance issues. |
Validation Difference GIFs
Generate per-camera GIF animations showing the pixel-wise difference between captured benchmark images and their golden references. Useful for debugging validation tolerance failures.
Prerequisites
- ImageMagick (
composite, convert) must be available on PATH.
- A completed validation capture run with matching directory structure to the golden data.
Paths
Captures and golden data live under the Isaac Sim build release directory:
standalone_examples/benchmarks/validation/captures/<run_name>/
standalone_examples/benchmarks/validation/golden_data/<benchmark_name>/
Both share a parallel directory tree (e.g. Robots/Robot_0/.../front_hawk/left/camera_left/rgb/).
Usage
Find the latest capture
ls -td standalone_examples/benchmarks/validation/captures/benchmark_robots_nova_carter_ros2_*/ | head -1
Run the script
The bundled script is resolved via ${CLAUDE_SKILL_DIR}; falls back to the canonical repo path skills/validation-diff-gifs/scripts/:
bash "${CLAUDE_SKILL_DIR:-skills/validation-diff-gifs}/scripts/generate_diff_gifs.sh" \
<captured_run_dir> \
<golden_benchmark_dir> \
[amplify] [fps]
| Argument | Default | Description |
|---|
captured_run_dir | (required) | Root of the capture run |
golden_benchmark_dir | (required) | Root of the golden data for the benchmark |
amplify | 10 | Multiply pixel differences by this factor for visibility |
fps | 5 | Frame rate for the output GIF |
Example
LATEST=$(ls -td standalone_examples/benchmarks/validation/captures/benchmark_robots_nova_carter_ros2_*/ | head -1)
GOLDEN="standalone_examples/benchmarks/validation/golden_data/benchmark_robots_nova_carter_ros2"
bash "${CLAUDE_SKILL_DIR:-skills/validation-diff-gifs}/scripts/generate_diff_gifs.sh" "$LATEST" "$GOLDEN"
Output
For each rgb/ directory under the capture, a diff_animation.gif is written alongside the captured PNGs:
<captured_run_dir>/Robots/.../front_hawk/left/camera_left/rgb/diff_animation.gif
- Black pixels = identical between captured and golden
- Brighter pixels = larger difference (amplified by the
amplify factor)
Interpreting results
- Uniform low-level noise across the frame → rendering non-determinism (likely acceptable)
- Bright regions concentrated on object edges → sub-pixel movement differences
- Entire frame bright → wrong timestamp match or completely different camera pose
- One camera consistently worse than others → possible per-camera issue (tick rate, initialization)
