simulation-dev

name	simulation-dev
description	General simulation development best practices for correctness, stability, and debuggability. Use when implementing or modifying simulation systems, solvers, constraints, or GPU kernels, especially for index safety, NaN/Inf issues, and diagnostics.

Simulation Development Best Practices

General rules for building simulation code that is correct first, debuggable second, and fast third.

When to Use

Use this skill when:

• adding or modifying simulation systems, solvers, integrators, constraints, or contact logic,
• touching CPU/GPU compute paths that can fail from invalid indices or unstable numerics,
• debugging crashes, divergence, NaN/Inf, or non-deterministic behavior,
• improving diagnostics, assertions, and reproducibility.

Core Principles

1. Correctness before performance.
2. Fail fast on invalid states.
3. Prefer deterministic behavior during debugging.
4. Prefer MUDA-native diagnostics for GPU paths whenever possible.
5. Validate data at subsystem boundaries.
6. Instrument first, optimize after evidence.

Safety Rules (Always Apply)

Input and shape validation

• Validate counts, offsets, dimensions, and index ranges before compute.
• Verify assumptions at boundaries (API input, scene load, assembly, solver entry).
• Use clear sentinel conventions and assert them (-1, empty, null, etc.).

Index safety for GPU/parallel code

• Guard every computed index before read/write.
• Return early on invalid thread work instead of continuing with partial writes.
• Prefer host-side prevalidation to avoid propagating bad mappings into kernels.

Numerical robustness

• Check for finite values (isfinite) at key checkpoints.
• Assert on first invalid scalar near source (residuals, energies, step sizes, norms).
• Add conservative guards for divide-by-near-zero and invalid square roots/logs.

Assertion Best Practices

Every assertion should answer:

1. What failed?
2. Where in the pipeline?
3. With what values?
4. What is the likely cause?

Checklist for high-quality assert messages:

• include stage/iteration/frame context,
• include offending value and expected range/invariant,
• include at least one hint for root-cause direction,
• keep wording short and actionable.

Diagnostics and Observability

• Use MUDA diagnostics as the default for GPU debugging:
  • attach file/line metadata on launches,
  • assign explicit names to views/buffers,
  • prefer MUDA launch wrappers over raw CUDA launch sites when practical.
• Attach source location metadata to kernel/parallel launches when available.
• Name critical buffers/views to make reports readable.
• Keep optional debug dumps for important intermediates (vectors, matrices, topology maps).
• Use scoped timers around major phases and hot sub-phases.
• Make debug instrumentation easy to toggle with config flags.

MUDA Debug Checklist

For each touched GPU code path:

• Do: add launch source mapping (for example, file_line(__FILE__, __LINE__)).
• Do: name captured views/buffers (for example, viewer().name("x")) for readable diagnostics.
• Do: use MUDA launch/parallel wrappers as first choice for new or modified kernels.
• Do: use muda::debug_sync_all() as a fast-fail barrier while debugging broken kernels.
• Do: keep index guards at the top of kernel bodies before any memory access.
• Do: preserve a low-overhead path when debug flags are off.
• Don't: add raw CUDA launch sites unless there is a clear, documented need.
• Don't: leave anonymous captures for critical buffers in hot or unstable paths.
• Don't: rely on post-mortem logs only; add immediate context at the failure point.

Deterministic Debug Workflow

1. Reproduce on the smallest case that still fails.
2. Freeze nondeterminism where possible:
   • fixed seeds,
   • stable ordering,
   • reduced parallel variability.
3. Enable strict checks and diagnostic output.
4. Bisect by pipeline stage (assemble -> solve -> update).
5. Narrow to first bad value and assert there.

Simulation Debug Loop (Iterative)

Use this loop as the default process for debugging simulation issues:

1. Build a minimal reproducible scene/case that is easy to debug.
2. Use diagnostics to understand behavior, form a concrete assumption, then add debug code to validate that assumption.
3. After validating the assumption, implement the fix in the minimal case first.
4. Re-run the original scene/case to verify the real issue is resolved.
5. Add a regression test for the confirmed fix.

If any step fails or new evidence contradicts the assumption, return to step 1 or 2 and iterate until root cause and fix are both confirmed.

During active GPU debugging, insert muda::debug_sync_all() at key boundaries to stop immediately when a kernel fails, then remove or gate it once diagnosis is complete.

Convergence and Stability Guidelines

• Separate stopping criteria from failure criteria.
• Distinguish "did not converge yet" from "invalid state".
• Track monotonic indicators where expected (energy, residual envelopes).
• For iterative methods, sample/check periodically to limit synchronization overhead.
• Provide explicit fallback or abort policy for non-convergence in strict mode.

Change Workflow

1. Write/modify logic with invariants first.
2. Add assertions at boundaries and first-use sites.
3. Add diagnostics (timers, buffer names, optional dumps).
4. Verify on:
   • minimal case,
   • representative case,
   • stress case.
5. Only then profile and optimize.