| name | verification-validation |
| description | Verification and Validation (V&V) — ASME V&V 10/20, model validation metrics, uncertainty quantification, code verification (Richardson extrapolation, MMS), validation hierarchy, AIAA standards. |
| metadata | {"priority":7,"promptSignals":{"phrases":["verification","validation","V&V","model validation","ASME V&V","code verification","Richardson extrapolation","uncertainty quantification","UQ","manufactured solution"],"minScore":3}} |
Verification & Validation (V&V) — Complete Skill
V&V Definitions (ASME V&V 10)
Verification: "Are we solving the equations right?"
→ Confirms that the numerical model correctly solves the mathematical model
→ No comparison with physical reality
Validation: "Are we solving the right equations?"
→ Confirms that the mathematical model accurately represents physical reality
→ Requires experimental data
Qualification: Is the validated model suitable for the intended use/decision?
Model: Mathematical representation of a physical system (CFD, FEA, system model, etc.)
Simulation Credibility: degree of trust justified by V&V evidence
Code Verification
Richardson Extrapolation (Order of Convergence)
For a code with theoretical order p, solution errors should converge as:
φ_exact = φ_h + C × h^p (h = grid/time step size)
Three-grid study (h₁ < h₂ < h₃, refinement ratio r = h₂/h₁ = h₃/h₂):
Observed order: p = ln[(φ₃-φ₂)/(φ₂-φ₁)] / ln(r)
Extrapolated exact value:
φ_exact ≈ φ₁ + (φ₁-φ₂)/(r^p - 1)
Grid Convergence Index (GCI) — Roache:
GCI_fine = F_s × |ε₂₁| / (r^p - 1)
ε₂₁ = (φ₂-φ₁)/φ₁ (relative error between grids 1 and 2)
F_s = 1.25 (safety factor for 3+ grids); 3.0 (for 2 grids)
GCI_fine represents estimated discretization error (as % of fine grid solution)
Acceptable: GCI < 5% for most engineering simulations
Asymptotic range check:
GCI_medium / (r^p × GCI_fine) ≈ 1.0 (confirms asymptotic convergence)
Method of Manufactured Solutions (MMS)
Insert known analytical solution into governing equations → derive source term
Apply source term to code → verify code produces manufactured solution
Steps:
- Choose smooth manufactured solution φ_mfg (e.g., sin/cos combinations)
- Compute residual: R = L(φ_mfg) (apply differential operator L to φ_mfg)
- Add R as source term to PDE: L(φ) = R → exact solution = φ_mfg
- Run code with source term; compare to φ_mfg
- Refine grid; verify observed order p = theoretical order
MMS advantages: tests all code terms; no simplified exact solution needed; applicable to any PDE
Solution Verification (Numerical Errors)
Sources of Numerical Error
- Discretization error (dominant): O(h^p) for spatial, O(Δt^q) for temporal
- Iterative convergence error: residual not driven to zero
- Round-off error: machine precision (double: ~10⁻¹⁵)
Iterative convergence criteria:
Residuals: R_i < 10⁻⁴ for engineering; 10⁻⁶ for sensitive quantities
Monitor: solution quantities at representative points until stable
Coupled simulations: tighter convergence often needed
Model Validation
Validation Hierarchy (Oberkampf-Trucano Pyramid)
Level 0 (base): Unit problems — simple, isolated physical processes
Level 1: Benchmark cases — well-characterized experiments, known geometry, BC
Level 2: Subsystem validation — complete component with multiple coupled phenomena
Level 3 (top): System validation — full system, realistic conditions, prediction mode
Key principle: validate at lower levels first; identify dominant physics; then validate complex system
Validation Metrics
E_val = |S - D| (simple difference)
S = simulation prediction, D = experimental measurement
Area validation metric (ASME V&V 10.1):
d = area between simulation and experiment CDF (cumulative distribution functions)
d = 0: perfect agreement; d = area of mismatch
u-pooling (Kennedy-O'Hagan):
z_i = (S_i - D_i) / √(u_S² + u_D²) (normalized residual)
Should be N(0,1) if all uncertainties properly quantified
ANOVA validation: test if residuals are random (no systematic bias)
Uncertainty Budget for Validation
Total simulation uncertainty:
u_simulation = √(u_numerical² + u_input² + u_model²)
u_numerical = discretization + convergence errors (from GCI)
u_input = uncertainty in boundary conditions, material properties
u_model = structural model uncertainty (unknown physics)
Experimental uncertainty:
u_exp = √(u_meas² + u_rep²) (measurement + repeatability)
Validation comparison:
Agreement if: |S - D| ≤ u_val = √(u_sim² + u_exp²) (within combined uncertainty)
If |S - D| > u_val: model inadequacy exists → identify source
Standards and References
| Standard | Organization | Scope |
|---|
| ASME V&V 10 (2019) | ASME | V&V in computational solid mechanics |
| ASME V&V 20 (2009) | ASME | V&V in computational fluid dynamics & heat transfer |
| AIAA G-077-1998 | AIAA | CFD V&V (pioneering document) |
| NASA-STD-7009 | NASA | Standards for modeling and simulation |
| MIL-STD-3022 | DoD | M&S V&V for defense acquisition |
| NRC NUREG/CR-7049 | NRC | V&V for nuclear reactor safety |
| ISO/IEC 15288 | ISO | System V&V in systems engineering |
Uncertainty Quantification (UQ)
Forward UQ: Propagating Input Uncertainty to Output
Monte Carlo (MC): sample from input distributions → run N simulations → output distribution
Convergence: uncertainty in mean ~ σ/√N → need N > 1000 for 1% accuracy
Latin Hypercube Sampling (LHS): stratified sampling → fewer samples needed (~100-500)
Polynomial Chaos Expansion (PCE): approximate output = polynomial in input parameters; efficient for smooth problems
Surrogate-Based UQ
For expensive simulations: build surrogate (Gaussian Process, Kriging, Neural Network)
Sample: Design of Experiments (DoE) to build surrogate
Then: MC or analytical moments from surrogate
Sobol indices (variance-based sensitivity):
S_i = V_i / V_total (first-order = main effect)
S_T,i = (total variance due to X_i) / V_total (includes interactions)
S_T > 0.1: significant factor; S_T < 0.05: negligible
Epistemic vs. Aleatory Uncertainty
Aleatory (irreducible, random): natural variability — represented by probability distributions
Epistemic (reducible, lack of knowledge): model form uncertainty — can be reduced with more data
V&V targets: reduce epistemic uncertainties; characterize aleatory remaining uncertainties
Validation Evidence Assessment
AIAA V&V Terminology
Validation domain: range of conditions (Re, load, T) over which model is validated
Extrapolation: using model outside validation domain — requires additional uncertainty
Confirmation bias: tendency to interpret results favorably — requires pre-test predictions
Model validation checklist:
☐ Physics: all dominant phenomena included?
☐ Geometry: adequate fidelity?
☐ Boundary conditions: match experimental conditions?
☐ Material models: appropriate (linear/nonlinear, isotropic/anisotropic)?
☐ Solution verification: GCI < 5%? Iterative convergence?
☐ Experimental data: uncertainty documented? Independence of data?
☐ Comparison: within u_val?
☐ Sensitivity: dominant parameters identified?
Output
Provide: verification results (GCI [%], observed order p vs. theoretical), validation metric (|S-D| vs. u_val), uncertainty budget (u_numerical, u_input, u_model [%]), Sobol sensitivity ranking, confidence statement on model suitability for intended use.
