physics-informed-ml

name	physics-informed-ml
description	Physics-informed machine learning (PIML) — physics-informed neural networks (PINNs), loss function with PDE residuals, neural operator learning (FNO, DeepONet), Gaussian process regression with physics constraints, data-efficient training, hybrid physics/ML surrogate models, applications in CFD/FEA surrogate, turbulence modeling, material discovery, and uncertainty quantification.
metadata	{"priority":7,"promptSignals":{"phrases":["physics-informed machine learning","PINN","physics-informed neural network","neural operator","FNO Fourier neural operator","surrogate model physics"],"minScore":3}}

Physics-Informed Machine Learning — Complete Skill

Motivation and Overview

Why Physics-Informed ML?

Pure data-driven ML limitations: Requires large datasets (10³–10⁶ samples) — expensive for simulations No physical consistency: violates conservation laws, thermodynamics Poor extrapolation outside training distribution Cannot handle out-of-distribution scenarios (novel geometry, new material)

Pure physics simulation limitations: Expensive: CFD/FEA may take hours per case Cannot handle unknown physics (turbulence closures, material microstructure) Cannot efficiently assimilate experimental data

PIML hybrid approach: Embed physics (PDEs, conservation laws, symmetries) into ML architecture or loss function Reduce data requirements by orders of magnitude Guarantee physical consistency (at least approximately)

Physics-Informed Neural Networks (PINNs)

Problem Formulation

Governing PDE: F(u, ∂u/∂t, ∂u/∂x, ∂²u/∂x², ...; x, t, λ) = 0 in Ω × [0,T] BC: B(u; x_boundary) = 0 on ∂Ω IC: u(x, 0) = u₀(x)

PINN approach: Approximate u(x,t) by neural network: û(x,t; θ) where θ = network parameters Minimize composite loss that enforces PDE + BC + IC + data

Loss function: L(θ) = λ_PDE × L_PDE + λ_BC × L_BC + λ_IC × L_IC + λ_data × L_data

PDE residual loss: L_PDE = (1/N_pde) × Σᵢ |F(û, ∂û/∂x, ...; xᵢ, tᵢ)|² [collocation points in domain] ∂û/∂x = automatic differentiation through network (autograd)

Boundary/initial condition loss: L_BC = (1/N_bc) × Σⱼ |B(û; x_j)|² [boundary collocation points] L_IC = (1/N_ic) × Σₖ |û(xₖ, 0) - u₀(xₖ)|²

Data loss (if observations available): L_data = (1/N_data) × Σₗ |û(xₗ, tₗ) - u_meas(xₗ, tₗ)|² [measured/simulated data points]

Network Architecture

Standard architecture: Fully connected (FC) MLP: 4–10 hidden layers × 50–200 neurons each Activation: tanh (smooth; required for automatic differentiation of high-order derivatives) Avoid ReLU (non-smooth; second derivative undefined at zero)

Input: [x, t] or [x, y, z, t]; Output: [u, p, T, ...] For Navier-Stokes: output = [u_x, u_y, u_z, p] For heat equation: output = [T]

Modified MLP (enhanced PINNs): Fourier feature input: embed x as [sin(2πk·x), cos(2πk·x)] → better spectral representation Adaptive activation (LAAF): a_i × tanh(n × z) where a_i trainable → faster convergence Residual connections: skip connections across layers for deep networks

Collocation and Training

Collocation point sampling: Uniform grid: simple; may miss regions with high PDE residual Latin Hypercube Sampling (LHS): better coverage; standard Adaptive sampling (RAR — Residual-Adaptive Refinement): add points where |F|² is large

Number of collocation points: N_PDE = 1,000–100,000 (depends on problem complexity; 3D problems need more) N_BC = 100–10,000 per boundary; N_IC = 1,000–10,000 Training time: 10,000–100,000 iterations; L-BFGS or Adam optimizer

Weighting λ_PDE, λ_BC, λ_IC, λ_data: Imbalanced losses → training instability; set λ to normalize each loss contribution Adaptive: λ_i = max_k(|∇L_k|) / mean_k(|∇L_k|) [GradNorm; Kuang et al. 2022] Or: λ_PDE >> λ_data initially; anneal to λ_data >> λ_PDE as data loss decreases

Inverse Problems with PINNs

Identify unknown parameters λ from measurements: Add λ as additional trainable parameters alongside θ L(θ, λ) = L_PDE(θ, λ) + L_data(θ) [minimize jointly] Examples: identify fluid viscosity, reaction rate, elastic modulus from displacement data

PINN inverse for turbulence: Given velocity measurements u_meas at sparse points; identify turbulent viscosity ν_t(x) Represent ν_t as separate NN; train jointly with Navier-Stokes PINN Output: full ν_t field at all locations (not just measurement locations)

Neural Operator Learning

Operator Learning Concept

Motivation: Traditional PINN: solves ONE specific PDE instance (one set of IC/BC) Neural operator: learns the mapping from PDE parameters/IC/BC → solution (operator G: IC → u) One trained operator → instantly evaluate for ANY new IC/BC (without retraining)

Mathematical formulation: G: A → U [G maps input function a(x) (IC, BC, PDE parameters) to solution u(x,t)] Discretization-invariant: same operator for different grid resolutions

Fourier Neural Operator (FNO)

Key idea (Li et al. 2021): Apply integral operator in Fourier space → efficient; captures global dependencies

FNO layer: (K(a)(v))(x) = ∫ κ(x, y; a(y)) v(y) dy [integral operator; kernel κ parameterized as NN] In Fourier space: K̂(v)_k = R_k × v̂_k [R_k = learnable weight matrix in frequency domain] Truncate at k_max modes: only keep low k modes (high-frequency physics regularized out)

FNO architecture: Lift input a → higher-dimensional space v₀ via pointwise linear layer Apply L Fourier layers: v_{l+1} = σ(W × v_l + K × v_l) [K = Fourier integral; W = local linear layer] Project to output u via final linear layer

Performance (FNO): Navier-Stokes 2D (turbulent): 1,000× faster than standard solver; L2 error 1–5% Darcy flow (porous media): 1,000× speedup; < 1% L2 error 3D turbulence: challenging; FNO-3D exists but slower

Implementation: github.com/neural-operator/neuraloperator (PyTorch)

DeepONet

Deep Operator Network (Lu et al. 2021): G(u)(y) = Σₖ bₖ(u) × tₖ(y) [branch network bₖ encodes input function; trunk network tₖ encodes output location] Branch: takes input function u sampled at m points → [b₁, ..., b_p] Trunk: takes output location y → [t₁, ..., t_p] Output: G(u)(y) = Σₖ bₖ × tₖ [dot product]

Advantage over FNO: works for any output location (mesh-free); handles multi-physics Disadvantage: less efficient for regular grid data than FNO

Graph Neural Operators (GNO)

For irregular meshes (finite element meshes): Graph structure: nodes = mesh points; edges = connectivity Message passing GNN: aggregate neighbor information → learn spatial operators on arbitrary geometry Applications: mesh-based CFD/FEA surrogate; topology-varying problems

Gaussian Process (GP) with Physics Constraints

Physics-Constrained GP

Standard GP: u(x) ~ GP(μ(x), k(x, x')) [prior; μ = mean function; k = covariance/kernel]

Physics-constrained GP: Encode PDE as linear operator on GP: Lu = f(x) Since L is linear and GP is Gaussian: Lu ~ GP(L[μ], L[k]) Posterior given PDE-constrained observations → physics-informed GP

Advantage: Automatic uncertainty quantification (posterior variance = confidence) Works with very few data points (O(10–100)); ideal for expensive simulations Exact: no training, closed-form posterior

Disadvantage: O(N³) cost (GP inversion); N limited to O(10³) without approximation Doesn't scale to high-dimensional problems (> 10 inputs)

Application: infer full stress field from sparse strain gauge measurements; assimilate sensor data + FEM

Hybrid Physics/ML Models

Turbulence Closure

RANS turbulence modeling: Reynolds stress: -ρ u'_i u'_j = μ_t × (∂U_i/∂x_j + ∂U_j/∂x_i) - 2/3 ρ k δ_ij [Boussinesq] Traditional: k-ε or k-ω closure for μ_t (inaccurate for separated flows, strong curvature)

ML-augmented turbulence: Data-driven correction: μ_t = μ_t_kω × (1 + Δμ_ML) [NN corrects baseline model] Trained on high-fidelity DNS/LES data; significantly improves RANS accuracy for complex flows Ling et al. (2016): tensor-basis NN with Galilean invariance; MLP correction to k-ε

Neural network Reynolds stress: b_ij = NN(invariants of S_ij, Ω_ij, k/ε) [normalized anisotropy tensor from 5 invariants] Ensures frame invariance; feed into RANS solver (weakly-coupled or embedded)

Material Response Surrogate

FEA surrogate: Train NN on FEA results: input (geometry parameters, loads) → output (stress field, displacement) Autoencoders: compress field output to latent representation → predict latent from inputs → decode POD-NN: Proper Orthogonal Decomposition modes as basis; predict mode coefficients from NN

Crystal plasticity surrogate: Input: texture (Euler angles), loading path → output: stress-strain response Physics constraint: enforce volume consistency (det(F) = 1) via softmax output layer Reduces CP simulation time from hours to milliseconds

Active Learning

Select next simulation point to maximize information: Uncertainty-based: run simulation where GP posterior variance is highest Expected improvement: maximize E[improvement over current best surrogate] Reduces required simulations by 5–10×; useful for design optimization with expensive FEA

Implementation

PINNs Python Implementation (DeepXDE)

import deepxde as dde
import numpy as np

# Burgers' equation: du/dt + u*du/dx - ν*d²u/dx² = 0
def pde(x, u):
    u_x = dde.grad.jacobian(u, x, i=0, j=0)  # du/dx
    u_t = dde.grad.jacobian(u, x, i=0, j=1)  # du/dt
    u_xx = dde.grad.hessian(u, x, i=0, j=0)  # d²u/dx²
    return u_t + u * u_x - 0.01/np.pi * u_xx

geomtime = dde.geometry.GeometryXTime(
    dde.geometry.Interval(-1, 1), dde.geometry.TimeDomain(0, 1)
)
bc = dde.icbc.DirichletBC(geomtime, lambda x: 0, lambda _, on_boundary: on_boundary)
ic = dde.icbc.IC(geomtime, lambda x: -np.sin(np.pi * x[:, 0:1]),
                  lambda _, on_initial: on_initial)

data = dde.data.TimePDE(geomtime, pde, [bc, ic], num_domain=2540, num_boundary=80, num_initial=160)
net = dde.nn.FNN([2] + [20]*3 + [1], "tanh", "Glorot normal")
model = dde.Model(data, net)
model.compile("adam", lr=1e-3)
model.train(iterations=10000)

Libraries: DeepXDE: PINNs framework (PyTorch/TensorFlow/JAX); github.com/lululab/deepxde neuraloperator: FNO, GNO; github.com/neural-operator sciml (Julia ecosystem): DifferentialEquations.jl + Flux.jl for PINNs in Julia

Applications Summary

Application	Method	Speedup vs. FEA/CFD	Accuracy
Steady Navier-Stokes	FNO	1,000×	L2 error 1–3%
Turbulence RANS closure	GP + ML	2–5× (accuracy gain)	20–40% error reduction
Stress field from BC	PINN (inverse)	No simulation needed	1–5% vs. FEA
Material constitutive law	DeepONet	10,000× vs. crystal plasticity	L2 < 2%
Fatigue life prediction	Hybrid (Paris + NN)	100×	< 15% error

Standards and References

Source	Scope
Raissi, Perdikaris, Karniadakis (2019) JCP	Original PINN paper
Li et al. (2021) ICLR	FNO original paper
Lu et al. (2021) Nature Machine Intelligence	DeepONet paper
Ling et al. (2016) J. Fluid Mech.	ML turbulence closure
Brunton, Noack, Koumoutsakos (2020) Ann. Rev. Fluid Mech.	ML for fluid mechanics review

Output

Provide: problem type (forward/inverse/surrogate), governing PDE (write equation explicitly; linear/nonlinear), domain geometry and BCs, method selected (PINN/FNO/DeepONet/GP-physics/hybrid) with justification (data availability, complexity, speedup need), PINN architecture (layers, neurons, activation, collocation points N_PDE/N_BC/N_IC), loss function components (L_PDE, L_BC, L_IC, L_data; λ weights), training setup (optimizer; iterations; convergence metric), operator learning (if FNO: k_max modes; input/output functions), physics constraints embedded (conservation law; symmetry; thermodynamic constraint), accuracy metrics (L2 relative error; physics residual ||F||₂ after training), comparison baseline (FEA/CFD time vs. PIML inference time), uncertainty quantification (if GP: posterior variance bounds), and applicable reference (Raissi 2019, Li 2021, Lu 2021).