neqsim-model-calibration-and-data-reconciliation

name	neqsim-model-calibration-and-data-reconciliation
description	Model calibration and data reconciliation workflow for NeqSim digital twins. USE WHEN: reducing model-vs-plant mismatch, tuning process model parameters from historian data, reconciling noisy measurements, or building validation reports for calibrated simulations. Covers tag mapping, data quality gates, steady-state window selection, bounded optimization, residual diagnostics, and results.json reporting.
last_verified	2026-04-19

Model Calibration and Data Reconciliation

A practical workflow for calibrating NeqSim models against plant historian data and producing auditable validation output.

When to Use This Skill

• When simulated values do not match measured plant data.
• When building digital twin loops for separators, compressors, or heat exchangers.
• When you need reproducible parameter tuning with physical bounds.
• When a study requires train/validation split and objective fit metrics.

Scope and Non-Goals

This skill focuses on steady-state calibration and data reconciliation.

This skill does not replace dynamic controller tuning. For transient tuning, load neqsim-dynamic-simulation in addition.

Required Companion Skills

• neqsim-plant-data: historian read patterns (PI/IP.21), tag maps, quality flags.
• neqsim-input-validation: parameter bounds and physically realistic inputs.
• neqsim-troubleshooting: convergence recovery when each case is re-simulated.

Calibration Workflow

1. Define objective and acceptance criteria.
2. Build a trusted tag map from plant tags to model variables.
3. Read, clean, and quality-filter plant data.
4. Detect steady-state windows for fitting.
5. Choose tunable model parameters with hard physical bounds.
6. Run bounded optimization and re-simulate the full NeqSim process each iteration.
7. Validate on holdout windows and publish fit diagnostics.

Typical Tunable Parameters

Equipment	Parameters to Tune	Keep Fixed Unless Needed
Compressor	Polytropic efficiency, pressure ratio correction, anti-surge margin	Driver type, map family
Heat exchanger	UA correction factor, fouling factor	Duty sign convention, stream topology
Separator	Entrainment assumptions, liquid carry-over tuning factors	Phase model and EOS
Valve/choke	Cv multiplier, discharge coefficient	Control architecture
Pipeline	Roughness, overall heat transfer coefficient	Length and diameter

Data Reconciliation Quality Gates

Apply these gates before calibration:

• Unit harmonization complete (temperature, pressure, flow basis).
• At least 3 independent steady-state windows.
• Bad-quality tags removed or masked.
• Outlier handling documented.
• Sensor drift flags noted in notes and results.

Python Pattern: Bounded Calibration Loop

import numpy as np
from scipy.optimize import least_squares

# Example: tune compressor efficiency and HX UA multiplier
# x[0] -> compressor efficiency, x[1] -> UA multiplier
LOWER = np.array([0.60, 0.60])
UPPER = np.array([0.90, 1.40])
X0 = np.array([0.75, 1.00])

# Weights for [discharge pressure, discharge temperature, outlet temperature]
W = np.array([1.0, 1.0, 0.8])

def run_model_and_extract_outputs(process, x):
    # Example parameter writes using automation addresses
    auto = process.getAutomation()
    auto.setVariableValue("Main Compressor.polytropicEfficiency", float(x[0]), "")
    auto.setVariableValue("Main Cooler.UA_multiplier", float(x[1]), "")

    process.run()

    p_out = auto.getVariableValue("Main Compressor.outStream.pressure", "bara")
    t_out = auto.getVariableValue("Main Compressor.outStream.temperature", "C")
    t_cool = auto.getVariableValue("Main Cooler.outStream.temperature", "C")
    return np.array([p_out, t_out, t_cool])


def residuals(x, process, y_target):
    y_model = run_model_and_extract_outputs(process, x)
    return W * (y_model - y_target)


# y_target should come from a selected steady-state time window average
res = least_squares(
    fun=residuals,
    x0=X0,
    bounds=(LOWER, UPPER),
    args=(process, y_target),
    method=,
    max_nfev=,
)

x_opt = res.x
(, x_opt)
(, res.cost)

Objective Design Guidance

Use a weighted residual vector with consistent scaling.

A common objective is:

$$ J(\theta) = \sum_{i=1}^{N} w_i\left(\frac{y_i^{model}(\theta) - y_i^{meas}}{s_i}\right)^2 $$

Where:

• $\theta$ is the vector of tunable parameters.
• $w_i$ is the business/engineering importance weight.
• $s_i$ is a scaling term (for example nominal value or sensor sigma).

Train/Validation Split Pattern

1. Train on at least two steady windows from different operating regions.
2. Validate on one holdout window not used in fitting.
3. Report both train and validation metrics.

Recommended minimum metrics:

• RMSE for each calibrated output tag.
• MAPE where denominator is safe and meaningful.
• Mean signed error (bias).
• Max absolute error.

Common Failure Modes and Fixes

Failure Mode	Likely Cause	Corrective Action
Excellent train fit, poor validation	Overfit on narrow operating range	Add windows across wider throughput and pressure range
Non-physical fitted parameters	Missing bounds	Enforce hard bounds and pre-check with input validation
Optimization stalls	Flat objective or too many coupled variables	Reduce parameter count, stage the tuning in phases
Noisy residuals	Sensor spikes and transient periods	Increase filtering and steady-state screening
Unstable run during optimization	Recycle sensitivity	Start from converged base case and use troubleshooting recovery patterns

Reporting Pattern for results.json

Add a calibration block to the task results.json:

{
  "calibration": {
    "calibrated_parameters": [
      {
        "name": "Main Compressor.polytropicEfficiency",
        "unit": "",
        "value": 0.781,
        "lower_bound": 0.60,
        "upper_bound": 0.90
      },
      {
        "name": "Main Cooler.UA_multiplier",
        "unit": "",
        "value": 1.12,
        "lower_bound": 0.60,
        "upper_bound": 1.40
      }
    ],
    "fit_metrics": {
      "train_rmse": 
         
         
         
      
       
       
         
         
         
      
    
     
       
       
         
         
         
      
    
     
      
      
    
     
      
      
    
  

Validation Checklist

• Parameter bounds are physically meaningful and documented.
• Calibration windows and validation windows are explicitly listed.
• Fit metrics are reported for both train and validation sets.
• Any excluded tags/windows have a documented reason.
• Final recommendations include applicability range and limitations.

References

• ISO 5725 (accuracy and trueness concepts for measurement quality).
• IEC 61511 / ISA-95 practices for validated operational data usage.
• Internal NeqSim patterns in neqsim-plant-data and neqsim-troubleshooting.