neqsim-task-workflow

# NeqSim Task Workflow This is the detailed procedure behind `@solve-task`. The agent file holds the mandatory first action, the purpose, and the operating principles (section 0); sections 1-10 below are loaded on demand for Standard and Comprehensive work. Section numbers are kept so cross-references from the agent and from `docs/development/TASK_SOLVING_GUIDE.md` stay valid. --- ## 0.5 ── ALWAYS-ON OBLIGATIONS (moved from the agent) ### Keep `user_input.md` up to date through the whole conversation `user_input.md` is the **reproducibility log** — anyone re-running this task should be able to recover the exact starting input from it. **Whenever the user provides new information during the task, append it to `user_input.md` immediately and verbatim:** - Answers to your clarifying questions → append to **section 2 (Q&A)** - Mid-task corrections, scope changes, or new constraints → append to **section 3 (Follow-up)** - Any value you assume because the user did not specify it → record it in **section 4 (Inferred Assumptions)** with justification Do NOT paraphrase, summarise, or "clean up" the user's wording. Paste it as given. Never delete previous entries — only append. ### All downloaded documents go INSIDE the task folder **All documents retrieved during a task — STID drawings, PI historian exports, vendor datasheets, P&IDs, literature PDFs, downloaded standards — MUST be saved to `step1_scope_and_research/references/` within the task folder.** NEVER download or save task-related files to workspace-level directories like `output/`, `figures/`, or any path outside `task_solve/YYYY-MM-DD_slug/`. **For STID/document retrieval scripts:** Always pass the task's `references/` path as the output directory: ```python # CORRECT — saves inside the task folder: out_dir = os.path.join(TASK_DIR, "step1_scope_and_research", "references") # WRONG — saves outside the task folder: out_dir = os.path.join(os.path.dirname(__file__), "..", "figures", "stid_docs") # NEVER DO THIS ``` **For PDF-to-PNG conversion:** Output converted images to the task's `figures/`: ```bash <python-executable> devtools/pdf_to_figures.py task_solve/YYYY-MM-DD_slug/step1_scope_and_research/references/ \ --outdir task_solve/YYYY-MM-DD_slug/figures/ ``` This ensures every task is self-contained and portable — zip the task folder and everything needed is inside it. ### Extract every downloaded or supplied document before solving The `references/` folder is not passive storage. Before any document-derived value, topology, limit, requirement, or recommendation enters Step 2, route **every file** through `technical-document-intelligence-agent` using `neqsim-document-intelligence-extraction`, then through the applicable document-type/domain reader. For each file: 1. Preserve the original and record its hash, actual format, revision/date, and extraction-tool versions. 2. Run native text and structured-table extraction first. For spreadsheets, retain sheets, formulas, merged ranges, hidden rows/columns, headers, and units. 3. Render and assess every page that contains visual or spatial information. Use OCR for scans and multimodal vision for drawings, P&IDs, charts, maps, photographs, annotations, symbols, topology, and table layout. OCR alone is not sufficient for visual engineering content. 4. Extract comprehensively: body text, tables, title blocks, notes, legends, appendices, footnotes, figures, plots, revision clouds, and unexpected engineering constraints. Do not stop after finding the fields anticipated by the initial task description. 5. Package facts with original text/value/unit, page/cell/bounding-box locator, method, confidence, normalized value/unit, and review status. Reconcile native/OCR/vision outputs without silently resolving conflicts. 6. Record unsupported content, missing expected fields, conflicts, ambiguous facts, and safety-critical or confidence-below-0.85 facts as explicit gaps or `needs_review` items. Write `step1_scope_and_research/document_evidence_manifest.json` with one entry for every file under `references/`, its extraction operations and status, links to its evidence package(s), and counts of facts/conflicts/gaps/review items. Summarize the evidence in `notes.md` and preserve fact identifiers in downstream inputs/results. The number of manifest source entries MUST equal the recursive file inventory count, excluding generated evidence packages and the manifest itself. Do not begin Step 2 while any source is `not_started` or silently unprocessed; record a blocked/manual-review status when extraction cannot run. --- ## 0.6 ── OPERATING ENVIRONMENTS (what each step needs) The workflow below was written in the NeqSim source checkout. It also runs from the pip-installed toolkit (`neqsim-dev-setup`, installed automatically by the core agent plugin's SessionStart hook) and, degraded, from chat with only the MCP server. Detect the environment first (agent Step 0) and apply this table. | Step / tool | Workspace (checkout) | Toolkit (pip, no checkout) | Chat-only (MCP) | |---|---|---|---| | `neqsim new-task`, `user_input.md`, `study_config.yaml` | yes | yes | create folder by hand from the template layout | | `neqsim_dev_setup.neqsim_init()` | `target/classes` (latest Java) | packaged JAR from `pip install neqsim` (`ns.JAR_MODE`, `ns.MISSING_CLASSES`) | not available - use MCP `runFlash`/`runProcess`/`runPVT` | | `recompile=True`, `mvnw`, Java classes + JUnit (NIP implementation) | yes | **no** - write the NIP, do not attempt | no | | NeqSim Runner (`AgentBridge`, `neqsim_runner`) | yes | yes | no - run scripts directly or use MCP | | `validate_task_results`, `consistency_checker`, `TaskResultValidator` | yes | yes (Python validators; the Java `TaskResultValidator` needs the JAR, available) | Python validators not installed - state results.json was not gate-checked | | `neqsim report` / `generate_report.py`, `WORK_RECORD.md` | yes | yes (Word template optional) | no - deliver the summary in chat | | `skill_search` / `agent_search`, `capability_assessment.md` | yes | yes | by reasoning over the skill catalogue only | | Enterprise data readers (STID, SAP, PDM, OTS, Seeq, PEPR) | with credentials | with credentials (enterprise plugin installs the packages) | no | | Task log / PR back to NeqSim | yes | NIP + issue only | NIP in the answer | When a row says **no**, the deliverable matrix in the agent's section 0 still applies: do the step by the strongest available means and record what was skipped in `results.json` `data_gaps` (or in the chat answer) so the reader knows the evidence level. --- ## 1 ── OVERVIEW You follow the **3-step AI-Supported Task Solving While Developing** workflow. "While Developing" is key — solving the task and improving NeqSim happen in the same session. The workflow is aligned with the IPA Front-End Loading (FEL) framework and Stage-Gate methodology: ``` STEP 1 → Scope & Research (FEL-1: Identify) Define standards, methods, deliverables + gather background STEP 2 → Analysis & Eval (FEL-2: Evaluate) Build simulation, run, validate, iterate until accepted STEP 3 → Report (FEL-3: Define) Generate Word + HTML deliverables ``` Each step transition is a **gate** (per Stage-Gate, Cooper 1990) — the quality gates in Section 2 define the go/no-go criteria before advancing. The framework also draws on: - **AACE 18R-97** — Cost estimate classification (Class 5→1 maturity) - **ISO 15288** — Systems engineering life cycle (needs → requirements → architecture → verification) - **VDI 2221** — Systematic engineering design (clarify → concept → embody → detail) - **NORSOK Z-013** — Risk and emergency preparedness (risk matrix, ALARP) - **DNV-RP-A203** — Technology qualification (TRL assessment for novel solutions) Your deliverable is a populated task folder under `task_solve/`. ### Post-Folder Preflight (mandatory before Step 1 work) After the task folder exists and `user_input.md` contains the prompt, do these checks before writing Step 1 content: 1. **Route check.** If the task spans multiple disciplines (e.g. process + economics + safety), invoke `@router` first to confirm which specialist agents to compose. For single-discipline tasks, skip. 2. **Agent & skill review (ALWAYS — thorough).** Before writing any analysis, do a deliberate review of the available **agents and skills** to find what can help solve this task instead of solving from scratch: - Run `<python-executable> devtools/skill_search.py "<task title>" --top 5` and load the top-3 SKILL.md files with `read_file`. - Scan the agent list (community + enterprise, e.g. `.github/agents/`, `neqsim-community-agents/`, `neqsim-enterprise-agents/`) for discipline specialists whose description matches the engineering question. - Record the shortlist in `step1_scope_and_research/notes.md`: which agents/skills you will use and why, and note explicitly where **no** suitable agent/skill exists (that gap is a candidate improvement in the Continuous Improvement Rule). 3. **NeqSim codebase review (ALWAYS).** Before writing custom code, review the NeqSim codebase to find existing functionality that solves the task: - Consult `neqsim-capability-map` and `CHANGELOG_AGENT_NOTES.md`, then use `semantic_search` / `grep_search` / `file_search` over `src/main/java/neqsim/` for the relevant classes (equipment, EOS, standards, mechanical design). - Prefer an existing NeqSim class over a simplified Python correlation. - Record in `notes.md` which NeqSim classes/methods will be used, and note any missing capability as a gap to be closed (NIP + Java implementation per Section 0 principle 2 and the Continuous Improvement Rule). 3b. **Data-acquisition screening (ALWAYS — before assuming stand-in data).** List the input data the task needs (equipment curves, geometry, fluid composition, operating point, etc.) and, for **each** input, screen for a retrieval tool/agent that can obtain the *real* value before falling back to a representative assumption. Do NOT proceed with stand-in data for an input that an available data-source agent could supply. Map each need to its source using this table (community names shown; use the `enterprise-` twin when on an enterprise task): | Data need | Retrieve with | Notes | |-----------|---------------|-------| | Compressor/pump curves, vendor performance maps, datasheets, P&IDs, mechanical drawings, line lists, material certs | `@stid.reader` / `stid-reader-agent` + `neqsim-stid-retriever` (STID doc download → PDF/OCR → `neqsim-technical-document-reading`) | Vendor "map"/curve docs live in STID; download then extract the map digitally | | Piping class, pipe internal diameter, wall thickness, valve (VDS/MDS) data, flange rating | `@tr2000` / `tr2000-agent` + `enterprise-tr2000-api` | Resolve line tag → PCS → internal Ø | | Live instrument data / operating point (P, T, flow, speed, valve %, current recycle) | `@plant-data` / `plant-data-agent` + `neqsim-plant-data` (tagreader PI/IP.21) or `@seeq.connect` / `enterprise-seeq-connect` | Resolve historian tag from STID tag; prefer `/PRIM` channel; check units | | Measured production volumes (well/field) | `rigga-production-agent` + `enterprise-rigga-production` | Measured production as feed / validation basis | | Equipment / maintenance / failure history | `@maintenance` / `maintenance-agent` + `enterprise-maintenance-api` | Read-only work orders, notifications | | Fluid composition / gas-sample molweight | STID/lab report via `@stid.reader`; else record as an explicit gap | A wrong MW mis-places compressor maps — always try to obtain it | | Public literature / standards | `@literature-scout` | See step 6 | Record in `notes.md`, for every required input: the source agent/skill you will use, or an explicit statement that the value is a documented **stand-in assumption** because no retrieval path is available (and why). "The data is a field unknown" is only acceptable **after** the relevant retrieval agent has been screened (and, where feasible, attempted). For governed enterprise sources (STID, TR2000, tagreader, Seeq, Rigga, Maintenance, PEPR), prefer the corresponding coordinator agent so authentication and read-only governance are handled correctly. 4. **Repo memory scan.** List `/memories/repo/*.md` via the `memory` tool. Read any whose filename contains a keyword from the task title — this surfaces prior solved tasks and known gotchas before you reinvent them. 5. **Capability assessment + agent/workflow and code discovery.** For Standard/Comprehensive tasks, invoke `@capability-scout` and write the result to `step1_scope_and_research/capability_assessment.md`. As part of this step you MUST run BOTH discovery tools and inspect the actual NeqSim implementation before selecting a method. Never rely only on the static routing table, capability-map prose, or mirrored task sources: ```bash <python-executable> devtools/skill_search.py "<task title>" --top 5 <python-executable> devtools/agent_search.py "<task title>" --top 8 \ --json --out step1_scope_and_research/agent_plan.json ``` Then search `CHANGELOG_AGENT_NOTES.md`, `src/main/java/neqsim/`, neighboring JUnit tests, and the locally resolved NeqSim dependency/JAR for the required equipment, thermodynamics, standards, safety, mechanical-design, and utility functionality. Verify constructors, method signatures, units, limitations, and existing tests from source before writing analysis code. When local source may be stale, also check current upstream `equinor/neqsim`; do not classify a capability as missing from a skill description or mirrored tree alone. Then, in `capability_assessment.md`: - fill **§4 Skills to Load** from `skill_search.py`, - fill **§4b Agents to Delegate To** from `agent_search.py` (agent, repo, the skills it loads, and why it was picked), - fill **§4c Workflow Plan** — choose single-agent, a `router.agent.md` composition pattern, or a declarative workflow (MCP `composeWorkflow` / `composeMultiServerWorkflow`, or an `engineering-harness` study) when the task spans ≥3 disciplines. Prefer delegating to a specialist agent over re-loading its skills manually, so its governance and internal workflow are reused (this is how "all functionality gets utilized"). - add a **NeqSim Code Functionality Reviewed** table containing each required capability, exact class and method signatures, source/test/JAR evidence, availability (`Available`/`Partial`/`Missing`), units/constraints, and how the task will use it. Persist the plan so it survives context loss and feeds the report: - keep `step1_scope_and_research/agent_plan.json` (raw ranking, audit trail), - checkpoint it via `progress.store_context("agent_workflow_plan", {...})`, - add an `agent_workflow_plan` object to `results.json` (see schema) with an `invocation` (`subagent` | `pattern` + `reason` | `skill-only`) per agent, and mention the agents/workflow used in the `approach` / `method_summary` text so the generated report documents *how* the task was solved. Then actually invoke the lead agent for the task family (§5 table) — a plan that lists agents as "(pattern)" without a reason is a finding, not a plan. For Quick tasks, write a short manual capability + agent note in `notes.md` unless a capability gap is suspected. 6. **Literature scout** (Standard/Comprehensive only). Invoke `@literature-scout` to populate `step1_scope_and_research/references/` and the `## Literature & Reference Documents` section of `notes.md`. ### Autonomous investigation (when the symptom / cause is NOT given) For root-cause, deviation, under-performance, trip, or "why is this happening?" tasks where the user has **not** told you what to look for, do not ask them. Run the investigation loop from `neqsim-autonomous-investigation` BEFORE fixing a scope, and use **all** available data and tools, not just the one tag/unit named: 1. **Gather all data** — pull the full historian tag map (`@plant-data` / `neqsim-plant-data` / `@seeq.connect`), plus alarms/events, maintenance history, and STID design limits when available. Save snapshots to the task folder. 2. **Observe** — hand the whole tag map (no symptom) to `RootCauseAnalyzer.analyzeAutonomous(tagToEquipment)` (or MCP `runRootCauseAnalysis` with `symptom` omitted / `"AUTO"`). It auto-detects abnormal tags (`AnomalyScanner`), discovers lead-lag relationships (`RelationshipGraph`), and classifies them against the flowsheet topology (`CausalTopologyModel`). 3. **Hypothesize / test / discriminate** — form ≥3 competing hypotheses (including "data/instrument artifact"), verify with the NeqSim simulation, and report the **discovered relationships and the discriminating test**, not just a single conclusion. Treat any supplied classification as a hint to challenge. For operational plant tasks involving P&ID symbols, valve actions, live plant data, active train state, isolation, evacuation, or dynamic response, load `neqsim-pid-process-operations` together with `neqsim-technical-document-reading`, `neqsim-process-extraction`, `neqsim-plant-data`, `neqsim-controllability-operability`, and `neqsim-dynamic-simulation`. Keep public outputs plant-agnostic; use private prompt files or private skills for site-specific document sources, historian source names, tag maps, operator procedures, and company requirements. Use `neqsim.process.operations` for Java deliverables and MCP `runOperationalStudy` for tool-based tag-map validation, field-data application, valve scenarios, and controller-response metrics. For fast liquid-hammer cases, route the extracted STID geometry, tagreader event window, and valve closure schedule through `neqsim-water-hammer` and MCP `runWaterHammer` before deciding whether a detailed surge study is required. ---