| name | audit-modeling-evidence |
| description | Audit existing mathematical-modeling data, backend ownership, experiments, innovation claims, results, constraints, reproducibility, and claim-to-evidence links. Use when the primary request is to check or validate an existing modeling project rather than solve a complete new problem. |
Audit Modeling Evidence
Audit without silently repairing, rerunning, or promoting unsupported claims. Return the smallest defensible repair and a clear status: pass, warn, failed, or blocked.
Workflow
- Identify source-of-truth files, active subquestions, evidence/data mode, ambition, workflow stage, result status, and audit scope.
- Select only applicable lanes: intake/data; backend; method/experiment; candidate validation; simulator/decision trace; innovation/readiness; frozen evidence/manuscript.
- Resolve
SKILL_ROOT as this Skill directory and invoke scripts by absolute quoted path.
- Run the smallest deterministic check first. Preserve input hashes, commands, environment, and artifact paths.
- Distinguish observed, inferred, diagnostic-only, validated, frozen, and out-of-scope evidence. Rewriting a claim cannot turn failed evidence into a pass.
- Report each issue with file, field/row, observed condition, required condition, affected downstream artifacts, and smallest repair.
Script routing
- Data/leakage:
scripts/audit_dataset.py.
- MATLAB capability and numerical ownership:
scripts/audit_backend_contract.py.
- Experiment budgets/runs:
scripts/register_experiment.py, scripts/run_experiment.py.
- Candidate evidence/promotion:
scripts/audit_candidate_evidence.py, scripts/promote_validated_candidate.py.
- Innovation and competition readiness:
scripts/audit_innovation_portfolio.py.
- Exact/surrogate decisions:
scripts/audit_decision_trace.py, scripts/check_transition_fidelity.py.
- Numeric MATLAB–Python round trip:
scripts/preflight_matlab_python_bridge.py, only when numerical values cross languages.
- Whole project:
scripts/audit_modeling_project.py.
- Optional registries:
scripts/init_evidence_bundle.py, scripts/audit_evidence_bundle.py.
- Gate applicability:
scripts/resolve_required_gates.py.
Conditional rules
- Accept equivalent official MATLAB MCP capability names; require a real call, recorded environment, and artifact. Local MATLAB is the next route; another backend needs a concrete auditable reason.
- Require challengers only when empirical or heuristic model comparison is meaningful. Analytical/exact work may use a bound, proof obligation, limiting case, or independent checker.
- Apply action masks only to constructive/policy search where illegal actions can enter the generated set.
- Choose exact/surrogate fidelity cases from horizon, action, branch, boundary, and rare-state coverage; no universal transition count is sufficient.
- Separate verification, reproducibility, sensitivity, robustness, and generalization. An independent rerun does not prove transfer.
- Audit Pareto claims on raw-objective dominance, span, coverage, stability, and selection rule.
- An innovation is not validated by a nonempty path string: evidence artifacts must exist inside the project. Negative candidates may be downgraded honestly.
- Freeze only validated claims with traceable hashes and artifacts. A self-review may approve internal freeze; user input is required only for a material interpretation choice or external submission.
Output
Return:
- source-of-truth lock and applicable gates;
- checks run and deterministic commands;
- pass/warn/failed/blocked findings with evidence paths;
- stale or affected downstream artifacts;
- validated versus diagnostic-only claims;
- smallest next repair and stop condition.
Read only references needed by the lane: references/data-and-reproducibility.md, references/matlab-native-workflow.md, references/matlab-mcp-setup.md, references/candidate-validation-contract.md, references/validation-playbook.md, references/experiment-budget-and-promotion.md, references/innovation-and-competition.md, references/exact-simulation-contract.md, references/discrete-event-scheduling.md, references/hard-constraint-action-mask.md, references/score-gap-analysis.md, or references/evidence-registry-contract.md.
