validate-parameters

name	validate-parameters
description	Use after the napkin_math pipeline has produced parameters.json (from extract-parameters-from-digest or extract-parameters-from-full) to validate it against the 16 structural checks the rest of the pipeline assumes. Writes validation.json next to parameters.json. Deterministic Python — no LLM call.

Validate napkin_math parameters.json

Overview

A thin wrapper around experiments/napkin_math/validate_parameters.py. The script reads parameters.json and emits validation.json next to it. Output shape is what summarize_assessment.py consumes (named checks_performed list + per-violation rule_id/severity/path/message/suggested_fix).

This replaces the older LLM-driven validate-parameters skill, which was written against an earlier schema and rejected the output_name/output_unit fields the digest extractor is required to emit. The Python validator runs in milliseconds, costs no tokens, and is deterministic.

When to Use

• The extractor (digest or full) has just produced parameters.json and the pipeline needs validation.json before running scenarios or Monte Carlo
• The user asks to "validate", "check", or "lint" a parameters file
• After hand-editing parameters.json, to confirm the result is still structurally valid

Not for: regenerating the parameters (use extract-parameters-from-digest or extract-parameters-from-full), generating bounds, or running calculations.

Workflow

1. Get the parameters path. If the user did not provide one, ask. Do not guess.

2. Invoke the script. Requires Python 3.11+ (no extra deps):

   /opt/homebrew/bin/python3.11 experiments/napkin_math/validate_parameters.py \
     --parameters <path>/parameters.json \
     [--output    <path>/validation.json]
   

   Default output: <dir-of-parameters>/validation.json. Exit code 0 on valid: true, 1 on validation errors, 2 on JSON parse failure. The script prints the output path on stdout.

3. Report back. Tell the user the output path, the verdict (valid / INVALID), and the error count. If there are violations, quote the first few rule_id + message pairs verbatim. Don't paraphrase the verdict — the rule ids are how downstream tooling routes the next step.

The 16 checks

Check	Severity bias	What it checks
json_parse	ERROR	the file parses as JSON (failure handled with a json_parse violation)
top_level_structure	ERROR	plan_summary + four arrays present
required_fields	ERROR	each entry carries its required keys
array_length_caps	ERROR	≤8 key_values, ≤5 derived_questions / missing_values_to_estimate / recommended_first_calculations
global_id_uniqueness	ERROR	ids unique across all four arrays
snake_case_ids	ERROR	ids match ^[a-z][a-z0-9_]*$
depends_on_declared	ERROR	every id in depends_on matches a declared id or output_name
formula_rhs_declared	ERROR	every snake_case identifier on the RHS of formula_hint is declared (or is the entry's own output_name); built-ins like min/max are exempt
fraction_value_range	ERROR	unit == "fraction" ⇒ value in [0, 1] or null
comment_word_caps	ERROR	key_value comment ≤25 words
source_text_word_caps	ERROR	key_value source_text ≤20 words
output_name_present_when_formula_hint	ERROR	non-empty formula_hint ⇒ output_name not null
output_unit_present_when_formula_hint	ERROR	non-empty formula_hint ⇒ output_unit not null
no_dead_end_variables	ERROR	every key_value and missing_value is consumed (transitively) by some calculation
threshold_friendly_naming	WARN	output_names ending in _gap / _deficit / _shortfall are flagged because they read ambiguously under a >= 0 / <= 0 threshold
shared_pool_legitimacy	(no-op)	listed in checks_performed for completeness; enforcement is upstream in the extractor's system prompt (requires reading source narrative to verify pool legitimacy, which is not a structural check)

valid is true iff error_count == 0. WARN-level findings do not invalidate the file.

Output shape

{
  "valid": true,
  "error_count": 0,
  "warn_count": 0,
  "violations": [],
  "summary": {
    "counts": {"key_values": 8, "derived_questions": 3, "missing_values_to_estimate": 4, "recommended_first_calculations": 5},
    "rule_id_breakdown": {},
    "checks_performed": ["json_parse", "top_level_structure", ...]
  }
}

The summary.checks_performed list is what summarize_assessment.py surfaces as the "Validated" line under ## Confidence and trust boundaries. Keep it as the authoritative list of what the validator actually ran.

Common mistakes

Mistake	Fix
Running before parameters.json exists	Run the extractor first (extract-parameters-from-digest or extract-parameters-from-full).
Treating WARN findings as blocking	They are not. valid is set only by ERROR-level findings.
Editing parameters.json by hand to silence a no_dead_end_variables ERROR	The right fix is usually to add a calculation that consumes the variable, not to drop the variable. Re-validate after either change.
Editing parameters.json to silence a threshold_friendly_naming WARN	If the output is actually threshold-tested, rename and flip the formula sign. If it isn't, the WARN is just advisory.

Reference

• Script (authoritative): experiments/napkin_math/validate_parameters.py
• Companion skills: ../extract-parameters-from-digest/SKILL.md, ../extract-parameters-from-full/SKILL.md, ../generate-bounds/SKILL.md, ../monte-carlo/SKILL.md, ../summarize-assessment/SKILL.md
• Example output: any validation.json under experiments/napkin_math/output/<version>/<plan>/