inference-format-optimizer

name

inference-format-optimizer

description

Iterative benchmarking, evaluation, and algorithmic optimization of alternative A2UI inference formats (such as Express, Atom, and Elemental). Trigger when asked to: (1) Run optimization passes or loops on an inference format, (2) Evaluate or benchmark format accuracy, latency, or token efficiency, (3) Create parallel worktree subagents for format iteration, or (4) Benchmark format trade-offs against baselines.

Inference Format Optimizer

This skill provides procedural workflows, CLI orchestrators, decision guardrails, and subagent protocols for iteratively optimizing A2UI inference formats.

---

Quick-Start CLI Cheatsheet

All execution scripts live under scripts/ in this skill:

Action	Executable Command
Run Fast Validation Eval	python scripts/optimize_format.py --format <format>
Run Full Evaluation Suite	python scripts/optimize_format.py --format <format> --full
Test Parsing / Compilation	python scripts/optimize_format.py --format <format> --compile "(Card (Text \"Hi\"))"
Compare vs Baseline	python scripts/compare_results.py --baseline eval/iterative_format_optimizer/baselines/<format>/unbounded_run_meta.json eval/iterative_format_optimizer/logs/temp_optimization/
Archive Run Artifacts	python scripts/optimize_format.py --format <format> --archive --hypothesis "..." --status KEEP [--history-dir <path>]