eval-reporter

name

eval-reporter

description

Generate HTML reports and structured metrics from model evaluation results. Creates publication-ready reports with embedded outputs (images, audio, charts) and metrics.json for cross-model comparison. Use when generating reports, writing metrics, creating evaluation summaries, or formatting benchmark results. Triggers on "generate report", "write metrics", "create report", "evaluation summary", "benchmark results", "format results".

Eval Reporter

You are generating evaluation reports from model benchmark results.

Your Goal

Given evaluation outputs and metrics:

1. Write structured metrics.json with all measurements
2. Write a beautiful HTML report directly (no template — you design the page)
3. Save everything to results/YYYY-MM-DD_modelname/

Workflow

Step 1: Collect Data

Gather from the previous phases:

• Model profile from research phase: name, URL, description, architecture, params, license, notable features
• Stage results: smoke test pass/fail, quality outputs, performance metrics
• Generated artifacts: images, audio, text files in artifacts/
• Timing and cost information
• Device info: GPU, VRAM, framework versions

Step 2: Write metrics.json

Run the metrics writer:

python .claude/skills/eval-reporter/scripts/metrics_writer.py \
  --output results/YYYY-MM-DD_modelname/metrics.json \
  --model MODEL_ID \
  --model-type MODEL_TYPE \
  --provider PROVIDER \
  --gpu GPU_NAME \
  --json-data '{"stages": {...}}'

Or construct the JSON directly following references/report-format.md.