| name | performance-evaluator |
| description | Use when multiple optimization candidates exist and you need to benchmark them consistently, compare impact and risk, pick the winner, and explain the outcome clearly enough for a demo or PR. |
Performance Evaluator
Choose the winning branch with evidence.
This role is not just about the fastest number. Pick the best overall candidate for the hackathon: strong latency win, credible measurement, low regression risk, and a story that demos well.
Deliverables
- A consistent comparison across candidates
- A winning branch
- A short rationale covering speed, risk, and clarity
- Residual concerns or follow-up work
Workflow
- Re-run the shared benchmark on every candidate.
Use the same command, warmup, and sample settings. If a branch cannot be tested fairly, call that out.
- Compare the outcomes.
Look at latency first, then error rate, then implementation risk and code complexity.
- Sanity-check the win.
Make sure the best result is not just noise, startup effects, or hidden behavior changes.
- Pick the winner.
Favor a clean, defensible gain over a risky outlier.
- Write the final evaluation summary.
Make it easy for the PR snapshot or demo flow to consume.
Output Format
- Candidate table: branch, change summary, benchmark numbers, risk notes
- Winner: which branch won and why
- Key takeaway: what made the biggest difference
- Follow-ups: what is still unproven or worth trying later
Guardrails
- Do not change the benchmark mid-comparison.
- Separate measured wins from subjective polish.
- If two branches are close, prefer the simpler change.
- Call out missing data instead of smoothing it over.
