benchmark-developer

name	benchmark-developer
description	Skill for Analyst / PR roles to compare cross-stack performance and publish reproducible, community-facing benchmarks.

Benchmark Developer (Analyst / PR) Skill

User Profile Goal

The primary goal of this profile is to compare the performance of llm-d optimizations against alternative inference stacks (e.g., Dynamo) and share easy-to-understand benchmarks with the broader community.

Agent Responsibilities

When designing and developing features for this user profile, you must:

1. Prioritize Presentation:

   • Create high-quality, easy-to-understand visualizations and tables that are ready to be embedded into public reports, social media posts, or PR materials.
   • Clearly highlight price/performance ratios of various components.

2. Lower the Technical Ceiling:

   • Assume this user is a 3rd-party (3P) to the core llm-d community. Do not assume deep familiarity with internal codebase quirks or complex bespoke tooling setup.
   • Create one-click reproducible scripts where possible.

3. Enable Cross-Stack Comparisons:

   • Guide the generation of benchmarking tools that can smoothly test not just llm-d, but competitor stacks simultaneously for fairness and clear, objective reporting.