ai-benchmark-distribution

name	ai-benchmark-distribution
description	Designs, publishes, and distributes AI benchmarks and evals as a growth channel. Covers what makes a benchmark worth publishing, the benchmark-to- distribution pipeline, writing rules for benchmark posts, platform-specific formats for HN and Twitter, and when NOT to publish. Use when planning a model release, comparing against alternatives, or turning internal eval results into external trust signals for a developer audience.
triggers	["/ai-benchmark-distribution","user wants to publish benchmark or eval results","user is planning distribution of model performance data","user is writing an HN or Twitter post about benchmark results","user wants to turn eval results into credibility content"]
role	workflow
version	1.0.0
sources	["ai-benchmark-distribution (growth-skills v1.0)"]
feeds	["pmm/content-review","pmm/positioning"]
related	["pmm/DOMAIN.md","pmm/ai-product-gtm/ai-trust-design","pmm/ai-product-gtm/ai-api-developer-gtm"]

AI Benchmark Distribution

Role / Purpose

Eval Strategist. For AI products competing on capability — foundation models, AI coding tools, AI infrastructure — benchmarks are not just technical artifacts. They are the primary distribution mechanism in the developer and research community. This skill is about designing that moment intentionally.

Note: the capability claims precision framework, unsubstantiated comparison rules, and benchmark methodology requirements are in pmm/DOMAIN.md (AI product content section). This skill covers the distribution pipeline — what to publish, where, how, and when.

---

Contract

This skill guarantees:

• No benchmark is published without a documented methodology (this is a hard gate, not a preference)
• The "when NOT to publish" check runs before the distribution plan is built
• Platform-specific format is defined for each distribution surface
• The 48-hour seed window is designed before public posting
• Failure cases appear in the same post as the headline result

---

Before starting

Confirm (ask or infer):

• Benchmark type — head-to-head comparison / task-specific deep dive / eval methodology contribution / community leaderboard entry?
• Result novelty — does this show something the audience doesn't already know?
• ICP relevance — is this the capability that matters to the target engineering audience?
• Methodology status — is the methodology fully documented and reproducible?
• Response availability — can the team respond to technical questions within 72 hours of posting?

---

Inputs

Required before proceeding:

• The benchmark result or eval finding
• The task, evaluation setup, and metric (must be fully described)
• The comparison baseline (or confirmation that none is claimed)
• The target distribution surfaces
• Response availability in the 48–72 hours after posting

---

Step 1 — Publish decision gate

Run this before any distribution planning. All three must be true to proceed:

PUBLISH DECISION GATE

Criterion 1 — Methodology is credible:
  [ ] Test set is fully described and could be reproduced by a third party
  [ ] Evaluation criteria and scoring are documented
  [ ] Comparison baseline is named explicitly (model name + version + date)
  IF any box unchecked → DO NOT PUBLISH. A benchmark without methodology is
  a marketing claim. It invites replication attempts that undermine trust.

Criterion 2 — The result is novel:
  [ ] Shows something the audience does not yet know about the model or category
  [ ] Does not merely confirm what everyone already believes
  IF criterion fails → consider whether the distribution value justifies the effort;
  a benchmark that confirms the obvious gets no distribution.

Criterion 3 — ICP-relevant:
  [ ] The capability being measured matters to the target engineering community
  [ ] Example: a coding benchmark for a developer tool; a legal reasoning benchmark is not
  IF criterion fails → wrong benchmark for this audience; reconsider the claim.

ALSO CHECK — when NOT to publish (see Step 2)