메뉴
Write a hypothesis, define success metrics, and plan a holdout strategy. Use when designing A/B tests or experiment plans.
Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.
SOC 직업 분류 기준
| name | craft-experiment-design |
| description | Write a hypothesis, define success metrics, and plan a holdout strategy. Use when designing A/B tests or experiment plans. |
| suggest_when | User wants to test something, says "A/B test", "experiment", "should we test this", "hypothesis", or is planning a holdout strategy for a feature launch. |
Write a hypothesis, define success metrics, and plan a holdout strategy.
You want to run an A/B test but need to get the plan straight first. This skill helps you go from "we should test this" to a well-structured experiment design that your team and data scientists can review.
You are an experienced product manager and experimentation specialist.
Here is what I want to test:
<context>
$ARGUMENTS
</context>
> If the above is blank, ask the user: "{{DESCRIBE THE CHANGE YOU WANT TO TEST AND WHY}}"
Help me design an experiment plan that includes:
1. **Hypothesis** — A clear, falsifiable statement in the format: "If we [change], then [outcome], because [rationale]."
2. **Primary Metric** — The single metric that determines success or failure.
3. **Secondary Metrics** — 2-3 supporting metrics to watch for unintended effects.
4. **Guardrail Metrics** — Metrics that must not degrade (e.g., error rates, latency, retention).
5. **Audience & Allocation** — Who should be in the test? What percentage split do you recommend?
6. **Holdout Strategy** — Should we maintain a holdout group after the test? Why or why not?
7. **Duration Estimate** — How long should we run the test and what assumptions drive that?
8. **Risks & Considerations** — What could go wrong or bias the results?
Be specific. Use real metric names where possible. Call out any assumptions I should validate with data or eng.
Instruments a pull request with Amplitude analytics that conform to the project's existing taxonomy. Reads the tracking plan via the Amplitude MCP server (events, properties, naming conventions), analyzes the PR diff to find the few user actions genuinely worth tracking, detects the codebase's SDK and tracking patterns, and adds instrumentation that matches both. Optionally (opt-in) stages new events and properties on an Amplitude tracking-plan branch for data-governance review. Use when asked to "instrument this PR", "add analytics to this change", "add tracking", "add Amplitude events", "instrument this feature", or "what should I track here".
Diagnoses product health by cross-referencing Amplitude analytics (dashboards, charts, funnels, feedback, AI agent analytics), optionally Datadog (errors, latency, stack traces), and optionally Slack (qualitative feedback, bug reports, feature requests). Identifies what's broken, what's working, and what to do about it — with root causes, not just symptoms. Use when asked to "diagnose my product", "what's going on", "product health check", "what's broken", "where are users struggling", "give me a product diagnosis", or "what should I focus on".
Use this skill whenever a user wants to improve existing pages on their website to get cited more by AI models — whether they say "our pages aren't getting cited", "improve this page for AI visibility", "which of our pages should we update", "make this article more cite-worthy", "our competitors are getting cited instead of us", "update our content for AI search", or any variation where the goal is improving an existing asset rather than creating something new. This skill pulls owned pages from AI Visibility, identifies which ones have citation potential but are underperforming, compares them against the external pages that are winning citations on the same topics, and produces section-level rewrites or a full-page update — then pushes the revision to the CMS as a draft. Trigger even if the user just says "help me get cited more" or "why is [competitor] getting cited instead of us".
Use this skill whenever a user wants to win AI citations on prompts that competitors currently dominate — whether they say "competitors are getting cited instead of us", "we're losing on these prompts", "how do I outrank [competitor] in AI answers", "find prompts where we should be winning", "create content to beat [competitor]", or any variation where the goal is capturing AI share on prompts a competitor currently owns. This skill pulls competitor visibility data from AI Visibility, identifies the specific prompts where competitors win and Amplitude is absent, clusters them by intent, and produces targeted comparison pages, alternatives content, or rebuttal assets — then pushes drafts to CMS. Trigger on any mention of competitor, prompt hijack, outrank, or "why is [competitor] getting cited instead of us".
Use this skill whenever a user wants to turn AI Visibility data into published content — whether they say "find content gaps", "what should we write about", "which topics have low visibility", "help me get cited by AI models", "create a blog post from our AI Visibility gaps", "we're losing to competitors on these prompts", or any variation where they want to go from AI visibility weakness to a draft article, landing page, or FAQ. This skill connects directly to Amplitude AI Visibility data (topics, prompts, visibility scores, citations, competitor data, full LLM responses and sources) and produces a publish-ready content brief plus full article draft. If the user mentions CMS (WordPress, Webflow, Contentful, Sanity, HubSpot, Ghost, Shopify), also trigger this skill to push the draft directly. Trigger even if they just say something vague like "what content should we create?" in an AI Visibility context.
Use this skill whenever a user wants to test content variants before publishing to find which one will get cited most by AI models — whether they say "which version of this content will perform better", "test this article before we publish", "simulate how AI will respond to this content", "which angle should we use", "generate content variants and pick the winner", "run a simulation before publishing", or any variation where the goal is data-driven content selection rather than gut-feel publishing. This skill takes an identified content opportunity, generates 2–3 distinct variants with different angles or structures, scores them against actual AI model responses from AI Visibility, references the Simulate Changes feature for pre-publish validation, and produces a clear recommendation on which variant to publish — then pushes the winner to CMS. Trigger on any mention of "simulate", "test variants", "which performs better", "A/B content", or "before we publish".