菜单
A/B test evaluation, cohort retention analysis, funnel metrics, and experiment-driven product decisions. Use when analyzing experiments, measuring feature adoption, diagnosing conversion drop-offs, or evaluating statistical significance of product changes.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/yonatangross/orchestkit --skill product-analytics命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
Chain patterns for multi-phase pipelines: MCP detection, handoff files, checkpoint-resume, worktree agents, CronCreate monitoring. Use when building or debugging a pipeline skill.
OrchestKit doctor for health diagnostics across manifest integrity, hook configuration, skill validation, agent frontmatter, MCP server connectivity, CC version compatibility, and permission rules. Reports issues with severity levels and auto-remediation suggestions. Validates component counts, detects orphaned entries, and checks CC version matrix compliance. Use when diagnosing plugin health, troubleshooting configuration issues, or running pre-release checks.
Inspects the OrchestKit telemetry pipeline for the current project — lists all known telemetry files with write counts, sizes, schema status, growth trend, and orphan detection. Use when verifying the observability pipeline is healthy, debugging a missing writer, or auditing which files have schema locks vs. which are drift-vulnerable. Read-only — never modifies telemetry files.
正在显示 SKILL.md
基于 SOC 职业分类
| name | product-analytics |
| license | MIT |
| compatibility | Claude Code 2.1.220+. |
| description | A/B test evaluation, cohort retention analysis, funnel metrics, and experiment-driven product decisions. Use when analyzing experiments, measuring feature adoption, diagnosing conversion drop-offs, or evaluating statistical significance of product changes. |
| tags | ["ab-test","cohort","retention","funnel","conversion","analytics","experiment","statistical-significance"] |
| context | fork |
| agent | product-strategist |
| version | 1.0.0 |
| author | OrchestKit |
| user-invocable | false |
| disable-model-invocation | false |
| complexity | medium |
| persuasion-type | guidance |
| metadata | {"category":"document-asset-creation"} |
| allowed-tools | ["Read","Glob","Grep","WebFetch","WebSearch"] |
Frameworks for turning raw product data into ship/extend/kill decisions. Covers A/B testing, cohort retention, funnel analysis, and the statistical foundations needed to make those decisions with confidence.
| Category | Rules | Impact | When to Use |
|---|---|---|---|
| A/B Test Evaluation | 1 | HIGH | Comparing variants, measuring significance, shipping decisions |
| Cohort Retention | 1 | HIGH | Feature adoption curves, day-N retention, engagement scoring |
| Funnel Analysis | 1 | HIGH | Drop-off diagnosis, conversion optimization, stage mapping |
| Statistical Foundations | 1 | HIGH | p-value interpretation, sample sizing, confidence intervals |
Total: 4 rules across 4 categories
Load rules/ab-test-evaluation.md for the full framework. Quick pattern:
## Experiment: [Name]
Hypothesis: If we [change], then [primary metric] will [direction] by [amount]
because [evidence or reasoning].
Sample size: [N per variant] — calculated for MDE=[X%], power=80%, alpha=0.05
Duration: [Minimum weeks] — never stop early (peeking bias)
Results:
Control: [metric value] n=[count]
Treatment: [metric value] n=[count]
Lift: [+/- X%] p=[value] 95% CI: [lower, upper]
Decision: SHIP / EXTEND / KILL
Rationale: [One sentence grounded in numbers, not gut feel]
Decision rules:
See rules/ab-test-evaluation.md for sample size formulas, SRM checks, and pitfall list.
Load rules/cohort-retention.md for full methodology. Quick pattern:
-- Day-N retention cohort query
SELECT
DATE_TRUNC(, first_seen) cohort_week,
( user_id) cohort_size,
(
activity_date first_seen
user_id )
( user_id) day_7_retention
user_activity
;
Retention benchmarks (SaaS):
See rules/cohort-retention.md for behavior-based cohorts, feature adoption curves, and engagement scoring.
Load rules/funnel-analysis.md for full methodology. Quick pattern:
## Funnel: [Name] — [Date Range]
Stage 1: [Aware / Land] → [N] users (entry)
Stage 2: [Activate / Sign] → [N] users ([X]% from stage 1)
Stage 3: [Engage / Use] → [N] users ([X]% from stage 2) ← biggest drop
Stage 4: [Convert / Pay] → [N] users ([X]% from stage 3)
Overall conversion: [X]%
Biggest drop-off: Stage 2→3 ([X]% loss) — investigate first
Optimization order: Fix the largest drop-off first. A 5-point improvement at a high-volume step is worth more than a 20-point improvement at a low-volume step.
See rules/funnel-analysis.md for segmented funnels, micro-conversion tracking, and prioritization patterns.
Plain-English explanations of the stats every PM needs. Load references/stats-cheat-sheet.md for formulas and quick lookups.
p-value in plain English: The probability that you would see a result this extreme (or more extreme) if the change had zero effect. p=0.03 means a 3% chance you're looking at random noise. It does NOT mean "97% probability the change works."
Confidence interval in plain English: The range where the true effect probably lives. "Lift = +8%, 95% CI [+2%, +14%]" means you are fairly confident the real lift is somewhere between 2% and 14%. If the CI includes zero, you cannot claim a win.
Minimum Detectable Effect (MDE): The smallest lift you care about detecting. Setting MDE too small forces impractically large sample sizes. Anchor MDE to business value — if a 2% lift is not worth shipping, set MDE = 5%.
Statistical vs practical significance: A result can be statistically significant (p < 0.05) but practically meaningless (lift = 0.01%). Always check both. A 0.01% lift that costs 6 weeks of eng time is not a win.
| Signal | Decision | Action |
|---|---|---|
| p < 0.05, CI excludes zero, guardrails green | SHIP | Full rollout, update success metrics |
| Positive trend, underpowered (p = 0.10–0.15) | EXTEND | Add runtime, do not peek again |
| p > 0.15, flat or negative | KILL | Revert, document learnings, re-hypothesize |
| Guardrail regression, any p-value | KILL | Immediate revert regardless of primary metric |
| SRM detected | INVALID | Fix assignment bug, restart experiment |
ork:product-frameworks — OKRs, KPI trees, RICE prioritization, PRD templatesork:monitoring-observability — Metric definition, alerting, and drift monitoringork:brainstorm — Generate hypotheses and experiment ideasork:assess — Evaluate product quality and risksrules/ab-test-evaluation.md — Hypothesis, sample size, significance, decision matrixrules/cohort-retention.md — Cohort types, retention curves, SQL patternsrules/funnel-analysis.md — Stage mapping, drop-off identification, optimizationreferences/stats-cheat-sheet.md — Formulas, test selection, power analysisVersion: 1.0.0 (March 2026)