Use this skill when analyzing product funnels, running cohort analysis, measuring feature adoption, or defining product metrics. Triggers on product analytics, funnel analysis, cohort analysis, feature adoption, north star metric, AARRR, retention curves, and any task requiring product data analysis or metrics design.
Use this skill when analyzing product funnels, running cohort analysis, measuring feature adoption, or defining product metrics. Triggers on product analytics, funnel analysis, cohort analysis, feature adoption, north star metric, AARRR, retention curves, and any task requiring product data analysis or metrics design.
Instrument before you need data - Tracking is a prerequisite, not an afterthought.
Add instrumentation when a feature ships, not when a stakeholder asks "do we track that?"
Retrofitting events means losing the baseline period and the ability to compare pre/post.
Define metrics before building features - Before writing a line of code, agree on
what success looks like and how it will be measured. A feature without a success metric
cannot be evaluated and cannot be killed. Write the metric definition into the spec.
Segment everything - Aggregate numbers hide the truth. Always break down metrics
by user segment (new vs. returning, plan tier, acquisition channel, geography) before
drawing conclusions. An overall retention rate that looks healthy can mask a collapsing
new-user cohort.
Retention is the ultimate metric - Acquisition and activation are table stakes.
Retention - whether users come back and get repeated value - is the only signal that
proves product-market fit. A product with strong retention can fix acquisition; a
product with broken retention cannot be saved by growth spend.
Correlation requires investigation, not celebration - Two metrics moving together
is a hypothesis, not a conclusion. Before attributing causation, check for confounders,
test the relationship with a controlled experiment, and document the evidence. Acting
on spurious correlations wastes engineering capacity and can harm users.
Core concepts
Event taxonomy
Events are the atoms of product analytics. An event represents a discrete user action
(or system action) at a point in time. A well-designed taxonomy makes querying intuitive
and avoids the "event graveyard" where hundreds of poorly named events accumulate with
no documentation.
Naming convention:object_action in snake_case. The object is the thing being acted
on; the action is what happened.
A funnel is an ordered sequence of steps a user must complete to reach a goal. Funnel
analysis reveals where users drop off and quantifies the conversion loss at each step.
Key measurements:
Step conversion rate - users who completed step N+1 / users who completed step N
Overall conversion rate - users who completed the final step / users who entered step 1
Time-to-convert - median and 90th percentile time between first and last step
Drop-off point - the step with the steepest conversion decline
Funnels should be analyzed with a defined window (e.g., within 7 days, within a single
session) to avoid counting users who convert months later by coincidence.
Common funnels by product type:
Product type
Acquisition funnel
Activation funnel
SaaS
Landing page -> Sign up -> Verify email -> First login
A cohort is a group of users who share a defining characteristic at a specific point in
time - most commonly the week or month they first signed up. Cohort analysis tracks how
that group's behavior evolves over time.
Reading across a row shows how a specific cohort retains over time. Reading down a column
shows whether a given time-since-signup period is improving or degrading across cohorts.
Improvement down a column - newer cohorts retaining better than older ones - is the
strongest early signal that product improvements are working.
Behavioral cohorts group users by an action rather than signup date (e.g., users who
completed onboarding vs. those who skipped it). Comparing behavioral cohorts quantifies
the impact of a specific behavior on downstream retention.
Retention curves
A retention curve plots the percentage of a cohort that remains active over successive
time periods. The shape of the curve matters as much as the final number.
Curve shapes:
Flat decay to zero - all users eventually churn; the product has no habit-forming
loop. Fundamental product problem.
Decaying to a stable floor - some users churn, but a core group stays. The floor
percentage is the product's "true retention." The goal is to raise the floor.
Smile curve (recovery) - users churn, then some return. Common in seasonal or
lifecycle products. Worth understanding the re-activation trigger.
D1 / D7 / D30 benchmarks by category (mobile apps):
Category
D1
D7
D30
Social / Messaging
40%+
20%+
10%+
Utilities
25%+
10%+
5%+
Games
35%+
15%+
7%+
Productivity (SaaS)
60%+
40%+
25%+
Metric hierarchy
A healthy metrics framework has three tiers. Conflating them creates confusion about
what the team is optimizing for.
North star metric - The single number that best captures the value delivered to users
and predicts long-term business success. It is a lagging indicator that changes slowly.
Examples: weekly active users completing a core action, number of projects with 3+
collaborators, monthly transactions processed.
Rules for a good north star:
It measures delivered value, not activity (DAUs alone is not a north star)
One team cannot game it without genuinely helping users
It is understandable by every person in the company
It moves on a relevant timescale (not too fast to be noisy, not too slow to provide signal)
Input metrics (leading indicators) - The behaviors that causally drive the north star.
These are actionable by product and engineering teams within a quarter. Examples: new user
activation rate, core action completion rate, feature engagement depth.
Health metrics (guardrails) - Metrics that must not regress while optimizing input
metrics. Examples: p99 API latency, error rate, customer support ticket volume, churn rate
for existing paid users. Health metrics prevent optimizing one thing by breaking another.
Common tasks
Define a metrics framework - north star + input metrics
Start with the business model: what user behavior creates sustainable revenue?
Identify the "aha moment" - the action that correlates most strongly with long-term retention
Express the north star as: [frequency] + [users] + [core action] - e.g., "weekly active
users who create at least one report"
Work backwards to list 3-5 behaviors that lead users to the north star
Map each behavior to a measurable event in the taxonomy
Define health metric guardrails for latency, errors, and churn
Document the framework in a single shared doc; every team should reference it
Build funnel analysis - conversion optimization
Define the goal event (purchase, activation, subscription) and work backwards to identify
each prerequisite step
Instrument every step with a consistent event if not already tracked
Set a conversion window appropriate to the product (1 session, 7 days, 30 days)
Compute step-by-step and overall conversion rates segmented by acquisition channel,
device type, and user plan
Identify the step with the highest absolute drop-off (not just lowest rate)
Generate hypotheses for the drop-off (UX friction, value not communicated, technical error)
Design experiments or targeted qualitative research to test hypotheses before building
Run cohort analysis - retention curves
Define the cohort grouping: signup week/month is the default; behavioral cohorts are
more diagnostic
Define "active" precisely: did the user complete the core value action, not just log in
Pull retention table for the last 6-12 cohorts
Plot retention curves and identify the stable floor (if one exists)
Compare cohorts over time: are newer cohorts retaining better than older ones?
Segment the best-retaining users: what did they do differently in their first week?
Translate the behavioral difference into a product hypothesis to test
Measure feature adoption - adoption lifecycle
Track four stages and their associated metrics:
Stage
Definition
Metric
Awareness
User sees the feature exists
Feature surface impression rate
Activation
User tries the feature at least once
First-use rate among eligible users
Adoption
User uses the feature repeatedly
Feature DAU/MAU ratio
Habit
Feature use is embedded in user's regular workflow
Feature retention at D30
Report adoption as a funnel: of all eligible users, what % reached each stage? Separately
track adoption among new users vs. existing users - adoption patterns often differ sharply.
Set up event taxonomy - naming conventions
Audit existing events to identify duplicates, inconsistencies, and orphaned events
Establish the object_action naming standard; document exceptions
Define the universal property set required on every event
Create a living event registry (spreadsheet or data catalog) with: event name, trigger
condition, owner, date added, properties, and example payload
Add instrumentation to the PR checklist: new features must include an event spec
Set a quarterly review to deprecate events with no active queries
Analyze A/B test results - statistical significance
Confirm the experiment was designed correctly before reading results: random assignment,
no novelty effect contamination, sufficient sample size via pre-test power calculation
Identify the primary metric and guardrail metrics upfront; do not add them post-hoc
Check for sample ratio mismatch (SRM): if the assignment split diverges more than 1-2%
from the intended ratio, the experiment is likely biased and results are invalid
Calculate statistical significance using the appropriate test (z-test for proportions,
t-test for means); use a two-tailed test unless there is a pre-registered directional
hypothesis
Report confidence intervals, not just p-values - a statistically significant but tiny
effect may not justify the maintenance cost
Check guardrail metrics for regressions before declaring a winner
Segment results by user cohort: a treatment that helps new users but hurts power users
is not a win
Create product dashboards - by audience
Build separate views for different audiences; combining them creates noise for everyone.
"Users who use feature X retain 30% better, so we should push everyone to feature X." X may be a symptom of already-engaged users.
Run a controlled experiment before attributing causation. Or instrument the counterfactual with propensity matching.
Moving the goalposts
Switching the primary A/B test metric after results come in because the original metric showed no effect.
Pre-register primary and guardrail metrics before the experiment starts. Honor the pre-registered outcome.
Ignoring qualitative signal
Optimizing quantitative metrics while ignoring support tickets, user interviews, and session recordings that explain the why.
Quantitative metrics tell you what is happening. Qualitative research tells you why. Both are required.
Gotchas
A/B test results are invalid if you peek before reaching the required sample size - Checking results daily and stopping when p < 0.05 is reached inflates the false positive rate to 30%+ (compared to the nominal 5%). This is p-hacking. Pre-calculate the required sample size using a power analysis before the experiment starts and do not evaluate results until that size is reached.
Funnel conversion windows that are too long inflate conversion rates - A 90-day conversion window for a trial-to-paid funnel will show a higher conversion rate than a 14-day window, but it mixes cohorts and obscures actual purchase latency. Choose conversion windows that match the actual product cycle; validate them by checking the distribution of time-to-convert before locking in a window.
Event naming changes retroactively break historical queries - Renaming user_signup to account_created splits the event stream at the migration date. Any retention or funnel query that spans the rename returns incomplete data silently. Before renaming an event, ensure both the old and new names are captured in parallel during a transition period, and update all dashboards and queries before deprecating the old name.
Session ID reuse across app restarts can merge separate user journeys - If your session ID is a persistent device identifier rather than a time-bounded session token, all activity from the same device over weeks may appear as one enormous session. This corrupts session-level funnel analysis. Define sessions with an inactivity timeout (30 minutes is standard) and generate new session IDs after each timeout.
North star metrics that include internal users overcount value delivered - If your product's north star includes employee accounts, test accounts, or bot activity, the metric is inflated by non-customer usage. Filter internal users from all product metrics from the start. Retroactively excluding them mid-measurement creates discontinuities that look like regressions.
References
For detailed content on specific sub-domains, read the relevant file from
references/:
references/metrics-catalog.md - Complete product metrics by category with definitions,
formulas, and benchmark guidance. Load when calculating specific metrics from raw data.
references/funnel-methodology.md - Deep dive on funnel construction, multi-step
vs. branching funnels, time-window selection, statistical significance for funnel
comparisons, and advanced segmentation techniques. Load when building or debugging
complex funnels.
references/feature-adoption.md - Feature adoption lifecycle measurement, adoption
scorecards, kill criteria for underperforming features, and adoption benchmarks by
product category. Load when measuring or planning feature launches.
Only load a references file if the current task requires deep detail on that topic.
Companion check
On first activation of this skill in a conversation: check which companion skills are installed by running ls ~/.claude/skills/ ~/.agent/skills/ ~/.agents/skills/ .claude/skills/ .agent/skills/ .agents/skills/ 2>/dev/null. Compare the results against the recommended_skills field in this file's frontmatter. For any that are missing, mention them once and offer to install:
