metrics-interpretation

name	metrics-interpretation
description	Use when a number moved unexpectedly and you need to understand why before deciding what to do. Triggers on: "지표가 떨어졌어", "retention 하락", "conversion drop", A/B test 결과 해석", "KPI 이상", "metrics dropped", "why did this metric change", dashboard 분석해줘".
type	workflow
theme	pm-data
best_for	["Diagnosing an unexpected metric drop or spike","Interpreting A/B test results and making a ship decision","Turning a weekly metrics review into action items"]
scenarios	["우리 retention이 갑자기 떨어졌어. 뭐가 문제인지 분석해줘.","Our conversion dropped 15% last week. How do I figure out why?","A/B test results are in — how do I interpret them and decide?","Weekly 지표 리뷰에서 이상한 숫자가 나왔어. 분석해줘.","How do I know if this metric change is real or just noise?","DAU 급락 원인을 체계적으로 찾는 방법 알려줘."]
estimated_time	30-60 min
compatibility	{"recommended":["think-tool"],"optional":["mcp-reasoner"],"remote_mcp_note":"think-tool이 있으면 세그멘테이션 가설을 체계적으로 검토하고 인과관계와 상관관계를 구분하는 데 도움이 됩니다. Claude 설정 → MCP Servers에서 remote SSE 엔드포인트를 추가하세요."}

Metrics Interpretation

Turning dashboard numbers into decisions — by finding what actually moved, why it moved, and what to do about it.

Core Principle

A metric change is not a diagnosis. It is a symptom. Your job is to work from symptom to cause to action, without skipping steps. Jumping from "DAU dropped 15%" to "let's ship a feature" skips the part where you find out whether it was a data pipeline issue, a seasonal effect, a competitor launch, or a real product problem.

---

The Investigation Framework

Use this three-step sequence every time a metric changes unexpectedly.

Step 1 — What moved?

Before investigating causes, confirm the change is real.

Dashboard reading checklist:

• Is the time range set correctly? (Day-over-day vs. week-over-week vs. month-over-month can show opposite signals)
• Is this a data pipeline issue? (Check if raw event counts dropped, not just derived metrics)
• Did the metric definition or tracking code change recently? (Check changelog, data team Slack)
• Is this affecting all platforms or just one? (iOS vs. Android vs. web)
• Is this affecting all user segments or just one? (new vs. returning, geo, cohort, plan tier)
• Is this within normal variance? (Check the same metric for the same period last year, last month)
• Did anything launch recently? (Feature release, marketing campaign, pricing change)

Output of Step 1: Confirm that the change is real, quantify the size, and isolate which segment(s) it affects.

---

Step 2 — Why did it move?

Work top-down: start with the highest-level segmentation, then drill.

Segmentation approach:

1. By acquisition channel — Did traffic quality change? (Paid users vs. organic vs. referral)
2. By user cohort — Is it affecting new users, existing users, or both?
3. By geography — Is it concentrated in one region?
4. By device / platform — iOS, Android, web, or specific browser/OS version?
5. By feature area — Which part of the product does the drop trace back to? (Funnel analysis)