菜单
Weekly product review. Reads KPI data from PostHog/Stripe, compares against targets, identifies funnel drop-offs, surfaces 2-3 hypotheses for improvement.
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
基于 SOC 职业分类
Reference copy of the operating contract for the defense lens at a human gate — input binding, the six sources of the case for crossing, the untrusted-input and exhibit-marker rules, the evidence whitelist, the degraded-input table, the severity/confidence/variance scales, the output contract, and the escalation table. NOT preloaded into any agent. agents/iron-loop/advocate-critic.md carries this contract in its own body; this file is loadable on demand through the Skill tool by any agent that needs to read the same rules.
Reviews AI-generated code for common pitfalls — over-engineering, missing edge cases, fabricated patterns, hallucinated imports, stale framework idioms, vacuous tests.
Detects AI-generated code that references non-existent packages, APIs, methods, or fabricated patterns.
Paranoid LLM red-team analyst — scans applications that call LLMs for OWASP LLM Top 10 v2 (2025) findings and maps them to MITRE ATLAS v5.4.0 adversary tactics.
Builds the import graph and detects circular dependencies, layer violations, instability mismatches, and cross-module coupling.
Detects which architecture pattern a codebase follows (Layered, Hexagonal, Clean, Onion, CQRS, DDD, Vertical Slice, Modular Monolith, Microservices) with confidence scoring across 7 languages.
| name | product-reviewer |
| description | Weekly product review. Reads KPI data from PostHog/Stripe, compares against targets, identifies funnel drop-offs, surfaces 2-3 hypotheses for improvement. |
| type | skill |
| tier | 2 |
| dispatch_protocol | v1 |
| confidence_calibration | enabled |
| parallel_safe | true |
| effort_budget | {"max_subagents":0} |
| when_to_load | ["product review","weekly review","kpi review","how is the product doing","activation drop-off","retention check","funnel analysis","north star","MRR review","churn analysis"] |
| related_skills | ["saas/posthog-analytics","saas/stripe-subscriptions","product/experiment-designer"] |
| effort_level | high |
| model | opus |
| tools | Read, Write, Bash, WebFetch |
Tier 2 specialist for the REVIEW step of the Product Loop (
docs/PRODUCT_LOOP.md). Persona-gated implicitly via CTO Chief: only dispatched when founder/pm/technical-founder requests. Converted to skill in CTOC v6.4 product-loop B2 sweep; refreshed for 2026 best practices.
You produce a weekly product review that answers: is the product working? Specifically:
< 5% monthly churn, > 30% activation rate. No target = no "good" definition = no review value. Color-code each row green / yellow / red against target.owner: <person> and due: <date>. Action items without owners die. CTOC writes them to .ctoc/product-loop/actions/<date>.yaml so they can be tracked across reviews.These are the patterns this skill is meant to catch in the existing review workflow — each becomes a severity: critical letter on the wire (per warnings-are-bugs).
good definition, no color coding. The review degrades into status reporting.team is no owner. The action evaporates.kpi_plan: plans/canvas/<slug>-kpis.yaml # the targets
posthog_export: .ctoc/product-loop/data/<date>.csv # OR call PostHog API
stripe_export: .ctoc/product-loop/data/stripe-<date>.json
date_range: { from: <last_review_date>, to: <today> }
prior_review: .ctoc/product-loop/reviews/<previous-date>.md # for trends
prior_actions: .ctoc/product-loop/actions/<previous-date>.yaml # for recap
segments: [free, pro, enterprise] # B2B segmentation, optional
Load prior_actions. For each action item, check status:
done with brief result.This is the FIRST section of the review.
const kpiPlan = readYaml(`plans/canvas/${slug}-kpis.yaml`);
const recentData = loadProductData(dateRange);
const priorReview = loadPriorReview();
const priorActions = loadPriorActions();
north_star:
id: weekly_active_invoices_sent # customer outcome, not activity
current_value: 1840
prior_value: 1720
delta_week: +7%
target_curve: "+5% w/w"
status: green
input_metrics:
- id: signup_completion
current_value: 68%
target: "> 60%"
status: green
- id: activation_rate
current_value: 22%
target: "> 30%"
status: red
delta_week: -3pp
trend: degrading
For the activation funnel signup_started → signup_completed → activation:
This week:
signup_started: 1000 (100%)
signup_completed: 680 (68%) ← drop of 32%
activation: 150 (15%) ← drop of 53% ← LARGEST DROP
Largest drop → primary opportunity. Spend the body of the review here.
Render the cohort triangle — each row is a signup-week cohort, columns are D1/W1/M1:
Cohort n D1 W1 M1
─────────────────────────────────────────
2026-05-12 1000 42% 18% --
2026-05-05 980 45% 21% 9%
2026-04-28 1020 44% 20% 11%
2026-04-21 950 46% 22% 12%
Trend: D1 stable, W1 slipping, M1 dropping. Cohorts are getting weaker — alarm.
For each segment in segments, recompute the worst KPI:
Aggregate free pro enterprise
activation 22% 18% 31% 47%
W1 retention 18% 12% 28% 42%
If aggregate hides a segment cliff, that's the real finding. Do NOT act on the aggregate alone.
Group subscription_canceled.reason events:
Reason Count %
─────────────────────────────────
too_expensive 12 40%
not_enough_value 8 27%
switched_to_competitor 5 17%
no_longer_needed 5 17%
The plurality reason is the lead.
From the largest drop + churn reason + segment view, surface 2–3 testable hypotheses; flag the recommended one:
hypotheses:
- id: simplify_onboarding
recommended: true
rationale: "53% of signups drop before activation — onboarding likely too long. Free-tier hit hardest (18% vs 47% enterprise)."
expected_impact: "+8-12% activation rate on free tier"
test_design: "A/B: 3-step vs 5-step onboarding flow"
next_step: "Dispatch [[experiment-designer]] with this hypothesis"
owner: "pm:alex"
due: 2026-05-26
estimated_duration_weeks: 2
- id: lower_free_tier_friction
rationale: "30% of users on Pro plan downgrade in month 2 → 'not enough value' (churn reason)"
expected_impact: "Reduce monthly churn by 1-2pp"
test_design: "Increase free tier limits temporarily, measure activation lift vs conversion loss"
next_step: "Founder decision required"
owner: "founder:tijn"
due: 2026-05-23
estimated_duration_weeks: 4
Output: .ctoc/product-loop/reviews/YYYY-MM-DD.md
# Product Review — 2026-05-19
**Prior review**: 2026-05-12
**Project**: freelance-invoices
## Recap of last week's actions
- [done] PM: Ship 3-step onboarding behind flag — shipped 2026-05-15.
- [carry-forward] FOUNDER: Decide on free-tier limits — still open, due 2026-05-23.
## North Star: Weekly Active Invoices Sent
1840 (prior 1720, +7% w/w) — GREEN against `+5% w/w` target.
## KPI snapshot
| KPI | Current | Target | Week Δ | Status |
|---|---|---|---|---|
| Signup completion | 68% | > 60% | +2pp | GREEN |
| Activation rate | 22% | > 30% | -3pp | RED |
| W1 retention | 18% | > 25% | flat | YELLOW |
| Free→paid | 4.1% | > 3% | +0.3pp | GREEN |
| Monthly churn | 7% | < 5% | +1pp | RED |
| MRR | $4,300 | growing | +$320 | GREEN |
## Drill: Activation (the red one)
### Funnel
signup_started → signup_completed → activation
1000 → 680 (68%) → 150 (22%)
Largest drop: signup_completed → activation (53%)
### Segments
| | aggregate | free | pro | enterprise |
|---|---|---|---|---|
| activation | 22% | 18% | 31% | 47% |
| W1 retention | 18% | 12% | 28% | 42% |
Free tier is the source of the aggregate weakness.
### Cohort retention triangle
| Cohort | n | D1 | W1 | M1 |
|---|---|---|---|---|
| 2026-05-12 | 1000 | 42% | 18% | -- |
| 2026-05-05 | 980 | 45% | 21% | 9% |
| 2026-04-28 | 1020 | 44% | 20% | 11% |
W1 slipping, M1 dropping → cohorts weakening over time.
## Churn analysis
40% "too_expensive" · 27% "not_enough_value" · 17% switched · 17% no_longer_needed
## Hypotheses (2–3 surfaced, 1 recommended)
1. **Simplify onboarding** [RECOMMENDED] — high confidence, +8–12% activation. Owner: pm:alex, due 2026-05-26.
2. **Increase free tier limits** — medium confidence, -1–2pp churn. Owner: founder:tijn, due 2026-05-23.
## Action items (this week)
| # | Action | Owner | Due |
|---|---|---|---|
| 1 | Design A/B for 3-step onboarding | pm:alex | 2026-05-26 |
| 2 | Decide on free-tier widening | founder:tijn | 2026-05-23 |
| 3 | Add segment filter to PostHog dashboard | eng:sam | 2026-05-22 |
## Open questions for founder
- Willing to widen free tier for 30 days to test the value gap hypothesis?
Also write .ctoc/product-loop/actions/YYYY-MM-DD.yaml so the NEXT review can recap.
If a hypothesis is selected, the founder/pm dispatches [[experiment-designer]] with the hypothesis.
response:
dispatch_id: <ulid>
protocol_version: 1
agent: product/product-reviewer
agent_version: 8.4.0
completed_at: <iso8601>
findings: []
synthesis:
review_path: .ctoc/product-loop/reviews/2026-05-19.md
actions_path: .ctoc/product-loop/actions/2026-05-19.yaml
north_star:
id: weekly_active_invoices_sent
delta_pct: 7
status: green
kpi_status:
green: 3
yellow: 1
red: 2
largest_funnel_drop:
from: signup_completed
to: activation
drop_pct: 53
segment_alert: true # set when Simpson's-paradox-style divergence detected
primary_hypothesis_id: simplify_onboarding
next_experiment_proposed: true
self_assessment:
coverage: 0.95
confidence_overall: HIGH
limitations:
- "Funnel data limited to events PostHog received; ad-blocker users not counted."
- "Churn reasons rely on user self-report (may not reflect true cause)."
- "Cohort retention requires >= 4 weeks of data before trends are reliable."
| Tool | Strengths | Best for | Note |
|---|---|---|---|
| PostHog | Open-source, self-hostable; analytics + session replay + feature flags + experiments + warehouse in one | Engineering-led startups, single-vendor | Use Insights → Funnels + Insights → Retention + SQL via HogQL for custom queries |
| Amplitude | Predictive cohorts (likelihood to activate / retain / churn), native A/B, AI-assisted analysis | Mid-stage B2C, when AI-suggested cohorts pay off | Heavier weight, separate billing-data integration |
| Mixpanel | Fastest funnel-building UX, strong retention cohort comparison | Teams that iterate on funnels weekly | Best ROI on retention-cohort UI |
| ChartMogul / Baremetrics / ProfitWell | Stripe/billing-native MRR, ARR, churn, expansion, NRR, LTV | Subscription revenue truth | Cross-check against PostHog event-derived revenue |
| Hex / Mode / Looker Studio | SQL notebooks for warehouse-level KPI rollups | When PostHog/Stripe aren't enough — joined warehouse views | Pin notebook to weekly review run |
| Linear / Notion | Action item tracking with owner + due date | Where the review's actions live after the review ends | Auto-create issues from actions/<date>.yaml |
| Slack / Discord rollup bots | Weekly digest posted to product channel: NSM + reds + recommended hypothesis | Distribution to the wider team | Keeps the metric alive beyond the room of reviewers |
Product review is a Python / TypeScript / SQL workflow in practice. Other stacks (C/C++/Java/C#) are rarely the front-of-house for KPI plumbing and are intentionally skipped per the skill's scope.
# Pull this week's activation funnel from PostHog and MRR from Stripe.
import os, requests, stripe
from datetime import datetime, timedelta, timezone
POSTHOG = os.environ["POSTHOG_API_KEY"]
PROJECT = os.environ["POSTHOG_PROJECT_ID"]
stripe.api_key = os.environ["STRIPE_API_KEY"]
since = (datetime.now(timezone.utc) - timedelta(days=7)).isoformat()
# HogQL: per-step funnel counts, last 7 days
hogql = """
SELECT event, count() AS n
FROM events
WHERE event IN ('signup_started','signup_completed','activated')
AND timestamp >= now() - INTERVAL 7 DAY
GROUP BY event
"""
r = requests.post(
f"https://us.posthog.com/api/projects/{PROJECT}/query/",
headers={"Authorization": f"Bearer {POSTHOG}"},
json={"query": {"kind": "HogQLQuery", "query": hogql}},
timeout=30,
)
r.raise_for_status()
funnel = {row[0]: row[1] for row in r.json()["results"]}
# Stripe MRR: sum active subscription items, monthly-normalized.
mrr = 0
for sub in stripe.Subscription.list(status="active", limit=100).auto_paging_iter():
for item in sub["items"]["data"]:
price = item["price"]
unit = price["unit_amount"] / 100
interval = price["recurring"]["interval"]
mrr += unit * (1 if interval == "month" else 1/12 if interval == "year" else 0)
print({"funnel": funnel, "mrr_usd": round(mrr, 2)})
// Post the weekly KPI rollup to Slack. Color-coded against targets.
// Runs on Node 18+ / Bun / Deno — global fetch, no deps.
type Kpi = { id: string; value: number; target: number; higher_is_better: boolean };
function statusEmoji(k: Kpi): string {
const hit = k.higher_is_better ? k.value >= k.target : k.value <= k.target;
if (hit) return ":large_green_circle:";
const margin = Math.abs(k.value - k.target) / k.target;
return margin < 0.1 ? ":large_yellow_circle:" : ":red_circle:";
}
export async function postRollup(webhookUrl: string, kpis: Kpi[], reviewUrl: string) {
const lines = kpis.map(k => `${statusEmoji(k)} *${k.id}*: ${k.value} (target ${k.higher_is_better ? ">=" : "<="} ${k.target})`);
const payload = {
text: `*Weekly Product Review*\n${lines.join("\n")}\n<${reviewUrl}|Full review>`,
};
const res = await fetch(webhookUrl, {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify(payload),
});
if (!res.ok) throw new Error(`webhook ${res.status}: ${await res.text()}`);
}
Activation funnel (7-day window):
-- Step counts and step-over-step conversion in one pass.
WITH steps AS (
SELECT event, COUNT(DISTINCT distinct_id) AS n
FROM events
WHERE event IN ('signup_started','signup_completed','activated')
AND timestamp >= NOW() - INTERVAL '7 days'
GROUP BY event
)
SELECT
MAX(CASE WHEN event='signup_started' THEN n END) AS started,
MAX(CASE WHEN event='signup_completed' THEN n END) AS completed,
MAX(CASE WHEN event='activated' THEN n END) AS activated,
ROUND(100.0 * MAX(CASE WHEN event='signup_completed' THEN n END)
/ NULLIF(MAX(CASE WHEN event='signup_started' THEN n END),0), 1) AS pct_complete,
ROUND(100.0 * MAX(CASE WHEN event='activated' THEN n END)
/ NULLIF(MAX(CASE WHEN event='signup_completed' THEN n END),0), 1) AS pct_activated
FROM steps;
Cohort retention (signup-week × weeks-since-signup). The DATE_TRUNC / DATE_DIFF syntax below is BigQuery/Trino/Snowflake-flavored; adapt to your warehouse dialect (Postgres uses EXTRACT(EPOCH FROM ...) / 604800 for week diffs):
WITH cohorts AS (
SELECT distinct_id,
DATE_TRUNC('week', MIN(timestamp)) AS cohort_week
FROM events
WHERE event = 'signup_completed'
GROUP BY distinct_id
),
activity AS (
SELECT e.distinct_id,
c.cohort_week,
DATE_DIFF('week', c.cohort_week, DATE_TRUNC('week', e.timestamp)) AS weeks_since
FROM events e
JOIN cohorts c USING (distinct_id)
WHERE e.event = 'session_started'
GROUP BY 1,2,3
)
SELECT cohort_week,
weeks_since,
COUNT(DISTINCT distinct_id) AS retained,
ROUND(100.0 * COUNT(DISTINCT distinct_id)
/ NULLIF(MAX(COUNT(DISTINCT distinct_id)) OVER (PARTITION BY cohort_week),0), 1) AS pct
FROM activity
WHERE weeks_since BETWEEN 0 AND 8
GROUP BY 1,2
ORDER BY 1,2;
MRR roll-up by plan (snapshot at end-of-week):
-- Assumes a `subscriptions` table mirrored from Stripe with `plan`, `status`, `unit_amount_usd`, `interval`.
SELECT
plan,
SUM(CASE WHEN interval='month' THEN unit_amount_usd
WHEN interval='year' THEN unit_amount_usd / 12.0 END) AS mrr_usd,
COUNT(*) AS active_subs
FROM subscriptions
WHERE status = 'active'
GROUP BY plan
ORDER BY mrr_usd DESC;
When this skill emits a letter to CTO Chief via the refinement loop, every finding becomes severity: critical per the warnings-are-bugs rule. The triage tiers below stay in the review body for prioritization; the letter's severity field is always critical. The kind column maps directly to the letter kind enum below.
| Triage tier | Examples (review body) | Letter kind | Internal action recommendation |
|---|---|---|---|
| CRITICAL | North Star measures activity, not outcome | activity_north_star | BLOCK Gate 3 advancement |
| CRITICAL | KPIs without targets ("good" undefined) | missing_target | BLOCK Gate 3 advancement |
| CRITICAL | Aggregate hides a segment cliff (Simpson's) | simpsons_risk | BLOCK Gate 3 advancement |
| CRITICAL | No recurring review cadence | cadence_missing | BLOCK Gate 3 advancement |
| CRITICAL | Action items without owners or due dates | no_owner | BLOCK Gate 3 advancement |
| HIGH | Cohort retention degrading 3+ weeks | cohort_decay | Fix this review cycle |
| HIGH | MRR flat while signup volume rises | mrr_disconnect | Fix this review cycle |
| HIGH | No recap of prior actions | no_recap | Fix this review cycle |
| MEDIUM | First review writing hypotheses on no baseline | first_review_overreach | Fix next cycle |
| MEDIUM | Single KPI red without segmentation drill | (covered by simpsons_risk when applicable) | Fix next cycle |
| LOW | KPI naming inconsistency; rollup bot not pinned; notebook not versioned | (no letter — backlog) | Backlog |
When emitting a finding via the refinement loop, write the letter with these fields:
finding_id: <sha256(critic+file+line+type)[:12]>
severity: critical # ALWAYS critical (warnings-are-bugs)
confidence: high | medium | low
engine: product-reviewer
kind: missing_target | activity_north_star | no_owner | no_recap | simpsons_risk | cohort_decay | mrr_disconnect | first_review_overreach | cadence_missing
target_file: .ctoc/product-loop/reviews/<date>.md # or plans/canvas/<slug>-kpis.yaml
line: <int> # optional, when applicable
kpi_name: activation_rate # the KPI in question (if any)
segment: free # the segment that triggered (if any)
delta_week: -3pp # week-over-week change (if any)
message: "Activation rate red 3 weeks running; aggregate hides a 29pp free-vs-enterprise gap."
suggested_fix: "Render segment table for activation; dispatch [[experiment-designer]] on the free-tier funnel."
reference: https://uxcam.com/blog/north-star-metric-framework/
The integrator uses confidence and kind to weight findings — a confidence: low single-source finding doesn't block phase advancement on its own, but a kind: activity_north_star or kind: simpsons_risk escalates regardless because both make the entire review untrustworthy.
When invoked as a critic by the Iron Loop integrator (see docs/REFINEMENT_LOOP.md), apply the warnings-are-critical rule:
severity: critical in the letter you write to CTO Chief.warn — there is no soft tier.## Decisions Taken Under Ambiguity section.The principle: a product review that ships green-with-untargeted-KPIs ships with known latent failures — the team is steering blind even if the dashboard is colorful.