Manages a portfolio of concurrent growth experiments: stage isolation rules, horizon balancing (quick/medium/long), interaction effect detection, cadence design, resource allocation, and portfolio health metrics. A single experiment tells you one thing; a portfolio managed deliberately compounds learning and prevents the most expensive growth mistakes — running experiments that contaminate each other, optimizing one stage at the expense of another, and misreading results because of simultaneous changes. Minimum viable invocation: 3+ experiments planned or active simultaneously. For a single experiment, use /growth-experiment directly.
Manages a portfolio of concurrent growth experiments: stage isolation rules, horizon balancing (quick/medium/long), interaction effect detection, cadence design, resource allocation, and portfolio health metrics. A single experiment tells you one thing; a portfolio managed deliberately compounds learning and prevents the most expensive growth mistakes — running experiments that contaminate each other, optimizing one stage at the expense of another, and misreading results because of simultaneous changes. Minimum viable invocation: 3+ experiments planned or active simultaneously. For a single experiment, use /growth-experiment directly.
triggers
["/experiment-portfolio","team is running 3 or more experiments and results are contradicting each other","growth sprint planning — team needs to sequence 4–8 experiments for the next month","experiment results keep showing No Signal despite multiple tests on the same stage","team is hitting a velocity wall — shipping experiments but not accelerating learning","quarterly experiment roadmap design — need to balance Quick/Medium/Long horizons"]
Active experiments — how many are currently running? On which funnel stages?
NSM and input metrics — are these defined? (portfolio must orient against them)
Team capacity — how many people can actually run and monitor experiments? (determines realistic portfolio size)
Signal windows — what are the measurement windows for current experiments? (determines when slots open up)
Interaction risk — are any active experiments targeting the same users, funnel stage, or feature surface?
Contract
This skill guarantees:
No more than one experiment runs on the same funnel stage at the same time (stage isolation rule)
Every experiment in the portfolio has an explicit horizon (quick / medium / long) and does not extend indefinitely
Interaction effects are assessed before any new experiment starts
Portfolio health is measured by learning rate, not just win rate
Resource allocation is explicit — no experiment is "approved" without an owner and a time budget
Role: Portfolio Manager. A single growth experiment is a data point. A portfolio of experiments, run with discipline, is a compounding learning machine. The difference between a team that learns fast and a team that runs many experiments without learning is portfolio design. Your job is to ensure that every experiment in the queue produces clean signal, that the portfolio is balanced across time horizons, and that nothing runs until it has cleared stage isolation and interaction checks.
The core problem this skill solves
WHAT HAPPENS WITHOUT PORTFOLIO MANAGEMENT:
Problem 1 — Stage contamination:
Two experiments run simultaneously on the same funnel stage.
Experiment A changes the onboarding flow. Experiment B changes the email sequence
for users who haven't activated. Both target the same new user cohort.
Result: Experiment A gets credit for changes that Experiment B caused.
You scale Experiment A's variation. It doesn't hold at scale. You've wasted
engineering time and damaged user experience.
Problem 2 — Short-termism:
The team only runs 7-day experiments because they're quick to read.
Long-cycle improvements (loop strengthening, retention mechanics, ICP expansion)
never get resourced. Growth plateaus at the top of the funnel.
Problem 3 — No compounding:
Wins are scaled, but learnings are not archived in a usable format.
The team repeats the same experiments 18 months later. Learning rate is near zero.
Problem 4 — Win rate theater:
The team optimizes for winning experiments rather than important experiments.
Easy, low-stakes tests have high win rates. Hard, strategic tests get avoided.
Win rate goes up. Business impact stays flat.
Portfolio management prevents all four.
Step 1 — Stage isolation rules
The most important rule in experiment portfolio management. Non-negotiable.
STAGE ISOLATION RULE:
No more than one experiment may run on the same funnel stage at the same time.
No exceptions. If a new experiment needs to run on a stage that already has an
active experiment, it waits until the active experiment closes.
FUNNEL STAGE DEFINITIONS (customize to your product):
Stage 1: Acquisition — from first contact to account creation
Includes: landing pages, ad copy, SEO content, referral invites, HN posts
Stage 2: Activation — from account creation to FVM
Includes: onboarding flow, welcome email sequence, in-product guidance,
first query/action/integration design
Stage 3: Engagement — from FVM to habit
Includes: in-product notifications, engagement emails, feature discovery,
daily/weekly use mechanics
Stage 4: Retention — from habit to long-term retention
Includes: reactivation campaigns, at-risk signals, account health,
expansion triggers
Stage 5: Monetization — from retained to paying
Includes: upgrade prompts, pricing page, trial extension, payment flow
Stage 6: Loop / Referral — from paying/retained to new acquisition input
Includes: share mechanics, invite flows, referral programs, public content creation
INTERACTION EXCEPTION:
Two experiments on different stages may interact if they target the same user
cohort at overlapping time windows.
Example: Acquisition experiment (landing page) AND Activation experiment (onboarding)
running simultaneously. The landing page experiment changes which users enter the
activation experiment. The activation results are confounded by the acquisition change.
Rule: if Experiment A changes who enters Experiment B's funnel stage,
they cannot run simultaneously even if they are on different stages.
TEST: "Could the change made by Experiment A affect the results of Experiment B?"
IF yes → they cannot run at the same time.
Step 2 — Horizon balancing
A portfolio of only 7-day experiments optimizes the top of the funnel. A portfolio of only 90-day experiments produces no signal for months. Balance across all three horizons.
THREE EXPERIMENT HORIZONS:
QUICK (signal in 7–14 days):
Characteristics: high-volume audience, clear metric, fast feedback
Examples: email subject line test, landing page headline, CTA copy, HN post framing
Risk: easy to run; team gravitates toward these; portfolio becomes short-term-only
Target: 40–50% of portfolio slots
MEDIUM (signal in 15–60 days):
Characteristics: requires enough cohort time to observe a meaningful behavior change
Examples: onboarding flow redesign, activation email sequence, feature discovery,
pricing page redesign, loop step improvement
Risk: results arrive infrequently; team loses focus mid-experiment
Target: 35–45% of portfolio slots
LONG (signal in 60–120 days):
Characteristics: retention, loop, or expansion mechanics that require sustained behavior
Examples: growth loop strengthening, community seeding, referral program design,
ICP expansion, PLG→SLG handoff design
Risk: high commitment; easy to cancel mid-flight; results are often ambiguous
Target: 10–20% of portfolio slots
Special rule: long experiments require a named executive sponsor who commits to
the full window before the experiment starts
PORTFOLIO HEALTH CHECK:
IF >70% of experiments are Quick → team is not investing in strategic growth
IF >40% are Long → team lacks fast feedback; decision velocity is too slow
IF no Long experiments in the queue → no investment in compounding improvements
Step 3 — Resource allocation
Every experiment must have explicit resource commitment before being approved. "We'll find time" is not a commitment.
RESOURCE BUDGET PER EXPERIMENT
Before any experiment is approved, define:
Owner: [Single named person responsible for shipping and reading results]
Copy hours: [Realistic estimate for writing, editing, and approving copy]
Design hours: [If any visual or UX change is involved]
Engineering hours: [If any code change, instrumentation, or infrastructure is needed]
Analytics hours: [Time to pull data, analyze, and write the result read]
Calendar time: [Experiment window in days]
Monitoring budget: [Who monitors for anomalies during the window; time/day]
Total person-hours: [Sum]
CAPACITY RULE:
Count total person-hours available for experiments per sprint.
Do not approve experiments that exceed available capacity.
Common mistake: approving 5 experiments that require 60 hours each when the team has
80 hours total. Result: all 5 start, none finish properly, none are read well.
Better: 2 experiments at 60 hours each = 120 hours = fully staffed, well-read results.
OWNER RULE:
Every experiment has exactly one owner.
An experiment with no named owner will not be monitored properly and will not
be read when the window closes. It becomes the most expensive kind of experiment:
one that consumed resources and produced no learning.
PRIORITY QUEUE:
Maintain a prioritized queue of approved experiments (not yet started).
New experiments go into the queue (with RICE score).
Experiments start when (a) a slot opens on the relevant funnel stage AND
(b) the capacity exists to staff it properly.
Nothing skips the queue without a named reason and decision-maker.
Step 4 — Interaction effect assessment
Run this check before starting any experiment, not just when adding to an existing portfolio.
INTERACTION EFFECT CHECKLIST
Before starting Experiment N, answer:
[ ] Which funnel stage does this experiment target?
[ ] Is any other experiment currently active on that stage? (Stage isolation rule)
[ ] Could any active experiment change the user population entering this experiment's stage?
[ ] Does this experiment share an audience segment with any active experiment?
[ ] Does this experiment test an element that another experiment also references?
(e.g., both experiments include a "Try for free" CTA that they each modify differently)
[ ] Would the results of this experiment be interpretable if read simultaneously with
any active experiment's results?
IF any answer indicates a conflict → either delay the new experiment OR close the
conflicting active experiment first.
INSTRUMENTATION CONFLICT:
If two experiments both require changes to the same analytics event or tracking tag,
they cannot run simultaneously. The tracking change contaminates both datasets.
SEASONAL CONFLICT:
If an experiment runs across a seasonal boundary (product launch, industry conference,
holiday period), the window must be designed to avoid the boundary or the window
must be documented as potentially confounded.
Step 5 — Portfolio cadence design
Cadence creates predictability. Without cadence, experiments run indefinitely, results are never acted on, and the queue grows without any experiments closing.
RECOMMENDED CADENCE (for a 2–5 person growth team):
Weekly (30 minutes):
- Monitor active experiments for anomalies (error rates, tracking failures, audience issues)
- Check if any Quick experiments have reached their window close date
- Update the priority queue: any new ideas to RICE-score and add?
Bi-weekly (1 hour):
- Read results of experiments that have closed in the past 2 weeks
(use /experiment-results for each)
- Decide: scale / iterate / kill / rerun?
- Archive learnings (see archive protocol below)
- Start new experiments from the queue into open slots
Monthly (2 hours):
- Portfolio health review (see Step 6 metrics)
- Horizon balance check: is the portfolio too short-term?
- NSM and input metric review: are experiments still oriented at the right metrics?
- Capacity planning for the next 4–6 weeks
Quarterly (half-day):
- Comprehensive learning review: what have we learned that we didn't know 90 days ago?
- NSM revision check (from /north-star-metrics)
- Portfolio reorientation: which input metrics are now the priority?
- Retrospective on win rate, learning rate, and experiment velocity
EXPERIMENT VELOCITY TARGET:
A team of 2–3 growth practitioners should be able to run, read, and archive:
- 4–8 Quick experiments per month
- 2–4 Medium experiments per month
- 1–2 Long experiments per quarter
These are ceilings, not targets. It is better to run 3 experiments well
than 8 experiments sloppily.
Step 6 — Portfolio health metrics
Win rate is the wrong metric. Teams that optimize for win rate run easy experiments. Measure learning rate instead.
PORTFOLIO HEALTH METRICS
Metric 1: EXPERIMENT VELOCITY
Definition: experiments completed (closed with a result) per month
Target: [team-size dependent; 4–8 Quick + 2–4 Medium per month for 2–3 person team]
Warning sign: velocity < half the target = pipeline is blocked; examine queue management
Metric 2: WIN RATE
Definition: % of experiments with verdict = WIN
Healthy range: 20–40%
WARNING: Win rate > 50% means the team is running experiments that are too safe.
Ideas with 50%+ win rate are execution tasks, not experiments.
Good experiments are uncertain — if you were certain, you wouldn't need to test.
WARNING: Win rate < 15% means hypotheses are not informed by data.
Apply more rigor to /growth-experiment idea qualification before running.
Metric 3: LEARNING RATE
Definition: % of experiments that produce an actionable learning regardless of verdict
Target: >80% (Win, No Signal, and Loss experiments should all produce learnings)
Warning sign: team is extracting learnings only from Win experiments
This is the most important metric. Learning compounds. Win rate does not.
Metric 4: TIME-TO-DECISION
Definition: median days from experiment window close to decision (scale / iterate / kill)
Target: < 5 business days
Warning sign: > 10 days = experiments are closed but not acted on;
results are accumulating without producing decisions
Metric 5: ARCHIVE COMPLETENESS
Definition: % of closed experiments with a complete archive entry
(verdict + one-line learning + next action)
Target: 100%
Warning sign: < 80% = learnings are not being captured; the team is running a
random walk instead of a compounding learning process
Metric 6: STAGE COVERAGE
Definition: which funnel stages have had at least one experiment in the last 90 days?
Target: all active stages should have at least one experiment per quarter
Warning sign: a stage with no experiments in 90 days is either saturated (healthy)
or neglected (investigate)
Step 7 — The experiment archive as a compounding asset
The archive is where the portfolio's compounding value accumulates. Without it, each experiment is an isolated event.
ARCHIVE STRUCTURE (required for every closed experiment):
Date: [YYYY-MM-DD]
Experiment: [Name]
Stage: [Funnel stage]
Horizon: [Quick / Medium / Long]
Hypothesis: [One sentence from original spec]
Verdict: [Win / No Signal / Loss / Confounded]
One-line learning: [What you know now that you didn't before — mechanism, not outcome]
Next action: [Scale / Iterate / Kill / Rerun — with specific next step]
Owner: [Name]
ARCHIVE USAGE PROTOCOL:
Before designing any new experiment:
→ Search the archive for experiments on the same funnel stage or same hypothesis
→ Use prior results to inform confidence level and hypothesis framing
→ Do not repeat an experiment that has already produced a clear verdict
without a documented reason (e.g., "audience has changed significantly since")
ARCHIVE REVIEW SCHEDULE:
Monthly: read all archive entries from the past 30 days
Quarterly: read all entries from the past 90 days; identify themes
THEME IDENTIFICATION: what keeps showing up?
- If multiple experiments on the same stage all show No Signal → the constraint
may be upstream, not in the tested mechanics
- If multiple experiments with similar hypotheses show Loss → the hypothesis
may be fundamentally wrong; escalate to a funnel audit or ICP review
- If multiple wins cluster on a specific channel or audience segment → this is
your highest-leverage area; concentrate investment here
HOW THE ARCHIVE COMPOUNDS:
Month 1: 6 experiments → 6 learnings
Month 3: 18 experiments → team knows which hypotheses to prioritize;
confidence levels improve; experiment design becomes more precise
Month 6: 40 experiments → team has a model of what works for their specific
ICP and product; new experiments are informed by prior results;
win rate may increase because hypotheses are better-informed
Month 12: 80+ experiments → strategic advantage; competitors running
random experiments; team runs informed experiments with high prior probability
Output format
## Experiment Portfolio Status
**Date:** [Date]
**NSM:** [Metric]
**Active experiment slots:** [N experiments, by stage]
### Active experiments
| Experiment | Stage | Horizon | Owner | Window closes | Interaction risk |
|---|---|---|---|---|---|
| [Name] | [Stage] | [Q/M/L] | [Name] | [Date] | [None / Flagged] |
### Queue (prioritized by RICE)
| Experiment | Stage | RICE score | Horizon | Blocking stage conflict? |
|---|---|---|---|---|
| [Name] | [Stage] | [Score] | [Q/M/L] | [None / Active experiment: name] |
### Portfolio health (last 30 days)
Experiments completed: [N]
Win rate: [X%]
Learning rate: [X%]
Time-to-decision (median): [N days]
Archive completeness: [X%]
### Horizon balance (current active)
Quick: [N] ([X%]) — target: 40–50%
Medium: [N] ([X%]) — target: 35–45%
Long: [N] ([X%]) — target: 10–20%
### Stage coverage (last 90 days)
[Stage 1]: [Last experiment date]
[Stage 2]: [Last experiment date]
...
### Interference map
[List any active experiments that share a test population, funnel stage, or feature surface]
| Experiment A | Experiment B | Shared population | Interference type | Mitigation |
|---|---|---|---|---|
| [Name] | [Name] | [Segment or stage] | [Stage conflict / User overlap / Feature surface] | [Sequential / Segment split / Redesign B] |
If no interference detected: "No interference detected. All active experiments target distinct funnel stages and user populations."
### Next actions
[Decision on any recently closed experiments]
[Experiments to start in next 2 weeks]
Brain reads / writes
If a companion brain repo is connected:
Before starting:
Read experiments/experiment-log.md — this IS the archive; review before adding new experiments to the portfolio
Read knowledge/icp-map.md — experiments targeting the wrong audience produce No Signal by construction; confirm ICP before adding to queue
Brain write (after each experiment closes):
Append archive entry to experiments/experiment-log.md
If experiment produced a WIN that is being scaled: update playbooks/messaging.md or relevant playbook with the winning variation as the new standard
Brain not connected: maintain the archive in a shared document. The archive is the portfolio's only long-term value; losing it is losing compounding.
Anti-patterns
Anti-pattern
Why it fails
Fix
Running experiments on the same funnel stage simultaneously
Results are confounded; you cannot attribute which experiment caused which change
Stage isolation is non-negotiable; queue experiments that conflict
Win rate as the primary portfolio metric
Teams run easy experiments to maintain win rate; strategic experiments get avoided
Measure learning rate (% of experiments producing actionable learning) as the primary metric
No experiment archive
Team repeats the same experiments; learnings don't compound; institutional knowledge lives only in people's heads
Archive entry is required to close any experiment
Quick experiments only
Top-of-funnel optimization without addressing retention, loop, or expansion; growth plateaus
Maintain 10–20% of portfolio in Long experiments
No named owner per experiment
Experiment is not monitored; window closes; results are never read
Every experiment has exactly one owner; no owner = experiment does not start
Extending experiments indefinitely
Window extension is a form of HARKing — you run until you see what you hoped for
Experiments close on their specified date; results are read at close, not at "when it looks right"
Approving experiments without capacity
Too many active experiments means none are monitored or read properly
Count available person-hours; never approve more than capacity supports
Benchmarks (2025–2026)
Benchmark
Value
Notes
Experiment velocity (2–3 person growth team)
4–8 Quick + 2–4 Medium per month
These are ceilings; quality over quantity
Win rate (healthy range)
20–40%
>50%: experiments are too safe; <15%: hypotheses are uninformed
Learning rate (target)
>80% of closed experiments
Win, No Signal, and Loss all produce learnings
Time-to-decision after experiment close
<5 business days
>10 days: results are accumulating without action
Archive completeness (target)
100%
<80%: learnings are not compounding
Maximum experiments on same funnel stage simultaneously
1
Zero exceptions
Long experiment horizon minimum
≥1 per quarter
No Long experiments in 90 days = no compounding investment
Portfolio size limit (2–3 person team)
4–6 active experiments
More = under-resourced; results will be poor
Related skills
Skill
When to use
growth/growth-experiment/SKILL.md
Designs individual experiments that go into the portfolio
growth/experiment-results/SKILL.md
Reads results for each experiment that closes; produces archive entries
growth/north-star-metrics/SKILL.md
Portfolio must orient against the NSM and input metric tree
growth/funnel-audit/SKILL.md
If multiple experiments on the same stage keep producing No Signal, a funnel audit may reveal an upstream constraint
growth/growth-loop-design/SKILL.md
Long-horizon experiments often target loop strengthening; portfolio must reserve slots for them
Validation criteria
Stage isolation rule applied: no two active experiments on the same funnel stage
Interaction check completed for all active experiments before any new experiment starts
Every experiment in portfolio has a named owner and explicit resource budget
Horizon balance checked: at least one Long experiment in active or queued state
Portfolio health metrics defined and tracked (win rate, learning rate, time-to-decision)
Archive entry format established; all closed experiments have complete entries
