| name | falsify |
| description | Popperian falsification method (after Karl Popper) that turns a vague problem, symptom, or mystery into 2-4 competing falsifiable theories, then asks ranked discriminating questions whose answers kill the weakest theories fastest. Never seeks confirmation — states in advance what observation would falsify each theory, prioritizes crucial-experiment questions that kill a theory whichever way they're answered, and treats survival as corroboration, never proof. Works on any domain: production incidents, flaky tests, moving business metrics, security anomalies, interpersonal dynamics, strategy debates. USE WHEN falsify this, falsifiable theory, Popper, competing theories, what would prove me wrong, crucial experiment, discriminating questions, narrow down the cause, diagnose this mystery, test my theory, am I fooling myself. NOT FOR brainstorming solution options to score and pick, designing instrumented experiments with pass/fail metrics, or writing up an already-known root cause. |
Falsify
Named for Karl Popper's core insight: knowledge grows by conjecture and refutation, not by piling up confirmations. This skill takes a fuzzy problem — a bug, a moving metric, a cold manager, a "why does this keep happening" — and forces it through falsificationist discipline instead of confirmation-seeking.
Core Concept
No theory is ever proven true by evidence; it is only corroborated by surviving genuine attempts to prove it false. A theory compatible with every possible observation forbids nothing and therefore explains nothing. The discipline this skill enforces:
- Plurality first. Never analyze one theory. Generate several competing, mutually-exclusive explanations before asking a single question — a lone theory shapes every question you'd think to ask into a confirmation of itself.
- Boldness over vagueness. Prefer theories that forbid more — that stick their neck out and could be cleanly wrong — over cautious theories compatible with almost anything. "It's probably a mix of factors" is a refusal to theorize, not a theory.
- Falsifiability is the price of entry. State the observation that would kill each theory before asking anything. If no conceivable observation could kill it, sharpen it until one could, or drop it.
- Questions exist to kill theories, not confirm them. Rank questions by how many theories a given answer could eliminate. The best question is a crucial experiment: whichever way the user answers, at least one theory dies.
- Survival is not proof. The theory that outlives repeated serious refutation attempts is the best-corroborated conjecture so far — never "true." Reopen it the moment new evidence contradicts it.
How to Run It
Follow the workflow in Workflows/ConjectureAndRefute.md. One workflow handles both the first pass (observation → theories → questions) and every later round (apply answers → kill theories → next questions) — it detects which mode applies from conversation state.
Gotchas
- Never present one theory dressed as "the answer." If only one explanation comes to mind, that's a signal to work harder at generating a genuine rival — not that the problem is solved. Two theories that are the same theory in different words don't count as plural.
- Unfalsifiable theories are the most tempting to write. Anything phrased as "it depends," "sometimes X, sometimes Y," or invoking unmeasurable inner states ("they just don't care," "the codebase is just flaky") fits every observation and forbids nothing. Rewrite until it makes a specific, killable prediction — or discard it.
- A question every theory answers the same way has zero discriminating power. Before asking, check: is there an answer that leaves all surviving theories equally alive? If every plausible answer does, cut the question — don't spend the user's attention on it.
- The user's phrasing usually smuggles in a theory. "Why is my API slow" presumes one cause called "slowness." Restate as a bare observation ("p95 went from 80ms to 900ms starting Tuesday") before generating anything. The smuggled theory then becomes one candidate among rivals — not the frame.
- Users answer with interpretations, not observations. "The deploy was fine" is a verdict; "the deploy ran at 14:02 and the canary passed" is an observation. When an answer arrives pre-interpreted, ask for the raw observation underneath before applying falsification — otherwise you're testing theories against someone else's conclusion.
- Don't rescue a falsified theory with ad hoc patches. "Well, except in this one case..." is exactly the unfalsifiable drift Popper warned about. If it's dead, say so plainly and either promote a survivor or conjecture something genuinely new — not a patched-up zombie.
- Resist closing after one round. One survived question is weak corroboration. Keep attempting refutation until the remaining questions all lose discriminating power — not until the user seems satisfied.
- Prefer cheap refutations. Among questions with equal discriminating power, ask the one the user can answer from memory or a 30-second lookup before the one requiring an hour of log-diving. Popper's asymmetry means one cheap "no" outweighs ten expensive "yes"es.
Examples
Example 1: Production mystery — full round-trip with a kill
User: "checkout conversion dropped 15% last week and nobody knows why — falsify this"
→ Observation: conversion fell ~15% starting ~[date]; no deploy correlation established yet
→ Theories:
T1: A deploy that week broke a checkout step for some segment (predicts: drop starts
at a deploy timestamp, concentrated in affected browsers/flows)
T2: Payment-provider degradation — higher decline rate (predicts: authorization
success rate fell; drop concentrated at the payment step specifically)
T3: Traffic-mix shift — more low-converting traffic, checkout itself is fine
(predicts: per-segment conversion is FLAT; only the blend changed)
→ Crucial question asked first: "Is conversion down within each traffic segment, or
only in the blended total?" — flat-within-segments kills T1 and T2 simultaneously;
down-within-segments kills T3.
→ User: "Down within every segment." → T3 falsified, stated plainly.
→ Next discriminator: "Did the drop begin at a deploy timestamp, or mid-window with no
deploy?" → user checks → began 40h before the only deploy → T1 falsified.
→ T2 is the last survivor and has withstood two refutation attempts → reported as
best-corroborated, with the next cheapest refutation named (pull the authorization
success rate) rather than declared true.
Example 2: Fuzzy interpersonal situation
User: "my manager has been cold in 1:1s for a month and I'm spiraling about it"
→ Observation: 1:1s shortened, fewer questions asked, flatter tone — starting ~4 weeks ago
→ Theories:
T1: Reaction to something specific the user did or decided (predicts: coldness is
directed — warm with others, cold with the user; onset maps to an event)
T2: External pressure on the manager unrelated to the user (predicts: coldness
generalizes — peers see the same shift in their 1:1s)
T3: The manager's own standing changed upstream — reorg, review, scope threat
(predicts: change in what they ASK for, not just tone — more status demands,
more forwarding upward, calendar suddenly full of skip-levels)
→ Crucial question: "Have peers noticed the same shift in their 1:1s?" — 'yes' kills T1;
'no, just me' kills T2. Cheap to answer, kills a theory either way.
→ Note surfaced to the user: T1 is the spiral-feeding theory, and it is also the most
falsifiable — one lunch conversation with a peer tests it. That's the Popperian
antidote to rumination: don't reassure, refute.
Example 3: All theories die — honest restart
User: "my integration test fails ~1 in 10 runs on CI, never locally"
→ Theories: T1 test-order dependence, T2 CI resource contention (timeout-shaped
failures), T3 timezone/clock difference between CI and local
→ Round 1 answers: fails even when run in isolation (T1 dead); failure is an assertion
mismatch, not a timeout (T2 dead); CI pinned to the same TZ and failures aren't
time-of-day clustered (T3 dead)
→ All initial conjectures falsified — the workflow says so outright instead of
stretching one to fit, then generates a NEW set informed by the kills: the
assertion-mismatch detail (learned while killing T2) points at nondeterministic
data — unseeded randomness, map/set iteration order, parallel fixture collision.
→ Failed conjectures weren't waste: each kill narrowed where the truth can live.
