incremental-belief-updating

# Incremental Belief Updating **What**: A forecasting and reasoning technique where you continuously update your probability estimates by small increments as new information arrives, rather than sticking to initial beliefs or making large sudden jumps. **When to use**: When making predictions or estimates in uncertain domains where new evidence arrives over time—project timelines, market forecasts, risk assessments, or any judgment requiring ongoing calibration. **Introduced by**: Philip Tetlock and colleagues through the Good Judgment Project (2011-2015), studying superforecasters ## Core Mechanism **Bayesian-inspired updating**: Each piece of new evidence should shift your probability estimate proportionally to its strength and relevance. Small evidence → small update. Large evidence → large update. No evidence → no update. **Key contrast**: - **Anchoring**: Stick to initial estimate despite new data - **Overcorrection**: Swing wildly with each new headline - **Incremental updating**: Adjust systematically in proportion to evidence weight **Why it works**: Reality reveals itself gradually. Superforecasters who update frequently (but not dramatically) calibrate toward accuracy faster than those who anchor or overreact. ## When to Apply **Use incremental belief updating when:** - Making predictions in domains with evolving information - Estimating project timelines, budgets, or resource needs - Assessing risks or probabilities over time - Evaluating market conditions, competitive moves, or strategic options - Tracking leading indicators for future outcomes **High-value contexts:** - Forecasting tournaments or prediction markets - Roadmap planning and deadline estimation - Risk management and threat assessment - Investment decisions with ongoing information flow - A/B test analysis as data accumulates ## Execution Steps ### 1. Set Initial Baseline Estimate Start with a base rate or reference class forecast. What's the historical frequency of similar outcomes? This is your prior. ### 2. Express Uncertainty as Probability Instead of binary (will/won't), use probabilities: "60% chance of shipping by Q2." Enables precise updates. ### 3. Identify Information Triggers What evidence would update your belief? Define upfront: "If X happens, I'll adjust by ~10% toward Y." ### 4. Update Small and Often When new evidence arrives, adjust your estimate proportionally. Tetlock's superforecasters updated every few days, not just after major news. ### 5. Track Direction and Magnitude Document: What changed? How strong is the evidence? By how much should I update? Did I update too much or too little? ### 6. Avoid Round Numbers Don't snap to 50%, 75%, 90%. Precise estimates (63%, 78%) reflect granular thinking and force you to process evidence carefully. ### 7. Review Calibration Over Time After outcomes resolve, compare your probability trajectory to reality. Were you too slow to update? Too fast? This trains judgment. ## Real-World Applications **Good Judgment Project**: Superforecasters who updated their predictions frequently (average 5+ times per question) and incrementally outperformed intelligence analysts with classified data. **Project Delivery Estimates**: Instead of "We'll ship in 6 weeks" (then missing deadline), track probabilities: Start at 40% on-time → Sprint 1 blockers emerge: 30% → Workaround found: 45% → Scope reduced: 65%. **Market Analysis**: Rather than "The market will crash" or "Bull run forever," track: "Recession probability: 25% → Fed minutes hawkish: 32% → Unemployment data weak: 28% → Credit spreads widen: 35%." **Incident Severity Assessment**: Initial alert: 60% chance this is customer-impacting → More users reporting issues: 80% → Root cause identified in non-critical service: 40% → Workaround deployed: 15%. ## Key Indicators **Signs you're doing it well:** - Probability estimates drift gradually, not jump wildly - You can articulate what evidence would change your mind - Updates correlate with information quality, not just recency - Calibration improves over time (Brier scores decrease) **Red flags:** - Estimates haven't moved in weeks despite new information - Large swings based on headlines without substance analysis - Can't explain why you're at 70% vs. 65% - Round numbers (10%, 50%, 90%) dominate estimates ## Common Mistakes **Under-updating (anchoring)**: Sticking too close to initial estimate despite contradictory evidence. Tetlock found this more common than over-updating. **Over-updating (recency bias)**: Swinging dramatically with latest news without weighting historical base rates. **Binary thinking**: "It will happen" or "It won't" leaves no room for calibration. Use probabilities. **Ignoring base rates**: Starting from intuition rather than reference class frequencies leads to miscalibration. ## Related Frameworks **Complementary**: Bayesian Reasoning (formal mathematical foundation), Superforecasting (broader methodology), Reference Class Forecasting (establishing priors) **Contrasting**: Gut Feel (no systematic updating), Prediction Markets (crowd-based), Scenario Planning (qualitative branches) **Sequential**: Establish base rate → Set initial probability → Monitor information sources → Update proportionally → Track calibration → Adjust update strategy ## Practical Examples **Feature Launch Timeline**: Week 1: "70% chance we launch by end of quarter" (based on past launches) Week 3: Backend integration more complex than expected → 55% Week 5: Frontend team ahead of schedule → 60% Week 7: QA finds critical bug → 40% Week 9: Bug fixed, testing clean → 70% Result: Launched 2 days into next quarter. Probabilities tracked reality better than binary prediction. **Hiring Candidate Assessment**: After resume: 40% chance of hire (base rate for this role) After phone screen: 60% (clear communication, relevant experience) After technical interview: 50% (strong on algorithms, weak on system design) After team fit interview: 70% (excellent culture match, team excited) After reference checks: 65% (one lukewarm reference, others strong) Decision: Make offer (hired, worked out well). **Security Threat Level**: Start: 5% chance of attack this month (historical rate) Unusual traffic pattern detected: 8% Pattern matches known bot signature: 12% Geo-source is known threat actor region: 25% Rate limiter successfully blocking: 10% Final assessment: Elevated but contained risk, maintain monitoring. ## Measurement **Quantitative signals:** - Brier Score (measures calibration, lower is better) - Update frequency (superforecasters average 5-10 updates per question) - Update magnitude distribution (should be mostly small, few large) - Correlation between evidence strength and update size **Qualitative indicators:** - Can articulate reasoning for each update - Written logs of what triggered updates - Diversity of information sources considered - Willingness to update in both directions (up and down) ## Scoring Criteria **Practitioner Weight**: 10/10 — Tetlock's research directly studied thousands of real forecasters making real predictions with measurable accuracy; superforecasters validated the approach **Clarity & Executability**: 8/10 — Conceptually clear, but requires discipline and judgment about update magnitudes; easier with practice **Proven ROI**: 9/10 — Good Judgment Project demonstrated dramatically improved forecasting accuracy; widely adopted in intelligence community, forecasting platforms **Novelty**: 8/10 — Validates Bayesian reasoning in practical form; counterintuitive that frequent small updates beat "wait for clarity" approach **Cross-Domain Applicability**: 9/10 — Applies to any domain with uncertainty and evolving information: business, engineering, investing, research, personal decisions **Total Score**: 44/50 (Tier 1: Canonical)