cognitive-biases-and-dual-process-reasoning

name	cognitive-biases-and-dual-process-reasoning
description	Audits an analytic judgment or forecast for fast, intuitive System 1 biases — motivated skepticism, asymmetric scrutiny, the affect heuristic — before it is finalized.
kind	skill
status	ready
provenance	{"principles":["P009","P038","P051","P053","P064","P065","P066","P067","P082"],"claims":["C00597","C00627","C00707","C00708","C00962","C00963","C00964","C01082","C01425","C01426","C01614","C01615"],"evidence":[],"source_anchors":[]}

Cognitive Biases And Dual Process Reasoning

Purpose

Audits an analytic judgment or forecast for the fast, intuitive (System 1) failure modes that corrupt reasoning before slower, structured correction has a chance to run — a preferred conclusion defended by discrediting dissonant evidence while a confirming result sails through unquestioned, an emotional stance quietly deciding which argument feels compelling, and a validated formula overridden on ordinary case knowledge instead of a rare individuating fact. It also checks whether the judgment was built by actively searching for missing evidence and by mixing theory with data, rather than letting one vivid case harden into a general rule, and whether any reported performance is honest about the gap between raw, unadjusted accuracy and whatever was adjusted after the fact. This skill audits the reasoning's psychological honesty — including whether the reviewed party keeps open the standing possibility that they are mistaken — not the substance of the conclusion itself.

When to use

• A judgment's written case needs auditing for whether disconfirming evidence received the same scrutiny as confirming evidence, or whether an inconvenient finding was quietly neutralized by questioning its source, method, or motives rather than its content.
• The case for a conclusion reads as heavily one-directional, and the actual balance of supporting-versus-opposing considerations needs to be counted rather than assumed.
• A prediction or estimate rests on information that was available but never actively sought, and it is unclear whether that gap was noticed before the judgment was finalized.
• The reviewed party holds a visibly strong like or dislike toward the subject, and the argument they find most persuasive lines up suspiciously well with what they would prefer to be true.
• An analyst or decision-maker wants to set aside a validated predictive formula for their own case read, and it needs checking whether they hold a rare, individuating fact or just ordinary case knowledge the formula already accounts for.
• Deciding whether a one-case lesson actually generalizes, or whether the reasoning leans entirely on preconception or entirely on raw data with nothing to check either against the other.
• A performance claim mixes raw, unadjusted results with subjectively adjusted ones, and it is unclear how much of the reported accuracy is genuine ex ante skill versus after-the-fact credit.
• A conclusion is delivered with total conviction and no acknowledgment, however small, that it could be wrong.

Procedure

1. Check whether disconfirming evidence was interrogated as hard as confirming evidence. Watch for a double standard: the same evidence quality earns credibility when it fits the preferred conclusion and gets neutralized — by attacking motives, authenticity, or interpretation — roughly four times more often when it does not, with an assurance of rigor barely closing the gap (P009, P051).
2. Count, or estimate, the ratio of supporting to opposing considerations raised in the written case. The average expert favors the preferred outcome by roughly three to one; a near one-directional ratio signals lopsided thinking rather than a genuinely weighed judgment (P053).
3. Check whether a vital prediction or estimate rests on information that was available and significant but never actively sought. Treat that search failure as a reasoning flaw in its own right, not a neutral gap, since sound judgment requires going to look for evidence rather than only weighing what happened to arrive (P038).
4. Check whether an emotional stance toward the subject is doing the arguing. Under the affect heuristic, likes and dislikes decide which arguments feel compelling, and this pull dominates most strongly exactly where the emotional stake is highest (P064).
5. If a validated formula was overridden by human judgment, examine the stated reason. A genuine broken-leg exception is a rare, decisive, individuating fact outside the formula's inputs — not ordinary case knowledge dressed up as special insight, which is what people wrongly believe justifies an override (P065).
6. Check whether the reasoning mixes theory-driven and data-driven thinking rather than leaning on only one, and whether a single case has been allowed to harden into a stable lesson or philosophy of history. A trait or lesson that reverses depending on whose interests are served should not be generalized from one instance (P066).
7. Where performance or accuracy is reported, confirm the raw, unadjusted score is stated alongside any requested adjustment, and treat a wide gap between the two with suspicion — unadjusted ex ante accuracy is a different claim from an ex post adjustment that only measures how well the gap was closed afterward (P067).
8. Check that the judgment under review — and this review itself — keeps open the declaratory stance that the conclusion could be mistaken, a standing check on conviction (P082).
9. Emit findings highest-risk-to-the-judgment first, each in the name-the-flaw / correction / residual-uncertainty / next-step format.

Inputs

• The analytic judgment or forecast under review, together with the written case made for it.
• The evidence treated as confirming the conclusion and the evidence treated as disconfirming it, so the two can be compared for equal scrutiny.
• Any information that was available and significant to the question but not actively sought.
• Signals of the reviewed party's emotional stance toward the subject, where the affect heuristic may be in play.
• Where a validated predictive formula exists: its output, any human override, and the stated reason for the override.
• The historical case or precedent, if any, that the judgment generalizes from.
• Any reported performance record, with both raw and subjectively adjusted figures if both exist.
• The confidence or hedging language used in the final write-up.

Output

Per finding: name the cognitive-bias or dual-process flaw (motivated skepticism, asymmetric scrutiny, an unchecked affect heuristic, an unjustified formula override, an over-generalized one-case lesson, a raw-versus-adjusted scoring gap, or a missing declaratory hedge), apply the principle(s) by ID, name the corrective (equalize the scrutiny applied to both sides, recount the pro-versus-con ratio, go and seek the missing evidence, separate the emotional stance from the argument, restrict formula overrides to a genuine broken-leg fact, report raw alongside adjusted scores, restate the declaratory stance), state the residual uncertainty the correction leaves, and end with a concrete next step. Order findings highest-risk-to-the-judgment first. Never substitute this review's own substantive judgment, forecast, or estimate for the one under review.

Anti-patterns to flag

• A confirming result waved through without question while a disconfirming result of equal quality is picked apart on method, motive, or authenticity — the same double standard dressed up as rigor (P009, P051).
• A written case that runs heavily one-directional, offering barely a counter-consideration, with no check on how lopsided the pro-versus-con ratio has actually become (P053).
• A prediction resting on information that was sitting available and significant, but nobody went and looked for it, waved through as a neutral gap rather than a reasoning failure (P038).
• An argument that tracks the reviewed party's likes and dislikes more closely than the evidence — the most emotionally comfortable conclusion is also the one argued most forcefully (P064).
• A validated formula set aside for "extra case knowledge" that is really just more of the same kind of information the formula already used, rather than a rare, decisive individuating fact (P065).
• A single case elevated into a stable philosophy of history or a general lesson, or a judgment built on preconception alone or raw data alone with nothing to check either against the other (P066).
• A performance claim that reports only the subjectively adjusted score, or shows a wide gap between raw and adjusted figures with no explanation, blurring genuine ex ante accuracy with ex post credit (P067).
• A conclusion delivered with total conviction and no room, even nominally, for the possibility that it is wrong (P082).

References

See ../../references/analytic-method-principles-index.md for the full principle catalogue. For adjacent concerns, see the sibling skills: competing-hypotheses-and-diagnostic-evidence owns the structured technique — weighing evidence by diagnosticity and driving toward refutation — that corrects the asymmetric-scrutiny and motivated-skepticism bias this skill catches; mindsets-schemata-and-perception owns the wider mental-model and schema catalogue this skill's fast pattern-matching runs inside; probabilistic-judgment-and-calibration owns the coherence and calibration mechanics of a probability estimate itself, while this skill catches the psychological source of a distortion — the affect heuristic, a raw-versus-adjusted gap — before it reaches the numbers.

Provenance

Derived solely from P009, P038, P051, P053, P064, P065, P066, P067, and P082 (Expert Political Judgment: How Good Is It? How Can We Know?; Thinking, Fast and Slow; Superforecasting; Perception and Misperception in International Politics — all distillation-only; see the frontmatter above for the full claim, evidence, and source-anchor list).