Think Clearly A path through The Default Library ======================================== The tools for not being fooled: by others, by numbers, by machines, or by yourself. Step 1. The Biases That Fool Everyone ------------------------------------- Why: The biases that fool everyone, including you. Your brain runs on mental shortcuts — fast, efficient, and usually good enough. But those same shortcuts misfire in predictable ways, and the misfires are invisible from the inside. Here are the big ones, so you can catch yourself. We like to think we reason our way to conclusions. Often it's the reverse: we arrive at a conclusion by gut, then hire our reason to defend it. Psychologists Daniel Kahneman and Amos Tversky showed that these errors aren't random — they're systematic, shared by nearly everyone, and they persist even when you know about them. You can't delete them. But you can learn their shapes and build habits that catch them. A bias you know about still fools you. The goal isn't a perfect mind — it's a suspicious one. Biases about beliefs & being right Confirmation biasWe seek, notice, and remember evidence that supports what we already believe — and skate past what contradicts it. The king of biases; it quietly feeds most of the others.googling to "confirm" a hunch, following only people you agree with, remembering the hits and forgetting the misses. Motivated reasoningWe reason harder to reach conclusions we want to be true, and demand far more proof for ones we don't. Our thinking bends toward our wishes.scrutinizing a study that challenges your diet while accepting a flattering one uncritically. The Dunning–Kruger effectIn Kruger and Dunning's 1999 studies, the least skilled overrated their performance the most, while top performers slightly underrated theirs. The idea: the less skilled we are, the less able we are to see our mistakes. Critics argue much of the pattern is a statistical artefact (regression to the mean plus everyone's habit of rating themselves above average), so it's a debated finding, not a law.supreme confidence after reading one article; humble caution after years of study. Hindsight biasOnce we know how something turned out, it feels like we "knew it all along" — erasing how uncertain it really was beforehand."the crash was obvious" (it wasn't, to almost anyone, at the time). Biases about numbers & judgment AnchoringThe first number we see sets a mental reference point that drags all later estimates toward it — even when it's arbitrary or irrelevant.a "was $200, now $120" price making $120 feel cheap; the first salary figure shaping the whole negotiation. The availability heuristicWe judge how likely something is by how easily examples come to mind — so vivid, recent, or scary events feel far more probable than they are.fearing plane crashes (rare, dramatic) more than car crashes (common, mundane) after seeing one in the news. Survivorship biasWe study the winners that are visible and ignore the failures that vanished — skewing our sense of what works."college dropouts get rich" — you see the few billionaires, not the millions of dropouts who didn't. Base-rate neglectWe focus on vivid specifics and ignore the underlying statistics (the base rate), leading to badly miscalibrated conclusions.a "90% accurate" test for a rare disease flags you — and you panic, forgetting that with a rare disease most positives are false alarms. Biases about loss, cost & risk Loss aversionLosses hurt roughly twice as much as equivalent gains feel good. So we cling to what we have and avoid risks even when the odds are in our favor.refusing a coin-flip to win $150 or lose $100, though it's a good bet. The sunk cost fallacyWe keep pouring in time, money, or effort to justify what we've already spent — even when quitting is clearly the better move. The past money is gone either way.finishing a bad movie "because we paid," staying in a dead-end project because you're "in too deep." Framing effectThe same fact feels different depending on how it's worded. Identical information, opposite reactions."90% survival rate" feels great; "10% mortality rate" feels grim — same number. Biases about other people Fundamental attribution errorWe explain others' behavior by their character ("he's rude") but our own by circumstances ("I was having a hard day"). We judge others by actions, ourselves by intentions.the driver who cuts you off is a jerk; when you do it, you were in a rush. In-group bias & the halo effectWe favor "our" group and assume good things cluster — someone attractive or likable is assumed smart, kind, and competent too, on no evidence.trusting a confident, good-looking spokesperson; distrusting an out-group by default. Negativity biasBad impressions and bad news grab and hold our attention far more than good — an ancient survival setting that skews our view of the world toward the threatening.one harsh comment outweighing ten kind ones; the news feeling like the world is ending. So what do you do about it? You can't will these away — they operate below awareness. But a few habits genuinely help: Argue the other side. Actively look for evidence you're wrong (the antidote to confirmation bias). Ask "what would change my mind?" If nothing could, you're not reasoning — you're defending. Slow down on big calls. Biases run on the fast, automatic system. Deliberately slowing down and writing out your reasoning brings the slower, checking system online. Use outside views and checklists. Ask what usually happens in cases like this (base rates), get an outsider's read, and use pre-committed criteria for big decisions so in-the-moment bias has less room. (See How to Make a Hard Decision.) Assume you're biased — especially when certain. The feeling of obvious rightness is exactly when bias is strongest. Humility isn't just nice; it's accurate. The takeaway Cognitive biases are systematic, universal glitches in mental shortcuts — confirmation bias, anchoring, availability, sunk cost, loss aversion, the fundamental attribution error, and more. Knowing about one doesn't switch it off. What helps: actively arguing the other side, slowing down on important decisions, leaning on base rates and outside views, and treating your own certainty as a warning light rather than proof. Try this: pick one belief you feel very sure about and spend five minutes genuinely building the case against it. Notice how hard your mind resists — that resistance is confirmation bias, caught in the act. Sources & Further Reading Sources. These biases are documented across decades of cognitive and social psychology, much of it originating with Daniel Kahneman & Amos Tversky. Dunning–Kruger: Kruger & Dunning, Journal of Personality and Social Psychology, 1999; the statistical critique, e.g. Nuhfer et al., Numeracy, 2016, and Gignac & Zajenkowski, Intelligence, 2020. Note that a few classic effects have faced replication debate — treat them as strong, well-supported tendencies, not iron laws. 📘 Thinking, Fast and Slow — Daniel Kahneman (the foundational, comprehensive account) 📘 The Art of Thinking Clearly — Rolf Dobelli (one bias per short chapter) 🌐 Wikipedia — List of cognitive biases (and the "Cognitive Bias Codex" infographic) 📘 The Scout Mindset — Julia Galef (how to want the truth more than being right) Step 2. Why Exponential Growth Rules Everything ----------------------------------------------- Why: Exponential growth rules pandemics, debt and technology; humans don't feel it. Our minds think in straight lines: steady, predictable, one-step-at-a-time. But the most important forces in the world — viruses, populations, technology, money, compound anything — don't grow in lines. They grow in curves that fool us every single time. There are two fundamental ways a thing can grow. Linear growth adds the same amount each step: +10, +10, +10. Exponential growth multiplies by the same factor each step: ×2, ×2, ×2. That difference sounds small. It is the difference between the ordinary and the unimaginable. The reason exponential growth fools us is its shape: for a long time it looks flat, unremarkable, even slower than linear — and then, seemingly out of nowhere, it erupts past everything. See it for yourself. Interactive · Linear vs. Exponential Step number10 Linear (start 10, +10 each step)Exponential (start 10, ×2 each step) 100Linear value 5,120Exponential value Exponential growth is quiet, quiet, quiet — and then it's everything, all at once. The story that makes it click An old legend: a king offers a reward to the inventor of chess. The inventor asks for something humble — one grain of rice on the first square of the board, two on the second, four on the third, doubling each square. The king laughs at such a modest request and agrees. He shouldn't have. Interactive · Fold a piece of paper A sheet of paper is about 0.1 mm thick. Each fold doubles its thickness. How tall does it get? Number of folds10 You can't actually fold paper this many times — but the math is the point. Doubling starts absurdly small and ends beyond the solar system. By the 42nd fold, the paper would reach the Moon. By the 103rd, it would span the observable universe. That is the violence of doubling — and it's exactly why the chessboard bankrupts the king: the full board needs more rice than has ever been grown in human history. Why this is one of the most useful ideas you can own Exponential (and its mirror twin, exponential decay) isn't a curiosity — it's the engine behind an astonishing range of real things: Pandemics. Each infected person infects a few others, who each infect a few more. Early on it looks like "just a handful of cases," which is precisely why exponential threats are so dangerous — by the time they *look* big, they're already enormous. The flat-looking early part is the window that gets missed. (This is why "flattening the curve" is a mathematical, not just medical, idea.) Money. Compound interest is exponential growth applied to your savings (see Compound Interest) — and to your debt. The same curve builds fortunes slowly and then suddenly, or buries people in credit-card interest. Technology. Computing power has doubled roughly every couple of years for decades (Moore's Law). Exponential improvement is why the phone in your pocket dwarfs the computers that sent people to the Moon. Anything that compounds. Skills, relationships, a reputation, a small daily habit — things that build on themselves grow exponentially. This is why tiny consistent efforts feel pointless for ages and then, past the bend in the curve, become extraordinary. The lesson of the flat early stretch is: don't quit before the curve turns. The mind-bug worth fixing Because we're wired for linear intuition, we systematically underestimate exponential processes — in both directions. We panic too late about fast-spreading dangers, and we give up too early on slow-compounding rewards. Simply knowing that the world runs on curves, not lines, is a genuine cognitive upgrade. When something multiplies rather than adds, respect the curve. The takeaway Linear growth adds; exponential growth multiplies — and multiplying stays deceptively flat before erupting past everything. This one curve governs pandemics, populations, compound money, technology, and any habit or skill that builds on itself. Human intuition is linear, so we underestimate exponentials both ways: reacting too late to fast threats and quitting too early on slow-compounding gains. Try this: drag the step slider to 20 and notice how far apart the two lines have become — then imagine that gap is cases of a disease, or dollars of interest, or days of a small habit. Respecting the curve is the whole lesson. Sources & Further Reading Sources. Paper-folding-to-the-Moon and the chessboard-rice legend are standard illustrations of exponential growth (0.1 mm × 2⁴² ≈ 440,000 km, roughly the Earth–Moon distance). Moore's Law is the observed ~2-year doubling of transistor counts. Educational overview. 📺 3Blue1Brown — "Exponential growth and epidemics" 📘 The Joy of x — Steven Strogatz (exponentials and much more, delightfully) 📺 Search "Albert Bartlett exponential function" — the famous "greatest shortcoming of the human race" lecture 🌐 Companion piece: Compound Interest Will Blow Your Mind (in this library) Step 3. Probability: How to Not Fool Yourself --------------------------------------------- Why: Probability: the maths of not fooling yourself. The human brain is spectacularly bad at chance. We see patterns in randomness, misjudge risk, and get famous puzzles confidently wrong. Probability is the math that overrules our broken intuition — and the interactive below will prove your gut wrong in real time. Probability is just a number between 0 (impossible) and 1 (certain) describing how likely something is. Simple to define — and yet it's the branch of math that most reliably humiliates our intuition. Nowhere is that clearer than a puzzle that fooled thousands of mathematicians. Play it yourself. The Monty Hall problem — play it Three doors. Behind one is a car 🚗; behind the other two, goats 🐐. Pick a door. The host — who knows what's behind each — then opens a different door revealing a goat, and offers you the choice: stay with your door, or switch to the other closed one. Does switching help? Trust your gut, then test it. Interactive · Play Monty Hall Door 1🚪 Door 2🚪 Door 3🚪 Pick a door to begin. Stay Switch Play again –Win % when you STAY –Win % when you SWITCH ⚡ Auto-simulate 1,000 games each way Your own plays count too. Or hit simulate to fast-forward the law of averages. Switching wins two out of three times. Almost everyone's gut says it's 50/50. The gut is wrong. Why switching wins (the "aha") Here's the key: your first pick has a 1/3 chance of being the car. That means there's a 2/3 chance the car is behind one of the other two doors. The host then does you a huge favor — he opens the goat door among those two, effectively concentrating that entire 2/3 chance onto the single remaining door. Staying keeps your original 1/3. Switching claims the 2/3. Still feels wrong? Imagine 100 doors. You pick one (1% chance you're right). The host opens 98 other doors, all goats, leaving your door and one other. Would you switch now? Obviously — that other door is almost certainly hiding the car. Monty Hall is that exact situation, just shrunk to three. The birthday paradox — another gut-punch How many people need to be in a room before it's more likely than not that two of them share a birthday? Most people guess 180-ish (half of 365). The real answer is shockingly small. Drag to explore: Interactive · Shared Birthdays People in the room23 50.7% chance that at least two people share a birthday Just 23 people gives better-than-even odds. At 50 people it's 97%; at 70, it's 99.9%. Why so few? Because you're not looking for someone matching your birthday — you're counting every possible pair. With 23 people there are 253 pairs, and each is a fresh chance to match. Our intuition tracks "matches with me," but the math tracks all the pairs. The deeper lessons about chance These puzzles aren't just party tricks. They expose the exact ways our minds misread randomness — the same errors that cost people money, health, and clear judgment: We ignore how the evidence was produced. The Monty Hall answer depends on the setup (the host knows where the car is and always opens a goat door), which our gut skips. A close cousin is ignoring the base rate, as with medical tests, gambling, and risk — the underlying rates matter more than the vivid moment. (See The Biases That Fool Everyone.) We see patterns in noise. Flip a fair coin and "HHHH" feels due to break — the gambler's fallacy. But the coin has no memory; each flip stays 50/50. Random streaks look meaningful to us, but randomness is streakier than we expect. Rare things happen — because there are so many chances. Any specific coincidence is unlikely, but with billions of people and events, some astonishing coincidence is nearly guaranteed. "What are the odds?!" is usually higher than it feels, because we notice the hit and forget the countless misses. The takeaway Probability is the math that overrules our unreliable intuition about chance. Switching doors in Monty Hall wins 2/3 of the time (your first pick was probably wrong, and the host concentrates the odds onto the other door). Just 23 people make a shared birthday likelier than not, because you're counting all the pairs. The recurring lesson: respect base rates, don't read meaning into random streaks, and remember that rare events are common when there are enough chances. Try this: hit "auto-simulate" above and watch the switch rate climb toward 67% while stay sits near 33%. Then next time your gut is sure about odds — a "hot streak," a "sure thing," a scary-sounding risk — pause and ask what the actual numbers say. Sources & Further Reading Sources. The Monty Hall result (2/3 for switching) is mathematically proven and confirmed by the simulation above; it famously stumped many professional mathematicians when Marilyn vos Savant published it in 1990. The birthday figure (50.7% at 23) comes from standard probability (1 − 365!/(365ⁿ·(365−n)!)). 📺 Numberphile — "Monty Hall Problem" 📘 The Drunkard's Walk — Leonard Mlodinow (how randomness rules our lives) 📘 Innumeracy — John Allen Paulos (the cost of not grasping probability) 📺 Search "birthday paradox explained" — the all-the-pairs insight, visualized Step 4. How to Make a Hard Decision ----------------------------------- Why: A method for hard decisions. Big decisions feel paralyzing because we treat them as one giant knot. Break the knot into steps, apply a few sharp tools, and even life's hardest calls become navigable — not painless, but clear. Hard decisions are hard for real reasons: uncertainty (you can't know how it'll turn out), stakes (it matters), and trade-offs (every option loses something). You can't remove those. But most decision paralysis comes from tangling them together and letting emotion and bias run the show. A process untangles them. A process for the big ones Six steps 1 Define the real decision Name precisely what you're deciding — and question the framing. "Should I take this job?" might really be "what do I want my next few years to look like?" Many bad decisions are sharp answers to the wrong question. 2 Get the information that would change your mind Gather what actually matters — but only what would alter your choice. Endless research is often procrastination in disguise. Ask: "what do I need to know to decide well?" then stop. 3 Generate more options than two "This or that" is usually a false dilemma. Force a third and fourth option — combine, delay, negotiate, do a smaller test first. The best choice is often one you hadn't listed. 4 Clarify what you actually value Weigh options against your real priorities, not what you think you should want or what impresses others. What matters most here — money, freedom, security, growth, relationships? Name it before you compare. 5 Decide — and set a deadline At some point, more analysis stops helping and starts hurting. Give yourself a decision date. A good-enough choice made and acted on usually beats a perfect one made too late. 6 Commit, then review Once decided, commit fully — second-guessing drains the energy execution needs. Later, review honestly: was it a good decision given what you knew, separate from how the luck broke? Sharp tools for step 5 When you're stuck at the actual choice, these cut through: 🚪 One-way vs. two-way doorsIs this reversible? Two-way doors (you can undo it) deserve speed — just choose and adjust. One-way doors (hard to reverse) deserve real care. Most decisions are two-way doors we agonize over as if they were one-way. ⏱️ 10 / 10 / 10How will I feel about this in 10 minutes? 10 months? 10 years? It rescues you from giving a fleeting emotion veto power over a long-term choice — and sometimes reveals the opposite. 😌 Regret minimizationPicture yourself old, looking back. Which choice will you regret not having made? Great for big, values-driven crossroads — it surfaces what you'll actually care about in the end. 📊 Weighted criteriaList what matters, give each a weight, and score each option against them. Turns a vague "I dunno" into something visible — and the moment you resist the "winning" answer tells you your real preference. 💀 The pre-mortemImagine it's a year later and the choice failed badly. Why? Listing the failure modes before deciding surfaces risks you can plan around — or that change your mind. 🪙 The coin-flip checkAssign the options to a coin, flip it — and notice your gut reaction to the result. Relief or disappointment reveals what you were quietly hoping for. (Don't obey the coin; listen to your reaction to it.) Match your effort to the door. Reversible? Decide fast. Permanent? Slow down and use the tools. The traps that wreck decisions Even with a process, watch for these (see The Biases That Fool Everyone): Sunk cost. Don't choose based on time or money already spent — that's gone whatever you pick. Decide from where you are now, looking forward. Analysis paralysis. Past a point, more options and more research reduce satisfaction and stall you. Set criteria, then choose. Perfectionism is often fear wearing a productive costume. Deciding while flooded. Strong emotion narrows thinking. For big calls, sleep on it — literally. Your calmer, rested brain sees options the panicked one couldn't. (See Why You Feel Before You Think.) Outsourcing to others' expectations. Advice is useful; living someone else's answer is not. Weigh input, then decide by your own values — you're the one who lives with it. The liberating truth Most decisions are not as permanent or as consequential as they feel in the moment, and there is rarely one perfect answer waiting to be found. Often several options would work out fine, and much of a choice's success comes from committing to it and making it work — not from having divined the one right path. Decide well with what you know, commit, adapt as you learn. That's not a consolation prize; it's how good lives are actually built. The takeaway Untangle a hard decision with a process: define the real question, gather only what would change your mind, generate more than two options, clarify your true values, decide by a deadline, then commit and review. Use sharp tools — reversible-or-not, 10/10/10, regret minimization, weighted criteria, pre-mortem — and match your effort to how reversible the choice is. Avoid sunk cost, analysis paralysis, and deciding while emotionally flooded. Try this: take a decision you're stuck on and ask two questions — "Is this a one-way or two-way door?" and "Which option will I regret not taking in 10 years?" Those two often dissolve the stuckness on their own. Sources & Further Reading Sources. "One-way/two-way doors" is Jeff Bezos's framing; "10/10/10" is Suzy Welch's; "regret minimization" is also Bezos; the pre-mortem is from psychologist Gary Klein. The decision-vs-outcome distinction is central to Annie Duke's work. General decision-science, presented as practical tools. 📘 Thinking in Bets — Annie Duke (deciding well under uncertainty; separating decisions from outcomes) 📘 Decisive — Chip & Dan Heath (a full, practical decision process) 📘 The Paradox of Choice — Barry Schwartz (why more options can paralyze) 📺 Search "pre-mortem Gary Klein" — the failure-imagining technique in 3 minutes Step 5. Statistics for Reading the World ---------------------------------------- Why: Statistics for reading the world (and spotting misleading charts). Every day the world hands you numbers and asks you to draw conclusions — a study, a poll, an average, a risk, a trend. Statistics is the toolkit for doing that honestly instead of being fooled. You don't need the formulas. You need the handful of concepts that let you see what a number is really saying, and what it's quietly hiding. Statistics is often taught as a bag of procedures. But at its heart it's a way of thinking about uncertainty and variation — about how to learn something true from data that's noisy, partial, and easy to misread. Master a few core ideas and headlines stop pushing you around. The average hides as much as it reveals "The average" feels like solid ground. It isn't always. Two things to hold onto: An average can describe almost nobody. If Bill Gates walks into a small café, the average wealth of everyone inside is now hundreds of millions — but no one there feels richer. A single extreme value (an outlier) drags the average (the mean) far from the typical experience. This is why the median — the middle value, where half are above and half below — often tells the truer story for things like income or house prices. When you hear "average," ask: mean or median? And what's the shape underneath? The spread matters as much as the center. Two towns can both have an average temperature of 18°C — one mild year-round, the other baking summers and freezing winters. The average is identical; the experience is nothing alike. That spread is variance (or standard deviation), and ignoring it is one of the most common ways numbers mislead. A comfortable average can hide brutal extremes. "The average person" is often a fiction. Always ask about the spread and the shape, not just the center. Correlation is not causation (the big one) This is the single most important statistical idea for daily life. When two things move together — they're correlated — it is tempting, and usually wrong, to assume one causes the other. Ice-cream sales and drowning deaths rise together. Ice cream doesn't cause drowning; hot weather drives both. That hidden third factor is a confounder, and confounders are everywhere. "People who do X are healthier" almost never establishes that X makes you healthy — maybe healthy people are simply more likely to do X, or something else drives both. How do scientists get to actual causes? Chiefly the randomized controlled trial: split people randomly into two groups, apply the thing to one, and compare. Randomizing scrambles the confounders so any difference is likely due to the treatment. When you see a causal claim ("X reduces Y!"), ask: was it a controlled experiment, or just an observed correlation? The difference is enormous. (See also How to Spot Bullshit.) Sample size and who you asked A statistic is only as good as the data behind it. Two quick checks defuse a lot of nonsense: How many, and who? A pattern in 10 people could easily be chance; the same pattern in 10,000 is far more trustworthy. Small samples are noisy — they swing wildly by luck alone. And who was sampled decides who the result applies to. A poll of one website's visitors doesn't describe the whole country. Selection bias — when the sample isn't representative — quietly wrecks more conclusions than bad math ever does. The classic cautionary tale: in 1936 a magazine polled 2.4 million people and confidently predicted a landslide — for the candidate who then lost in one of history's biggest upsets. Their enormous sample was drawn from car and phone owners, skewing wealthy during the Depression. Size didn't save them; a biased sample doomed them. Representative beats large. Probabilities, risks, and scary headlines Numbers about risk are engineered to alarm. Two habits keep you grounded: Relative vs. absolute risk. "This doubles your risk of a rare disease!" sounds terrifying. But if the risk went from 1 in a million to 2 in a million, the doubling is real and the actual danger is still negligible. Relative change (doubled!) without absolute numbers (…of a tiny thing) is a favorite trick. Always ask: doubled from what to what? Base rates. How common is the thing to begin with? A test that's "99% accurate" for a disease that afflicts 1 in 10,000 people will still flag far more healthy people than sick ones, simply because there are so many more healthy people to misflag. Ignore the base rate and you'll badly misjudge what a positive result means — a mistake even doctors make. Regression to the mean — the invisible trickster Here's a subtle one that fools almost everyone. Extreme results tend to be followed by more ordinary ones, purely by chance. The star athlete who lands a magazine cover after a phenomenal season then "slumps" — not because fame jinxed them, but because a phenomenal season was partly luck that doesn't repeat. A student who bombs a test unusually badly usually does better next time regardless of what you do. This regression to the mean creates illusions of cause everywhere: we credit the intervention (the pep talk, the punishment, the supplement) for an improvement that was going to happen anyway as things drifted back to normal. Whenever you're reacting to an extreme, ask whether plain averaging-out explains the change before you credit the cause. Why this is a superpower You're not going to run the calculations — that's what statisticians are for. But holding these concepts turns you into a far harder person to fool. You'll instinctively ask the right questions: Mean or median? What's the spread? Correlation or cause? How big and how representative was the sample? Relative or absolute risk? Could this just be regression to the mean? Those questions are most of statistical literacy, and they'll protect your money, your health decisions, and your grip on reality. The takeaway Statistics is the art of reading noisy, partial data honestly. Averages can hide outliers and spread (ask mean vs. median, and about variance); correlation isn't causation (beware confounders — controlled experiments establish cause, observations don't); small or biased samples mislead more than bad math; scary risks need absolute numbers and base rates, not just relative change; and extreme results naturally drift back toward average (regression to the mean), faking cause-and-effect. Knowing the questions to ask is most of the skill. Try this: next time you see a health or social-science headline, ask two questions — "correlation or actual experiment?" and "what are the absolute numbers behind that scary percentage?" You'll be amazed how often those two questions deflate the claim. Sources & Further Reading Sources - Mean-vs-median and outlier sensitivity, correlation ≠ causation and confounding, selection bias (the 1936 Literary Digest poll), relative vs. absolute risk, base-rate neglect, and regression to the mean are all standard, foundational statistics. - Regression to the mean was first described by Francis Galton; base-rate reasoning underlies Bayesian statistics. Go deeper - 📘 How to Lie with Statistics — Darrell Huff (the classic, tiny, and still devastating) - 📘 The Art of Statistics — David Spiegelhalter (modern, clear, authoritative) - 📘 Naked Statistics — Charles Wheelan (statistics without the pain) - 🌐 Companion piece: How to Spot Bullshit (in this library) — the media-literacy side of the same skill Step 6. Bayes' Theorem and Positive Tests ----------------------------------------- Why: Why a positive test often doesn't mean what you think. A test that catches 90% of cases comes back positive. What are the odds you actually have the disease? Most people say about 90%. Often the honest answer is about 1 in 10. The difference is a 260-year-old rule about how evidence should change your mind. A note before we start: this piece is about how to read test results, not medical advice. Real tests vary by type, age and setting, and a doctor weighs far more than one number. If you get a positive result, ask your doctor what it means for you. In 2007 the psychologist Gerd Gigerenzer put a version of this problem to about 160 gynaecologists. Imagine screening women for breast cancer. About 1% of them have it. If a woman has it, the mammogram is positive 90% of the time. If she doesn't, it is still positive 9% of the time. A woman tests positive. How likely is it that she has cancer? The most common answer was 90%. The right answer is about 9%, roughly 1 in 10. These were specialists who give such results to patients, and the error is not about medicine. It is about a piece of arithmetic almost nobody is taught to see. Turn percentages into people Gigerenzer's fix is simple: stop thinking in percentages and think in people. Picture 1,000 women. Test positiveTest negative 10 have cancer (1%)91 990 don'tabout 89about 901 Now count the positives: 9 real cases plus about 89 false alarms, about 98 in all. Only 9 of those 98 have cancer. That's 9.2%. The test isn't bad. The problem is that healthy women vastly outnumber sick ones, so even a small false-alarm rate, applied to a big crowd, swamps the real cases. A positive result doesn't tell you that you have the disease. It tells you that you've moved into a smaller, more worried crowd. What matters is who else is in that crowd. Try it: the positive-test calculator Interactive · 1,000 people, one test Screening (Gigerenzer's numbers) Rare disease, excellent test Same test, high-risk patient How common is it? (prevalence) Sick people the test catches (sensitivity) Healthy people it wrongly flags (false-positive rate) sick, tests positive sick, missed healthy, false alarm healthy, negative chance that a positive result is a real case The dots are rounded to whole people; the percentages use the exact rates. Play with it. Drag prevalence up and the same test becomes trustworthy. Drag the false-positive rate down and watch the orange dots vanish. The "Same test, high-risk patient" button is the most important lesson here. Someone who already has symptoms belongs to a very different starting crowd from someone at a routine screening. If her starting chance were, say, 30% (an illustrative figure), the very same positive result would mean about 81% rather than 9%. The rule itself What you just did by counting has a name. Bayes' theorem says, in words: the chance you have the disease, given a positive test, equals the share of all positive tests that come from sick people. In symbols: P(sick | positive) = P(positive | sick) × P(sick) ÷ P(positive) The left side is what you want to know. The right side contains what the test-maker tells you (sensitivity) and the thing everybody forgets: P(sick), the base rate, how common the condition was before you took the test. Leave it out and you get the gynaecologists' 90%. The version almost nobody learns: odds There is a cleaner way to write the rule, and it is the version that changes how you think. Use odds instead of probabilities. Odds after the evidence = odds before × the strength of the evidence. The strength of a positive test is its likelihood ratio: how much more often it happens to sick people than to healthy ones. Here that is 90% ÷ 9% = 10. Before the test, the odds were 1 to 99 (10 sick women against 990 healthy ones). Multiply by 10 and you get 10 to 99, which is 9.2%. Same answer, no table needed. This form tells you something deep. Evidence doesn't replace what you knew before. It multiplies it. A test that multiplies the odds by 10 is strong evidence, but ten times "very unlikely" is still "unlikely". The calculator's readout shows both steps. Two positives are much stronger than one Suppose the woman takes a second test whose errors are independent of the first. Now her starting point is no longer 1%. It is 9.2%, the result of the first test. Multiply those odds by 10 again and she is at about 50%. A third positive would take her to about 91%. Each piece of evidence starts from wherever the last one left you. That is what learning looks like when it's done properly. The catch is the word independent. Repeating the same test on the same person often repeats the same error (the same harmless shadow on a scan, say). That is why a follow-up test that works in a different way tells you more than simply repeating the first one. A rule found twice Thomas Bayes died in Tunbridge Wells, in England, in 1761 without publishing it. His friend Richard Price found the essay among his papers, edited it, added to it and sent it to the Royal Society, which published it in 1763. How much of the final text is Price's is still debated. In 1774 the French mathematician Pierre-Simon Laplace arrived at the same idea, apparently independently, and developed it into a general method. Historians often say Bayes deserves the name for being first and Laplace deserves the credit for making it a tool. Why this is more than a medical trick Bayes' rule is the arithmetic of changing your mind. Every belief starts somewhere, every piece of evidence pushes it by a factor, and the result is your new starting point. It quietly explains a lot. It explains why a surprising claim needs strong evidence: when the starting odds are tiny, a modest multiplier leaves them small. It explains why two honest people with different starting points can see the same evidence and still disagree, and why they should end up closer together than they began. It also contains a small warning. If your starting probability is exactly zero, no evidence can ever move it, because anything times zero is zero. The one belief Bayes' rule cannot repair is total certainty. The takeaway What a positive test means depends on how common the condition was to begin with. Imagine 1,000 people, count the true positives and the false alarms, and see what share of the positives are real. In odds form, Bayes' theorem says evidence multiplies your starting odds by its likelihood ratio. Strong evidence about something rare can still leave it unlikely, and a second, independent piece of evidence builds on the first. Try this: the next time you hear "the test is 95% accurate", ask two questions. How common is the thing in people like me? How often does the test flag people who don't have it? Then picture 1,000 of those people and count. Sources & Further Reading Sources. The screening problem (1% prevalence, 90% sensitivity, 9% false-positive rate) and the 160 gynaecologists whose most frequent answer was 90%: G. Gigerenzer, Cochrane Colloquium abstract (2008) and APS Observer (Nov 2007); see also Gigerenzer et al., "Helping Doctors and Patients Make Sense of Health Statistics", Psychological Science in the Public Interest 8(2), 2007. These are teaching numbers, not current statistics for any particular screening programme. Bayes' essay: Philosophical Transactions of the Royal Society 53 (1763), published by Richard Price; Bayes died 7 April 1761. Laplace's independent 1774 work: history-of-statistics literature (Stigler, Dale, Hald). Every percentage in this piece and in the calculator was computed in node and Python (e.g. 0.9×0.01 ÷ (0.9×0.01 + 0.09×0.99) = 9.17%; a second independent positive gives 50.3%, a third 91.0%). 📘 Calculated Risks (UK title Reckoning with Risk) — Gerd Gigerenzer (natural frequencies, explained for everyone) 🌐 Gigerenzer et al., "Helping Doctors and Patients Make Sense of Health Statistics" (the research behind the method) 📘 The Theory That Would Not Die — Sharon Bertsch McGrayne (the rule's dramatic history) 🌐 Companion pieces in this library: Probability: How to Not Fool Yourself and Statistics for Reading the World Step 7. Bayesian Thinking for Everyday Life ------------------------------------------- Why: Bayesian thinking: updating your mind the right amount. You take a medical screening test. It catches 90% of people who have the condition, and it's wrong for only 9% of people who don't. Your result is positive. How worried should you be? The instinct says: very likely, maybe 90%. Research on how people handle numbers like these, doctors included, finds that most get them badly wrong. If the condition affects 1 in 100 people of your age, the real answer is about 9%. That gap, between what the evidence feels like it says and what it actually says, is what Bayesian thinking fixes. It isn't a formula you have to memorize. It's a habit: start from how likely something was before the new evidence, then ask how much the evidence should move you. Doing it without any algebra The trick, championed by the German psychologist Gerd Gigerenzer and colleagues, is to stop thinking in percentages and think about a crowd of real people. Their studies found that people, experts included, reason far more accurately when the same numbers are given as "natural frequencies," counts of people, rather than as probabilities. Imagine 1,000 people taking that test: Have the condition Don't have it People 10 (1 in 100) 990 Test positive 9 (90% of 10) about 89 (9% of 990) Ninety-eight people test positive, and only 9 of them are actually ill. So a positive result means about a 9 in 98 chance, roughly 9%. The test is good. The problem is that the healthy group is so large that even a small error rate produces far more false alarms than there are true cases. This is not an argument against testing. It's why a positive screening result usually leads to a second, more precise test rather than straight to treatment, and why the numbers matter when you're the one being tested. Gigerenzer's team gives another example: the claim that mammography screening "reduces the risk of dying from breast cancer by 25%" turns out, in their analysis, to mean about 1 fewer death per 1,000 women screened. Both statements can be true. Only one helps you decide. The question is never just "how well does the evidence fit my idea?" It's "how well does it fit my idea compared with the alternatives, given how common each was to begin with?" The two ingredients Every Bayesian update has two parts: 1. The prior: how likely was this before the new evidence? Before the test, your chance was 1 in 100, because that's how common the condition is. Ignoring this starting point is called base-rate neglect, one of the biases covered in The Biases That Fool Everyone, this category. 2. The strength of the evidence: how much more likely is this evidence if the idea is true than if it's false? A positive result is 10 times more likely if you're ill (90%) than if you're healthy (9%). That ratio, called the likelihood ratio, is what tells you how far to move. Put together, the rule is short: your new odds = your old odds × the strength of the evidence. Old odds of 1 to 99, times 10, gives about 10 to 99, roughly 9%. A second independent positive test multiplies by 10 again, and now you're at about 50–50. Evidence accumulates; you don't have to get it all from one clue. Everyday uses Reading someone's mood. A colleague sends a curt two-line email. Is she angry with you? Start with the prior: in an ordinary week, maybe there's a 1 in 10 chance she's annoyed with you. Then the evidence: curt emails might be twice as common when she's annoyed as when she's simply busy. Odds go from 1 to 9 to 2 to 9, about 18%. Worth noticing, not worth a sleepless night. If she also skips your meeting, update again. Judging a news story. A dramatic claim surprises you. Before asking whether the evidence fits, ask how often claims like this, from sources like this, have turned out true. A strong prior against something isn't closed-mindedness; it's memory. (See How to Spot Bullshit, this category.) Diagnosing a problem. Your car won't start on a cold morning. A dead battery and a failed starter both explain it, but dead batteries are far more common, especially in the cold. Check the common thing first. Your own plans. The best guide to how long your project will take is usually how long similar projects took, yours and other people's, not your detailed plan for this one. Start there, then adjust for what's genuinely different. Two rules worth keeping Never be 100% sure. If your prior is exactly 0 or exactly 1, no evidence can ever move it: anything times zero is still zero. The statistician Dennis Lindley named this "Cromwell's rule," after Oliver Cromwell's 1650 plea to the Church of Scotland: "I beseech you, in the bowels of Christ, think it possible that you may be mistaken." Leave yourself a sliver of doubt about almost everything. Extraordinary claims need extraordinary evidence. Carl Sagan made the phrase famous in the late 1970s, but the idea is much older. David Hume made the argument about miracles in 1748, and the mathematician Pierre-Simon Laplace wrote that the weight of evidence for an extraordinary claim must match its strangeness. In Bayesian terms: the lower the prior, the stronger the evidence has to be to get you to "probably true." How the best forecasters do it In a US government-funded forecasting tournament that ran from 2011, the Good Judgment Project, led by Philip Tetlock and Barbara Mellers, recruited ordinary volunteers to predict world events. The project beat the other research teams in the tournament by a wide margin, and its best volunteers, the "superforecasters," were the most accurate of all. A 2020 study of the project's data had a title that sums up one of the reasons: "Small steps to accuracy: Incremental belief updaters are better forecasters." They didn't swing wildly with every headline, and they didn't ignore news either. They moved a little, often. The honest limits Priors can be hard to know, and two reasonable people may start from different ones. Bayesian reasoning doesn't remove that disagreement; it makes it visible, which is usually more useful than pretending it isn't there. It also depends on honest estimates of evidence strength, and it's easy to fool yourself about those. And it's a habit of mind, not a demand to put numbers on everything. Rough numbers ("about 1 in 10," "twice as likely") are enough to stop most of the big mistakes. The method is named after Thomas Bayes, an English minister who worked out the core idea. He never published it: his friend Richard Price found the essay after Bayes died in 1761 and presented it to the Royal Society in 1763. Laplace later developed it into the general approach we use today. The takeaway Bayesian thinking means starting from how likely something was before the evidence (the prior), then moving by how much more likely the evidence is if the idea is true than if it's false. Thinking in counts of people ("of 1,000 who take the test...") makes this easy and exposes how rare conditions plus decent tests still produce mostly false alarms. Never be completely certain, demand more evidence for more surprising claims, and update in small, frequent steps. Try this: the next time something worries you, an odd symptom, a rumour, a curt message, write down two numbers before reacting: how likely this was before (out of 10 or 100), and how many times more likely the evidence is if the worry is true than if it isn't. Multiply, and see where you land. Sources & Further Reading Sources - Natural frequencies: Hoffrage U., Lindsey S., Hertwig R. & Gigerenzer G., "Communicating statistical information," Science 290 (2000). - Mammography "25%" equals about 1 in 1,000: Gigerenzer G. et al., "Helping doctors and patients make sense of health statistics," Psychological Science in the Public Interest 8 (2007). - The worked test example (1% prevalence, 90% sensitivity, 9% false-positive rate) is illustrative; its numbers were computed directly. - Cromwell's rule: Dennis Lindley; Cromwell's letter of 3 August 1650 to the General Assembly of the Church of Scotland. - "Extraordinary claims": Sagan in Broca's Brain (1979) and Cosmos (1980); Hume, "Of Miracles" (1748); Laplace's essays (1810, 1814). - Good Judgment Project and Atanasov P., Witkowski J., Ungar L., Mellers B. & Tetlock P., "Small steps to accuracy: Incremental belief updaters are better forecasters," Organizational Behavior and Human Decision Processes 160 (2020). - Bayes's death (1761) and the essay's presentation by Richard Price (1763): standard histories of probability. Go deeper - 📘 Risk Savvy: How to Make Good Decisions — Gerd Gigerenzer (natural frequencies and medical numbers for non-mathematicians) - 📘 Superforecasting — Philip Tetlock & Dan Gardner (how the best forecasters actually update) - ▶️ 3Blue1Brown: "Bayes theorem, the geometry of changing beliefs" (a beautiful visual explanation) - 🌐 See also: Probability: How to Not Fool Yourself (Mathematics category), for Monty Hall and the birthday paradox Step 8. Thinking in Systems --------------------------- Why: Systems thinking: why fixes backfire. Some problems don't have a villain. They have a structure: things that pile up, loops that feed back, and delays that make sensible people overreact. Once you can see the structure, a lot of baffling behaviour stops being baffling. You step into an unfamiliar shower. The water is cold, so you turn the tap toward hot. Nothing happens, so you turn it more. Then the hot water arrives all at once and you're scalded, so you crank it back toward cold, and a few seconds later you're freezing again. Nobody in that shower is stupid. The problem is a delay between your action and its effect, and a brain that keeps reacting to the water it feels now instead of the water already on its way. That, in miniature, is what systems thinking is about. The same pattern shows up in supply chains, economies, diets, ecosystems and organisations, and you can learn to spot it. Three building blocks 🛁 Stocks and flowsA stock is anything that accumulates: water in a bathtub, money in an account, stock on a shop's shelves, trust in a relationship. It changes only through flows: water in through the tap and out through the drain, income and spending. A stock can't jump; it fills and drains over time. That's why a bank balance, a reputation or a forest can't be rebuilt overnight, however hard you push the inflow. ⚖️ Balancing loopsA balancing (or negative) feedback loop pushes a stock toward a goal. A thermostat turns the heating on when the room is cold and off when it's warm. Your body does the same with temperature and blood sugar. Balancing loops create stability, and they are why systems resist being pushed. 🔁 Reinforcing loopsA reinforcing (or positive) loop makes a change feed on itself. Interest earns interest; popular videos get recommended and become more popular; panic selling causes falling prices that cause more panic. Reinforcing loops create growth and collapse. (See Why Exponential Growth Rules Everything in Mathematics.) Add a fourth ingredient, delay, and things get interesting. A balancing loop with a long delay doesn't settle calmly at its goal. It overshoots, corrects, overshoots the other way, and can end up swinging harder and harder. That's the shower. Try it in a shop. Simulator · A shop with a slow supplier You run a shop that wants to keep 100 units on the shelf. Customers buy 20 a week, until week 5, when demand rises to 30 and stays there. Every week you reorder: what customers just bought, plus a correction toward your target. Orders take time to arrive. Change the delay and how aggressively you correct, and watch the shelf. Delivery delay4 weeks How hard you correct each week50% of the gap Count the orders already on their way before reordering Units on the shelf Units ordered that week Target (100) The model, so you can check it: each week, stock = last week's stock + the delivery arriving − demand. Order = this week's demand + correction × (100 − stock). With the box ticked, the gap also includes the difference between the orders that should be in transit and those actually in transit. Stock below zero means customers waiting for back-orders. Lines leaving the chart are clipped. With a one-week delay, almost any sensible rule works: the shelf dips when demand rises and recovers within a few weeks. Stretch the delay to four weeks and the same rule (correcting 50% of the gap) makes the shelf swing between 25 and 185 units, and the swings keep growing. Nothing outside changed after week 5. Demand was steady. The oscillation is created entirely by the ordering rule meeting the delay. Two fixes show up when you play. Correcting more gently (try 30%) slows the swings down so they fade. But the real cure is the checkbox: before ordering more, count what's already in the pipeline. That one change in what the manager pays attention to removes the oscillation entirely. When a system misbehaves, look at what the people in it can see, and how late they see it, before you blame the people. This isn't just a toy. At MIT around 1960, Jay Forrester, the founder of the field called system dynamics, developed the "Beer Game," in which teams play retailer, distributor and brewery passing orders down a supply chain with delays. Players routinely produce wild swings in orders and stock from a single small change in customer demand. Real companies see the same thing; the term "bullwhip effect" took hold around 1990 when Procter & Gamble noticed wildly erratic ordering of its nappies, a product whose actual use by babies is about as steady as demand gets. Why fixes backfire Because systems push back, a fix aimed at one part can produce the opposite of what was intended somewhere else. In 1902, French colonial authorities in Hanoi, fighting the rats that swarmed the city's sewers, paid a bounty for every rat tail handed in. Before long officials were seeing tailless rats running around the city: catchers had been cutting off tails and releasing the rats to breed. The episode is documented by historian Michael Vann. You may have heard a similar story about a cobra bounty in British India, the "cobra effect." Be careful with that one. A 2025 investigation of colonial records found no contemporary evidence of cobra farming, and the tale seems to trace back to an 1873 newspaper report of something "alleged." The Hanoi rats are the better example, because they actually happened. Where to push The environmental scientist Donella Meadows, lead author of The Limits to Growth (1972), spent her career on this question. In a 1999 essay, "Leverage Points: Places to Intervene in a System," she ranked twelve kinds of intervention. A few of them, from weakest to strongest: 12Numbers, such as taxes, subsidies, standards. Where most debate happens, and usually the least powerful lever. 9The length of delays, relative to how fast the system changes. Often you can't shorten them, but you can stop acting as if they weren't there. 6Information flows: who gets to see what, and when. The simulator's checkbox is exactly this. 5The rules: incentives, punishments, constraints. The Hanoi bounty was a badly designed rule. 3The goal of the system. Change what a system is trying to achieve and nearly everything inside it rearranges itself. 2The mindset the system grows out of. The hardest to change, and the most powerful. She added a warning that is worth remembering: people often find the right leverage point by instinct, and then push it in the wrong direction. The honest limits Systems thinking can be misused. "It's all connected" is true and useless; a systems explanation is only worth something when it names specific stocks, loops and delays that you could check. Real systems also have far more moving parts than any diagram, and models built on wrong assumptions can be confidently wrong. Use it as a way to ask better questions, such as what accumulates here, what feeds back, and what's delayed?, not as a machine that hands out answers. The takeaway Many stubborn problems come from structure rather than villains. Stocks change only through flows; balancing loops resist change; reinforcing loops create growth and collapse; and delays turn sensible corrections into overshoot and oscillation. When a system keeps misbehaving, check what the people inside can see and how late they see it, distrust fixes that create perverse incentives, and remember that changing information, rules and goals usually beats tweaking numbers. Try this: pick one recurring problem in your life or work (a budget that always overruns, a team that's always firefighting) and draw it as a stock with arrows in and out. Mark one delay. Then ask what you would see differently if you counted what's already "in the pipeline." Sources & Further Reading Sources. Donella Meadows, "Leverage Points: Places to Intervene in a System" (1999) and Thinking in Systems: A Primer (2008, published after her death in 2001). The Beer Game was developed in Jay Forrester's system-dynamics group at MIT; John Sterman's 1989 study in Management Science, "Modeling Managerial Behavior: Misperceptions of Feedback in a Dynamic Decision Making Experiment," analysed why players produce oscillations. Bullwhip effect and Procter & Gamble: standard supply-chain histories. Hanoi rat bounty: Michael G. Vann, "Of Rats, Rice, and Race," French Colonial History (2003), and The Great Hanoi Rat Hunt (2019). The term "cobra effect" was coined by economist Horst Siebert (2001); the doubts about its historical basis come from a 2025 investigation by the Friends of Snakes Society, summarized in Wikipedia's "Perverse incentive." The simulator's figures were computed and checked in a separate script before publishing. 📘 Thinking in Systems: A Primer — Donella Meadows (the clearest introduction there is) 🌐 "Leverage Points: Places to Intervene in a System" — Donella Meadows (the full essay, free) 📘 The Great Hanoi Rat Hunt — Michael G. Vann (the true story behind the best perverse-incentive example) 📘 Business Dynamics — John Sterman (the full technical treatment, for readers who want to build models) Step 9. Why You Understand Less Than You Think ---------------------------------------------- Why: You understand less than you think, and that is useful to know. Do you know how a bicycle works? Most people would say yes without hesitating. You've ridden one, seen hundreds, maybe fixed a chain. Now try this before you read on: picture a simple sketch of a bicycle frame, two wheels, pedals and chain. Where does the chain go? Which parts does the frame connect? When psychologist Rebecca Lawson tested people's understanding of basic bicycle design in 2006, many made frequent and serious mistakes, such as believing the chain went around the front wheel as well as the back one, which would make the bike impossible to steer. Bicycle experts made fewer errors but still made some, and so did people who had a real bicycle in front of them during the test. That's not a story about bicycles. It's about a gap almost everyone has between feeling that we understand something and actually understanding it. The illusion of explanatory depth In 2002, psychologists Leonid Rozenblit and Frank Keil of Yale University gave the gap a name: the illusion of explanatory depth. In a series of twelve studies, they asked people (Yale undergraduates) to rate how well they understood how various devices worked, then to write a detailed explanation of how each one works, and then to rate their understanding again. Ratings dropped once people tried to explain. They had mistaken familiarity for knowledge. Two of their findings are especially useful: The illusion is strongest for explanations, knowledge of how things work, and much weaker for facts (a country's capital), procedures (how to make a dish) or stories (the plot of a film). It's easy to check whether you know a fact. It's hard to notice that your "explanation" is a blank with a label on it. It's strongest for things with visible parts. When you can see the pieces of something move, like a bicycle's chain and gears, it feels as if seeing them means understanding them. Knowing what something is called, or how to use it, is not the same as knowing how it works. The feeling of understanding doesn't tell those apart. This is related to, but different from, the Dunning–Kruger effect (see The Biases That Fool Everyone, this category). Dunning–Kruger is about beginners misjudging their skill. The explanatory-depth illusion hits nearly everyone, experts included, outside their own narrow specialty, and often inside it. Why our brains get away with it Partly, it's efficient. You don't need to understand a fridge's refrigeration cycle to keep milk cold. Cognitive scientists Steven Sloman and Philip Fernbach argue in The Knowledge Illusion that human knowledge mostly lives in communities, in other people, books, tools and the world itself, rather than in any single head. We get by because someone, somewhere, does understand the fridge. The trouble starts when we forget that the understanding is borrowed, and act as though it were our own. Where it costs us: opinions That's where the illusion stops being harmless. In 2013, Fernbach and colleagues asked people about complex political policies they felt strongly about. People rated their understanding, then were asked to explain, step by step, how the policy would actually work. Their ratings of their own understanding fell, and their opinions became more moderate. The detail that matters most: when people were instead asked to list the reasons for their position, their views didn't soften. Arguing for a view doesn't expose the gaps. Explaining the mechanism does. In a third experiment, people who had explained a policy were also less willing to donate to advocacy groups on that issue. That's a sobering finding for anyone who argues about politics, health or economics online. Our strongest opinions are often about systems we couldn't explain in a paragraph. How to test yourself The fix is simple and slightly uncomfortable: try to explain it. Not "summarize it" or "argue for it," but explain how it works, step by step, as if to a curious 12-year-old who keeps asking "and then what?" Write it down. Explanations in your head skip steps without telling you. On paper, the gaps show up as the place where you stop being able to write the next sentence. Use "how," not "why." "Why is inflation bad?" invites reasons. "How does printing money raise prices, step by step?" exposes whether you know the mechanism. Mark the borrowed parts. When you hit something you can't explain, label it: "I'm trusting experts here." That's fine and usually unavoidable. It just changes how confidently you should talk about it. This is close to what's often called the Feynman technique, explaining something in plain language to find what you don't understand (covered as a study tool in How to Learn Anything Faster, this category). Here the point is broader: it's a check on your confidence in everything, not just in what you're studying. The honest limits You can't verify everything yourself, and you shouldn't try; relying on trustworthy expertise is a reasonable way to live. The goal isn't to understand every system. It's to know which of your beliefs you actually understand and which you're borrowing, and to hold the borrowed ones a little more loosely. The research also has limits: it rests on a fairly small set of studies, and the effect on opinions was a moderation, not a conversion. Explaining things won't make people agree. It tends to make them less certain, which is often the beginning of thinking well. The takeaway Most of us feel we understand how things work far better than we do, an effect called the illusion of explanatory depth. It's strongest for explanations of mechanisms, especially ones with visible parts, and it extends to political and economic opinions, which grow more moderate when people try to explain how a policy would actually work. The best test is to explain something step by step, in writing, using "how" rather than "why," and to mark honestly which parts of your understanding are borrowed from others. Try this: pick an opinion you hold strongly about a policy or a technology. Write, step by step, how it actually works or would work, without arguing for your side. Notice where you get stuck, and whether your certainty changes. Sources & Further Reading Sources - Rozenblit L. & Keil F., "The misunderstood limits of folk science: An illusion of explanatory depth," Cognitive Science 26 (2002). - Lawson R., "The science of cycology: Failures to understand how everyday objects work," Memory & Cognition 34 (2006). - Fernbach P. M., Rogers T., Fox C. R. & Sloman S. A., "Political extremism is supported by an illusion of understanding," Psychological Science 24 (2013). - Sloman S. & Fernbach P., The Knowledge Illusion: Why We Never Think Alone (2017). Go deeper - 📘 The Knowledge Illusion: Why We Never Think Alone — Steven Sloman & Philip Fernbach (the full argument, readable and funny) - 📘 The Scout Mindset — Julia Galef (on wanting to see things as they are, not as you'd like them to be) - 🌐 See also, in this category: How to Spot Bullshit and The Biases That Fool Everyone Step 10. Propaganda, Misinformation & How to Resist It ------------------------------------------------------ Why: Propaganda and misinformation: how they work and how to resist them. In 1983 a small pro-Soviet newspaper in India, the Patriot, printed an anonymous letter claiming that the United States had created AIDS as a biological weapon. Over the next few years the story was pushed by the Soviet KGB with help from East German and Bulgarian intelligence, dressed up in a pseudo-scientific study, and repeated in media in dozens of countries. In 1992 the head of Russia's foreign intelligence service publicly acknowledged the KGB's role. By then the lie had a life of its own, and it outlived the Soviet Union that launched it. That is how modern disinformation tends to work: start somewhere obscure, launder it through seemingly independent voices, and let ordinary people do the rest. The good news is that the tricks are old and learnable, and so are the defences. This piece describes techniques, not sides. Every political camp, in every country and era, has produced propaganda, and every camp's supporters have fallen for it. Three kinds of bad information In 2017 the researchers Claire Wardle and Hossein Derakhshan, in a report for the Council of Europe, proposed a now widely used vocabulary: Term Is it false? Is there intent to harm? Example Misinformation Yes No Your relative shares a wrong health tip, believing it Disinformation Yes Yes A fabricated story planted to discredit someone Malinformation No, it's genuine Yes Leaking someone's private records to hurt them Propaganda is broader and older: communication designed to shape opinion for a cause, usually by appealing to emotion and identity rather than evidence. It isn't always false. It can be entirely accurate and still be one-sided, selective and manipulative. The old toolkit In 1937, as propaganda was becoming a mass industry, a group of American educators founded the Institute for Propaganda Analysis and listed seven common devices. They still describe much of what fills a feed today: Name-calling: attach a frightening label so people reject an idea without examining it. Glittering generalities: wrap a cause in vague, warm words such as freedom, family or the people. Transfer: borrow the prestige of a respected symbol (a flag, a religion, science) for something it has nothing to do with. Testimonial: get a celebrity or authority to endorse it, whether or not they know anything about it. Plain folks: "I'm just like you," from someone who isn't. Card stacking: present only the facts that help, and bury the rest. Bandwagon: "Everyone is joining; don't be left behind." States have built whole machines around these. Nazi Germany created a Ministry of Public Enlightenment and Propaganda in March 1933 to control the press, radio, film, literature and the arts. Most propaganda, though, comes from much smaller players: campaigns, companies, activists, and individuals with an audience. Why it works on smart people Repetition feels like truth. Since a 1977 study by Lynn Hasher and colleagues, psychologists have repeatedly found the illusory truth effect: people rate statements as more believable simply because they have heard them before. A 2015 study found it works even on people who know the correct answer. Each time a claim is repeated, even to debunk it, it becomes a little more familiar. The brain uses "I've heard this before" as a shortcut for "this is probably true." Propagandists know it. Falsehood travels faster. A 2018 study in Science by researchers at MIT followed about 126,000 stories shared on Twitter between 2006 and 2017. False stories were about 70 percent more likely to be reshared than true ones, and true stories took about six times as long to reach 1,500 people. Bots didn't explain the gap; people did. False stories tend to be more novel and more emotional, which is exactly what makes us click and share. Identity beats evidence. We are far more critical of claims that threaten our group than of claims that flatter it. That's why the most effective propaganda is aimed at people who already half-believe it. How to resist it 1. Pause on strong emotion. Outrage, fear and triumph are the feelings disinformation is built to trigger. If a post makes you furious or gleeful, that's the moment to slow down, not to share. 2. Read laterally. Researchers at Stanford who watched professional fact-checkers at work found they behave differently from most of us. Instead of studying a website closely to judge it, they quickly open new tabs and ask: Who is behind this? What do other sources say about them? Checking what the rest of the web says about a source is faster and more reliable than judging it by its own appearance. 3. Find the original. Screenshots, clips and quotes are easy to crop and strip of context. Look for the full speech, the actual study, the original photo (a reverse image search often shows a picture is years old or from somewhere else entirely). 4. Ask what's missing. Card stacking only works if you don't notice the cards that aren't there. What would someone on the other side say? What number or comparison is left out? 5. Don't amplify by debunking. Because repetition breeds familiarity, lead with the truth rather than the myth when you correct something, and don't reshare a false claim even to mock it. 6. Get inoculated. Psychologists have found that, as with a vaccine, a small dose of manipulation, shown and explained in advance, builds resistance. In a 2022 study, researchers working with Google showed short "prebunking" videos explaining tricks such as scapegoating and false dichotomies; viewers became better at recognising those tricks. The effect was modest and fades after some weeks without a "booster," but it works across political lines because it teaches techniques rather than telling people what to believe. The hard part: who decides what's false? Any serious response faces a real dilemma. Platforms and governments can label, remove or down-rank false content, and some countries have passed laws against "fake news." Supporters argue that unchecked disinformation can incite violence and undermine elections and public health. Critics point out that such powers can be, and have been, used to silence journalists, dissidents and inconvenient truths, and that yesterday's "misinformation" is occasionally today's accepted fact. Most democracies are still searching for the balance, and reasonable people disagree about where it lies. That's one more reason the most durable defence is a public that can spot manipulation for itself. The takeaway Misinformation is false but innocent, disinformation is false and deliberate, and malinformation is true but weaponised. Propaganda uses old, nameable tricks, and it works because repetition feels like truth, falsehood is more novel and emotional, and identity outweighs evidence. You can resist it by pausing on strong emotion, reading laterally, finding originals, asking what's missing, and learning the techniques in advance. Deciding who gets to label something false is a genuine democratic dilemma, which makes individual skill all the more important. Try this: play Bad News, a free browser game built by Cambridge researchers in which you play a disinformation producer using the classic tricks. Fifteen minutes on the other side of the screen makes the techniques hard to miss afterwards. Sources & Further Reading Sources - Operation INFEKTION / Operation Denver: the 1983 Patriot letter, KGB, Stasi and Bulgarian involvement, and the 1992 admission by Yevgeny Primakov, head of Russia's foreign intelligence service (historical accounts summarised in standard references). - Claire Wardle & Hossein Derakhshan, Information Disorder (Council of Europe, 2017). - Institute for Propaganda Analysis (1937–1942) and its seven devices. - Reich Ministry of Public Enlightenment and Propaganda, established 13 March 1933. - Hasher, Goldstein & Toppino (1977) and Fazio et al. (2015) on the illusory truth effect. - Vosoughi, Roy & Aral, "The spread of true and false news online," Science (2018). - Stanford researchers' studies of how professional fact-checkers evaluate sources (Wineburg & McGrew), the basis of the Civic Online Reasoning curriculum. - Roozenbeek et al., "Psychological inoculation improves resilience against misinformation on social media," Science Advances (2022); research on the decay of inoculation effects. Go deeper - 🌐 Bad News (the inoculation game) - 🌐 Civic Online Reasoning (free lessons in lateral reading and source evaluation) - 🌐 MIT News — "Study: On Twitter, false news travels faster than true stories" - 📘 Foolproof — Sander van der Linden (the psychology of misinformation and how inoculation works) Step 11. Critical Thinking & Epistemic Hygiene: How to Spot Bullshit in the AI Era ---------------------------------------------------------------------------------- Why: Critical thinking in the age of AI. For most of human history, information was scarce, expensive, and difficult to distribute. If you wanted to learn about history, science, or politics, you had to visit a physical library, buy an encyclopedia, or consult an expert. Today, artificial intelligence, synthetic media, automated social media bots, and algorithmic feeds have inverted that reality: information is now infinitely abundant and nearly free, but truth and verification have become dangerously scarce. In an era where AI can generate millions of plausible-sounding essays, fake news articles, and hyper-realistic deepfakes in seconds, the ability to practice epistemic hygiene and spot bullshit is no longer an academic exercise—it is a vital survival skill for a free mind. Information Nutrition: What Is Epistemic Hygiene? Your "epistemic state" is the sum total of beliefs, concepts, and assumptions you hold about reality. Just as eating unwholesome junk food degrades your physical health, consuming unverified claims, rage-bait headlines, and unchecked AI outputs degrades your epistemic health—your capacity to perceive reality accurately and make sound decisions. Epistemic hygiene means treating the information you allow into your mind with the same care and scrutiny you apply to the food you put into your body: Information Pollution: Unchecked algorithmic feeds flood your nervous system with sensationalized claims designed to trigger anger or confirmation bias. AI Hallucinations: Large Language Models (LLMs) do not "know" facts the way humans do; they are statistical prediction engines optimized to generate plausible-sounding text. When an AI generates a fake citation, a made-up historical date, or a non-existent legal case, it is committing an AI hallucination. Harry Frankfurt's Definition of Bullshit: In his classic philosophical essay On Bullshit (first published in 1986, as a short book in 2005), Princeton philosopher Harry Frankfurt drew a crucial distinction: A Liar knows the truth and deliberately tries to hide it. A Bullshitter does not care whether what they are saying is true or false; they care only about generating an impression or persuading the audience. Some philosophers (Michael Townsen Hicks, James Humphries and Joe Slater, in a 2024 paper titled "ChatGPT is bullshit") argue that AI text models fit Frankfurt's definition: they generate plausible-sounding language with no built-in concern for whether it is true. AI models do not optimize for factual truth; they optimize for statistical plausibility. Verifying primary facts remains a uniquely human responsibility. The Fact-Checker's Protocol: Lateral Reading How do professional fact-checkers verify claims quickly when evaluating unknown websites or AI-generated text? Stanford University researchers (Sam Wineburg and Sarah McGrew, 2017) watched 10 professional fact-checkers, 10 PhD historians and 25 Stanford undergraduates evaluate unfamiliar websites. The students and historians mostly engaged in Vertical Reading—reading down the page, examining the website's "About Us" page, and judging the site by its design and the text itself. They were often fooled by easily faked signals such as an official-looking logo, a polished design or a respectable-sounding domain name. Professional fact-checkers engaged in Lateral Reading: 1. After only a quick scan of an unfamiliar site or claim, they left the site. 2. They opened multiple new browser tabs to search for independent, authoritative sources about the author or organization making the claim. 3. They asked: "Who is behind this claim? What are their funding sources, credentials, and conflicts of interest?" Two ways to check a source: Vertical reading Lateral reading Verdict easily fooled the fact-checker rule First move stay on the page leave the site after a quick scan Next read 'About Us' open new tabs Judge by the design research on the author Carl Sagan's Baloney Detection Kit In his 1995 book The Demon-Haunted World, astronomer Carl Sagan set out a nine-point "baloney detection kit" for critical thinking. Six of its tools: Demand Independent Confirmation: Wherever possible, seek independent verification of facts across multiple distinct sources. Encourage Substantive Debate: Welcome knowledgeable proponents of all points of view to debate the evidence openly. Beware Arguments from Authority: Authorities make mistakes. "Because Professor X or Politician Y said so" is not proof; evidence is proof. Quantify Whenever Possible: Look for specific, measurable numbers and data rather than vague qualitative adjectives ("many people say" or "a massive increase"). Occam's Razor: When presented with two competing explanations for a phenomenon, choose the explanation that requires the fewest unproven assumptions. Falsifiability: Ask if a hypothesis can, at least in principle, be tested and proven wrong. As Sagan put it, propositions that are untestable and unfalsifiable "are not worth much." Steel-Manning Opposing Views The ultimate test of critical thinking is whether you can Steel-Man an argument you disagree with. Straw-Manning is building a weak, cartoonish version of an opponent's argument so you can easily knock it down. Steel-Manning is constructing the strongest, most intelligent, and most compelling version of an opponent's argument. Before you critique an idea, force yourself to state the opposing argument so accurately and fairly that your opponent would thank you for it. The philosopher Daniel Dennett, drawing on the game theorist Anatol Rapoport, put the test this way: re-express your target's position "so clearly, vividly, and fairly that your target says, 'Thanks, I wish I'd thought of putting it that way.'" Only then have you earned the right to critique it. Takeaways In the AI era, information is infinite, but truth and verification are scarce. Epistemic hygiene involves treating information intake with care, recognizing that AI text models optimize for plausible tone rather than factual truth. Professional fact-checking requires "Lateral Reading"—leaving claims immediately to research the source across independent tabs. By applying Sagan's Baloney Detection Kit and steel-manning opposing views, you protect your mind from algorithmic manipulation. Action step: Pick one surprising claim or news headline you encountered today, open two new browser tabs, and practice "Lateral Reading" by researching who made the claim and whether independent primary sources verify it. Sources & Further Reading Sources - Harry G. Frankfurt's On Bullshit (Princeton University Press, 2005). Carl Sagan's The Demon-Haunted World: Science as a Candle in the Dark (1995). Sam Wineburg and Sarah McGrew, Lateral Reading: Reading Less and Learning More When Evaluating Digital Information (Stanford History Education Group Working Paper 2017-A1). Michael Townsen Hicks, James Humphries and Joe Slater, "ChatGPT is bullshit" (Ethics and Information Technology, 2024). Daniel C. Dennett, Intuition Pumps and Other Tools for Thinking (2013), for Rapoport's rules. Go Deeper - 📘 On Bullshit — Harry G. Frankfurt (a brilliant, 60-page philosophical masterpiece on the nature of ungrounded speech) - 📘 The Demon-Haunted World — Carl Sagan (the classic manifesto for scientific skepticism and critical thinking) - 🌐 Stanford History Education Group — Civic Online Reasoning - 🌐 Related piece in this library: The Biases That Fool Everyone For Your Vocabulary Epistemic Hygiene (the practice of carefully scrutinizing and filtering information sources to maintain an accurate understanding of reality) · Lateral Reading (verifying claims by leaving the original page immediately to research sources across independent tabs) · AI Hallucination (when a Large Language Model generates false facts, quotes, or citations while maintaining a confident tone) · On Bullshit (Harry Frankfurt's definition of speech generated without regard for truth) · Steel-Manning (constructing the strongest, most intelligent version of an opponent's argument before critiquing it) · Straw-Manning (creating a weak or distorted caricature of an opponent's argument to knock it down easily) · Occam's Razor (the problem-solving principle that the simplest explanation with the fewest assumptions is usually preferred) · Falsifiability (the capacity of a statement or hypothesis to be proven wrong through observation or experiment) Step 12. First-Principles Thinking: Deconstructing Problems to Fundamental Truths --------------------------------------------------------------------------------- Why: First-principles thinking. Most human reasoning operates through a cognitive shortcut known as Reasoning by Analogy. We look at how other people or competitors solve a problem, make minor iterative tweaks, and copy the existing template: "We do it this way because that's how everyone in our industry does it." While reasoning by analogy saves energy, it traps individuals and organizations in conventional thinking, incremental progress, and industry blind spots. In contrast, visionary innovators, scientists, and deep problem-solvers utilize First-Principles Thinking—the practice of boiling a complex problem down to its most fundamental, indisputable truths, and constructing a solution up from scratch. What Is a First Principle? In Metaphysics Book V, Aristotle lists among the meanings of a "principle" (archē) "that from which a thing can first be known." In modern physics and philosophy, a first principle is a foundational proposition or fact that cannot be deduced any further and does not rely on assumptions or consensus. To see the difference between Reasoning by Analogy and First-Principles Thinking, consider how different people approach a challenge: Reasoning by Analogy (The Copycat): "Electric car battery packs cost $600 per kilowatt-hour. They've always been expensive, so electric cars will never be affordable for the average person." Reasoning from First Principles (The Deconstructor): "What is a battery pack physically made of? It's made of cobalt, nickel, aluminum, carbon, polymers, and a steel seal. What is the raw spot-market cost of those material elements on the commodities exchange? Only $80 per kilowatt-hour! How can we combine those raw materials differently to make a battery pack much cheaper?" Elon Musk has described making exactly this calculation for batteries, and similar reasoning about the raw-material cost of rockets led him to found SpaceX. Since then, reusable rockets have cut the cost of reaching orbit dramatically, and battery pack prices across the industry have fallen by more than 90% since 2010 (BloombergNEF's December 2025 survey put the average at about $108 per kilowatt-hour, against roughly $1,470 in 2010 in today's money). Reasoning by analogy is like playing a song by ear; reasoning from first principles is like learning the fundamental physics of sound and building an instrument from scratch. The 3-Step First-Principles Method To apply first-principles thinking to any business, technological, or personal challenge, use this three-step framework: Identify assumptions — write down all current "industry rules" & premises Deconstruct to truths — strip away assumptions until only indisputable facts remain Rebuild from scratch — construct a new solution using only those core facts Step 1: Identify and List Current Assumptions Write down every assumption you hold about the problem. If you are launching a product, your assumptions might be: "We need a physical office, we need a 10-person sales team, and we need to charge an annual subscription." Step 2: Deconstruct Down to Fundamental Truths Ask: "Is this assumption physically, legally, or logically necessary, or is it just a convention?" Strip away every assumption until you reach fundamental, indisputable facts (e.g., "Our customers need to solve Problem X, which requires computing power Y and distribution channel Z"). Step 3: Rebuild a Solution from the Ground Up Using only the fundamental truths you identified in Step 2, design a new solution without copying existing industry templates. Problem Framing: The 55-Minute Rule First-principles thinking is useless if you are solving the wrong problem. A saying often attributed to Albert Einstein (there is no evidence he said it) puts it this way: "If I had an hour to solve a problem and my life depended on the solution, I would spend 55 minutes determining the proper question to ask, for once I know the proper question, I could solve the problem in less than five minutes." Teams often pour effort into poorly framed problems. A classic problem-solving parable (retold, for example, by Thomas Wedell-Wedellsborg in Harvard Business Review, 2017) shows the difference: Poorly Framed Problem: "How do we make our elevator faster so occupants stop complaining about wait times?" (Focuses on expensive mechanical engineering). Reframed Problem: "Why are occupants frustrated by the wait?" (Focuses on human psychology. The real issue is boredom while waiting. The solution in the story: installing mirrors next to the elevators, so people have something to look at; the complaints drop sharply, at a fraction of the cost of a faster lift). The 5 Whys Technique Credited by Toyota to Sakichi Toyoda, founder of the Toyota group, and made famous by Taiichi Ohno's Toyota Production System, the 5 Whys method drills past surface symptoms to root causes by asking "Why?" five times in sequence: 1. Why did the machine stop? Because the fuse blew due to an overload. 2. Why was there an overload? Because the bearing was not sufficiently lubricated. 3. Why was it not lubricated? Because the lubrication pump was not pumping effectively. 4. Why was it not pumping? Because the pump shaft was worn and rattling. 5. Why was it worn? Because there was no strainer attached, allowing metal scrap to enter. (Root Cause: attach a strainer!). First Principles in the AI Era Because AI language models are trained on existing human internet text, AI models tend to reason by Analogy. If you ask an AI for a business plan or a creative strategy without first principles, it will give you the average of existing industry practices. When you provide the AI with first-principles constraints and fundamental truths, the AI becomes a high-speed execution engine for breakthrough, non-consensus human ideas. Takeaways Reasoning by analogy copies existing conventions, while First-Principles Thinking deconstructs problems down to their fundamental, indisputable truths to build original solutions. Paired with careful problem framing (the "55-minute rule") and the 5 Whys technique, first-principles thinking prevents you from wasting energy on surface symptoms, unlocking breakthrough innovation in work and life. Action step: Identify a persistent problem or high cost in your work or life, write down three "rules" or assumptions you have always accepted about it, and ask: "Is this rule a fundamental law of physics/logic, or just a convention I can bypass?" Sources & Further Reading Sources - Aristotle's Metaphysics (Book V, 1013a; W. D. Ross translation). BloombergNEF, "Lithium-Ion Battery Pack Prices Fall to $108 per Kilowatt-Hour" (December 2025). Thomas Wedell-Wedellsborg, "Are You Solving the Right Problems?" (Harvard Business Review, January–February 2017). René Descartes's Meditations on First Philosophy. Taiichi Ohno's Toyota Production System: Beyond Large-Scale Production (5 Whys method). Michael Michalko's Thinkertoys: A Handbook of Creative-Thinking Techniques. Go Deeper - 📘 Thinkertoys — Michael Michalko (a brilliant, practical handbook of creative problem-framing techniques) - 📘 The Design of Everyday Things — Don Norman (a classic on human-centered problem framing and design) - 🌐 Farnam Street — First Principles: The Building Blocks of True Knowledge - 🌐 Related piece in this library: Synthesis: The Art of Connecting Disparate Dots For Your Vocabulary First-Principles Thinking (the practice of boiling a problem down to its most fundamental truths and building up from there) · Reasoning by Analogy (reasoning by comparing a problem to existing templates or past conventions) · Problem Framing (the process of defining and structuring the underlying core question before seeking solutions) · 5 Whys Technique (an iterative interrogative technique used to explore the cause-and-effect relationships underlying a problem) · Deconstruction (breaking down a complex system or argument into its constituent fundamental parts) · Root Cause (the fundamental, primary reason for an observed problem or symptom) · Symptom (a superficial manifestation or effect produced by an underlying root cause) · Orthodoxy (authorized, generally accepted, or traditional theory, doctrine, or practice) Step 13. What Is a Good Argument? --------------------------------- Why: What a good argument is. Not a fight — a structure. Learning how arguments actually work is like getting X-ray vision: you start seeing the hidden skeleton under every claim, headline, and sales pitch, and you can tell the sturdy ones from the rotten. In everyday speech an "argument" is a quarrel. In logic it's something calmer and more useful: a set of reasons (premises) offered to support a claim (conclusion). That's it. "You should bring an umbrella (conclusion), because it's going to rain and you hate getting wet (premises)." Once you see the premises-to-conclusion skeleton, you can check whether it actually holds up. The anatomy of an argument Premise 1 — All humans are mortal. Premise 2 — Socrates is a human. ∴ Conclusion — Socrates is mortal. The premises are the support; the conclusion is what they're meant to hold up. A good argument is one where the support genuinely bears the weight. The two questions that test any argument To judge an argument, ask two separate things — and keep them separate, because this is where most confusion lives: 1. Is it valid? (Does the logic work?) If the premises were true, would the conclusion have to follow? This is about structure, not truth. "All cats can fly; Felix is a cat; so Felix can fly" is perfectly valid — the logic is airtight — even though a premise is false. 2. Is it sound? (Are the premises actually true?) An argument is sound only if it's valid and its premises are really true. The flying-cats argument is valid but not sound. Soundness is what you ultimately want. Valid means the logic holds. Sound means the logic holds and the facts are real. Always check both. This split is a powerful diagnostic. When you disagree with a conclusion, you can now pinpoint why: is the reasoning broken (invalid), or is a premise false (unsound)? "I accept your logic, but I reject premise 2" is one of the most useful sentences in clear thinking. Deduction vs. induction Two flavors of reasoning, good for different jobs. Deduction goes from general to specific with certainty (the Socrates example) — if valid and sound, the conclusion is guaranteed. Induction goes from specific observations to a general likelihood ("the sun has risen every day, so it'll rise tomorrow") — it never gives certainty, only strength. Most real-world reasoning, including science, is inductive: not "proven," but well-supported. Demanding deductive certainty from an inductive question is its own mistake. The rot to watch for: logical fallacies Fallacies are arguments that look persuasive but have a broken structure. They work by feeling right. Learn to name them and they lose their spell. A starter gallery: Common fallacies Ad hominemAttacking the person instead of their argument."You can't trust her climate data — she's not even a scientist." (Maybe the data's still right.) Straw manDistorting someone's position into a weaker one, then knocking that down."You want better transit? So you want to ban all cars." (They didn't say that.) False dilemmaPresenting two options as the only ones when more exist."Either we cut this program or we go bankrupt." (There's usually a middle.) Slippery slopeClaiming one small step must lead to an extreme outcome, with no real link."If we allow this, next we'll allow anything." (Says who, and how?) Appeal to authority (misused)Treating "an authority said it" as proof — especially outside their expertise."A famous actor endorses this supplement, so it works." Appeal to emotionSubstituting feelings (fear, pity, outrage) for actual reasons."Think of the children!" — used to dodge the real question. Circular reasoningThe conclusion is smuggled into the premises; it assumes what it's trying to prove."It's true because it says so right here, and it's reliable because it's true." Hasty generalizationDrawing a broad conclusion from too few examples."Two rude tourists — people from that country are rude." Correlation ≠ causationAssuming that because two things occur together, one causes the other."Ice-cream sales and drownings both rise in summer, so ice cream causes drowning." (It's the heat.) Whataboutism / tu quoqueDeflecting criticism by pointing at someone else's fault instead of answering it."How dare you criticize me — what about what they did?" Turning it on yourself The real power move isn't spotting fallacies in your opponents — that's easy and smug. It's catching them in your own thinking, where they hide best. Steel-man the other side (argue their position at its strongest before you disagree), ask what would change your mind, and separate "I don't like this conclusion" from "this argument is actually flawed." Good reasoning is a discipline you practice on yourself first. The takeaway An argument is premises supporting a conclusion. Test it with two separate questions: is it valid (does the logic hold?) and is it sound (are the premises true?). Know that deduction gives certainty and induction gives likelihood, and learn to name the common fallacies — arguments that feel right but are structurally broken. Then apply all of it to your own thinking first. Try this: take a strong claim you saw recently — an ad, a headline, a hot take. Find its premises and conclusion, then ask "valid? sound?" and check it against the fallacy list. With practice this becomes automatic — and the world gets a lot clearer. Sources & Further Reading Scope. An introduction to informal logic and critical reasoning; the field is deep, and some "fallacies" are context-dependent (an appeal to a genuine expert, for instance, can be reasonable). Use these as flags for closer inspection, not automatic disqualifiers. 📘 Thinking, Fast and Slow — Daniel Kahneman (why our reasoning goes wrong; see also The Biases That Fool Everyone) 📘 Being Logical — D.Q. McInerny (a short, clear primer on argument) 🌐 Your Logical Fallacy Is — a friendly illustrated fallacy reference 🌐 Stanford Encyclopedia of Philosophy — Fallacies (rigorous) Step 14. Epistemology: How Do We Know What Is Actually True? ------------------------------------------------------------ Why: Epistemology: how do we know anything? How do you know that you are reading these words right now? You might answer: "Because I can see them with my eyes." But how do you know your eyes aren't deceiving you, or that you aren't dreaming, or that you aren't a brain in a jar stimulated by a supercomputer simulation? While these questions sound like science fiction, they form the heart of Epistemology—the branch of philosophy derived from the Greek words episteme (knowledge) and logos (study). Epistemology investigates what knowledge is, how we acquire it, what justifies our beliefs, and how to distinguish genuine truth from opinion, illusion, or luck. The Classical Definition: Justified True Belief (JTB) Plato's dialogues (the Meno and the Theaetetus) first floated the idea that knowledge is true belief plus an "account" of why it's true, though the Theaetetus ends without accepting it. By the mid-20th century, philosophers commonly analyzed knowledge using a formula known as Justified True Belief (JTB). According to the classical JTB model, you possess knowledge of a proposition (let's say, "It is raining outside") if and only if three strict conditions are met: Truth: The statement must actually correspond to reality (it must actually be raining). You cannot "know" something that is false; you can only mistakenly believe it. Belief: You must personally believe that the statement is true. You cannot "know" a fact if you secretly doubt or deny it. Justification: You must have adequate evidence, reason, or proof for your belief (such as standing outside and feeling raindrops). To understand why justification matters, consider a person who buys a lottery ticket and claims: "I know this ticket is going to win." If the ticket happens to win, did the person "know" it beforehand? Philosophers answer no: it was a lucky guess, not knowledge, because the person lacked adequate justification at the time. Descartes' Radical Doubt: "Cogito, Ergo Sum" In his 1641 masterpiece Meditations on First Philosophy, the French philosopher René Descartes undertook one of the most famous intellectual experiments in history. Disturbed by how many false beliefs he had accepted during his youth, Descartes decided to demolish his entire belief system and start again from an unshakable foundation. To do this, he adopted methodological skepticism: doubting every belief that possessed even the slightest possibility of error: Sensory Illusion: Our senses sometimes deceive us (a stick looks bent in water; distant towers look round when they are square). Therefore, sensory evidence alone cannot guarantee absolute certainty. The Dream Argument: How do you know with 100% certainty that you are awake right now and not experiencing an ultra-vivid dream? The Evil Demon (Malin Génie): Imagine an all-powerful, malicious demon whose sole purpose is to deceive your mind about everything—including geometry, mathematics, and the existence of your physical body. After stripping away all uncertain beliefs, Descartes arrived at one single, indestructible truth that survived even the Evil Demon: In order to be deceived, doubted, or questioned, there must be an "I" that is doing the thinking. Descartes expressed this foundational certainty in the phrase Cogito, ergo sum ("I think, therefore I am"), first written in French in his Discourse on the Method (1637) and in this Latin form in his Principles of Philosophy (1644). Even if everything else is an illusion, your own conscious awareness is undeniably real. You can doubt whether the external world exists, but you cannot doubt that you are currently doubting. Rationalism vs. Empiricism: Reason or Senses? Descartes' work sparked the great epistemological debate of the Enlightenment between two opposing schools of thought: 1. Rationalism (Descartes, Spinoza, Leibniz) Rationalists argued that the primary source of human knowledge is reason, logic, and innate intellect, independent of sensory experience (a priori knowledge). Sensory evidence is messy, subjective, and prone to illusion; true knowledge (like mathematics, geometry, and logical deduction) exists in the mind prior to sensory experience. 2. Empiricism (Locke, Hume, Berkeley) Empiricists rejected innate ideas. John Locke proposed that the human mind is born as a Tabula Rasa (a blank slate). All knowledge, concepts, and ideas are written onto this blank slate through sensory experience and observation (a posteriori knowledge). The Scottish empiricist David Hume took this logic to its radical conclusion: he pointed out that we never actually observe "causality" directly. When one billiard ball hits another, we see Ball A move, and then we see Ball B move. We do not see the invisible force of cause and effect; we merely infer it through habit. Hume demonstrated that even scientific induction relies on an unprovable assumption: that the future will resemble the past. The Gettier Problem and Pragmatic Truth In 1963, an American philosopher named Edmund Gettier published a brief three-page paper that shattered the Justified True Belief model. Gettier presented clever counterexamples showing that a person can have a Justified True Belief that is true purely by luck: Example (a stopped-clock case Bertrand Russell gave in 1948, now the standard way to illustrate Gettier's point; Gettier's own cases involved job candidates and coins): You walk into a room, look at a wall clock that reads 3:00 PM, and form the belief that it is 3:00 PM. Unknown to you, the clock broke exactly 24 hours ago. However, by pure coincidence, the actual time happens to be exactly 3:00 PM when you look. Your belief was true, and you were justified in trusting the clock—yet you did not genuinely "know" the time; you got lucky. Gettier's paper launched a modern race to find a "fourth condition" for knowledge, such as reliability, undefeated justification, or virtue epistemology. Meanwhile, American Pragmatists (William James, Charles Sanders Peirce, John Dewey) offered an alternative approach: instead of hunting for abstract, unassailable certainty, pragmatism defines truth by its practical consequences and working value. A belief is "true" if holding it allows us to navigate reality, solve problems, and predict outcomes successfully. Takeaways Epistemology investigates the foundations, justification, and limits of human knowledge. René Descartes demonstrated through methodological doubt that even if sensory experiences are illusions, conscious awareness itself is undeniable ("Cogito, ergo sum"). Knowledge requires balancing rational logical deduction (Rationalism) with empirical sensory observation (Empiricism), while recognizing that justification must be reliable enough to exclude lucky coincidences. Action step: Choose a claim you read today in the news or social media, and evaluate it through the JTB lens: Is it demonstrably true? Do you believe it? And what specific, reliable evidence justifies that belief? Sources & Further Reading Sources - René Descartes' Meditations on First Philosophy (1641). David Hume's An Enquiry Concerning Human Understanding (1748). Edmund Gettier's paper "Is Justified True Belief Knowledge?" (Analysis, 1963). Bertrand Russell's stopped clock: Human Knowledge: Its Scope and Limits (1948). Plato's Theaetetus 201c–210b. William James's Pragmatism (1907). Go Deeper - 📘 Meditations on First Philosophy — René Descartes (the foundational text of modern Western epistemology) - 📘 Epistemology: A Contemporary Introduction — Robert Audi (a clear, thorough textbook covering modern knowledge theory) - 🌐 Stanford Encyclopedia of Philosophy — Epistemology - 🌐 Related piece in this library: The Biases That Fool Everyone For Your Vocabulary Epistemology (the branch of philosophy that studies the nature, origin, and limits of knowledge) · Justified True Belief (the classical three-part definition of knowledge) · Methodological Doubt (Descartes' systematic process of doubting all uncertain beliefs) · Cogito, ergo sum ("I think, therefore I am") · Rationalism (the view that reason is the primary source of knowledge) · Empiricism (the view that sensory experience is the primary source of knowledge) · Tabula Rasa (the concept of the mind as a blank slate at birth) · Pragmatism (philosophy assessing the truth of beliefs by their practical success) Step 15. How Do We Know Anything? Hume, Popper, Kuhn and the Problem of Science ------------------------------------------------------------------------------- Why: Hume, Popper and Kuhn: how science knows, and where it struggles. Science is the most successful way of knowing humans have ever found, and nobody has managed to say exactly why it works. That gap matters more than ever. Follows: Epistemology: How Do We Know What Is Actually True? and The Allegory of the Cave In this deep dive 1. Hume's bombshell: why the sun rising tomorrow can't be proved 2. Logical positivism: the attempt to make meaning scientific, and why it fell apart 3. Popper: science as bold guesses and failed refutations 4. Kuhn: paradigms, puzzles and revolutions 5. Lakatos and Feyerabend: rescuing method, and abandoning it 6. Bayes: confirmation as degrees of belief 7. The replication crisis: the whole debate, live Part 1 — Hume's bombshell The Allegory of the Cave asked how we could ever tell shadows from reality. Epistemology introduced the classic definition of knowledge as justified true belief, and mentioned in passing that David Hume showed "even scientific induction relies on an unprovable assumption: that the future will resemble the past." That passing mention is one of the most important ideas in the history of thought. This piece starts there, and follows it all the way to a crisis in modern laboratories. Two kinds of knowledge In An Enquiry Concerning Human Understanding (1748, Section IV), Hume divides everything we can know into two kinds. Relations of ideas are truths like those of mathematics: that three times five is half of thirty. You can know them by thought alone, and denying them produces a contradiction. Matters of fact are claims about the world, and they're different. Their opposite is always conceivable. "That the sun will not rise to-morrow," Hume writes, "is no less intelligible a proposition, and implies no more contradiction than the affirmation, that it will rise." So how do we know matters of fact we haven't observed? Through cause and effect, Hume says. And how do we know causes? Only through experience: we've seen bread nourish us, fire burn, one billiard ball move another. The step nobody can justify Here's the bombshell. "The bread, which I formerly eat, nourished me," Hume writes, but "does it follow, that other bread must also nourish me at another time?" There's a gap between I have found that such an object has always been attended with such an effect and I foresee that similar objects will be attended with similar effects. The step isn't a logical deduction, since nothing contradictory follows from denying it. Could it be justified by experience? No, because "all inferences from experience suppose, as their foundation, that the future will resemble the past." Using past experience to prove that past experience is a reliable guide goes in a circle. This is the problem of induction. It first appeared in Hume's Treatise of Human Nature (1739); the Enquiry gives a shorter version. Notice what Hume is not saying. He doesn't say we should stop expecting bread to nourish us. He says that our expectation isn't the product of reasoning at all. Custom, the great guide What produces the expectation, then? Habit. "Custom, then, is the great guide of human life. It is that principle alone which renders our experience useful to us" (Section V). We expect the future to resemble the past because we're the kind of creatures who form such expectations, not because we've proved anything. Bertrand Russell gave the problem its most memorable mascot in The Problems of Philosophy (1912): "The man who has fed the chicken every day throughout its life at last wrings its neck instead, showing that more refined views as to the uniformity of nature would have been useful to the chicken." The chicken's evidence was excellent, right up to the last day. Every scientific law is a bet that the future will behave like the past. Hume showed that the bet can't be proved to be a good one without assuming it already is. A newer riddle In 1955 Nelson Goodman made things worse with a "new riddle of induction." Define a word, "grue": an object is grue if it's green and observed before some future time t, or blue otherwise. Every emerald ever examined is green. But every emerald examined so far is also grue. So the evidence supports "all emeralds are green" and "all emeralds are grue" equally, and the two predict different colours after time t. Why is one a natural projection and the other absurd? Goodman's point: induction isn't just "extend the pattern." We need some account of which patterns are worth extending, and logic alone doesn't supply it. Philosophers have tried many escape routes. Hans Reichenbach offered a "pragmatic vindication": if any method of predicting works, induction will. P. F. Strawson argued that asking for a justification of induction as a whole is a confused demand. None has satisfied everyone. The rest of this piece is, in a sense, the story of how philosophers of science tried to build a theory of scientific knowledge on top of a foundation Hume had cracked. Part 2 — Logical positivism: making meaning scientific The Vienna Circle Between 1924 and 1936, a group of philosophers and scientists met regularly in Vienna under the nominal leadership of Moritz Schlick. They became known as the Vienna Circle, and their 1929 unofficial manifesto, "The Scientific World Conception," was signed by Rudolf Carnap, Hans Hahn and Otto Neurath. Their movement, logical positivism (or logical empiricism), wanted philosophy to be as rigorous as physics. They took a cue from the end of Hume's Enquiry, a passage that still startles. Pick up any book of theology or metaphysics, Hume says, and ask: "Does it contain any abstract reasoning concerning quantity or number? No. Does it contain any experimental reasoning concerning matter of fact and existence? No. Commit it then to the flames: for it can contain nothing but sophistry and illusion." The verification principle The positivists turned Hume's fork into a test of meaning. A statement is meaningful, they said, only if it's either true by definition (like logic and mathematics) or verifiable in principle by experience. "Water boils at 100 °C at sea level" is meaningful. "The Absolute is beyond time" isn't false. It's literally nonsense, because no observation could ever bear on it. A. J. Ayer brought the idea to English readers in Language, Truth and Logic (1936). It was an exhilarating idea. Whole libraries of metaphysics could be cleared away, and philosophy could become the clarifier of science. Kind of statement Example Positivist verdict Analytic (true by meaning) "All bachelors are unmarried." Meaningful, but says nothing about the world Empirically verifiable "This metal expands when heated." Meaningful, a scientific claim Neither "God is love." "Beauty is objective." "The Nothing nothings." Cognitively meaningless Why it fell apart The trouble came quickly, and much of it from inside the Circle itself. Universal laws can't be verified. "All metals expand when heated" covers infinitely many cases, so no finite set of observations can verify it. By the strict criterion, the laws of science would be meaningless. Carnap responded in 1936 by weakening the criterion to mere confirmability, partial testability, and the attempts to state that precisely ran into one technical problem after another. The principle seemed to refute itself. "Only verifiable or analytic statements are meaningful" isn't obviously verifiable or analytic. The SEP's entry on logical empiricism notes this charge "sounds more compelling than it is": Ayer treated the principle as a definition, a proposal about how to use the word "meaning." But then critics could simply decline the proposal. Quine's holism. In "Two Dogmas of Empiricism" (1951), W. V. O. Quine attacked the sharp line between analytic and empirical truths and argued that statements don't face experience one at a time. They face it as a whole web, so no individual sentence has its own neat set of verifying observations. History intervened too. The Circle was scattered by emigration and exile in the 1930s, and Schlick was murdered in 1936. In 1967, the philosopher John Passmore wrote the famous obituary: "Logical positivism, then, is dead, or as dead as a philosophical movement ever becomes." Logical positivism died, but its question didn't: what makes a claim scientific, rather than just confident-sounding? The obituary was partly premature. The SEP's entry on the Vienna Circle notes that while some of its doctrines may be "as dead as philosophies can be," others "show signs of surprising vitality," and that which ones is still debated. Its demand for clarity, its respect for logic and its insistence that claims about the world answer to evidence shaped all the philosophy of science that followed, including its fiercest critics. Part 3 — Popper: bold guesses and failed refutations Four theories in Vienna Karl Popper was born in Vienna in 1902. His lifelong concern with what makes a theory scientific overlapped with the Circle's, though his answer was very different. As a young man around 1919 he encountered four big theories: Marx's theory of history, Freud's psychoanalysis, Alfred Adler's individual psychology, and Einstein's general relativity. What struck him was a difference in attitude. As the SEP summarises it, "The dominance of the critical spirit in Einstein, and its total absence in Marx, Freud and Adler, struck Popper as being of fundamental importance." Marxists and psychoanalysts saw confirmations everywhere. Any behaviour at all could be interpreted to fit their theory. Einstein's theory, in contrast, stuck its neck out. It predicted that light passing near the Sun would be bent by a specific amount, an outcome that, given Newtonian physics, was highly improbable. Eddington's eclipse observations in 1919 tested the prediction, and if it had failed, the theory would have been in serious trouble. Psychoanalytic theories, Popper later wrote in Conjectures and Refutations (1963), were "simply non-testable, irrefutable. There was no conceivable human behaviour which could contradict them." Falsifiability From this, Popper built a new answer to Hume and a new criterion for science. In Logik der Forschung (1935; English as The Logic of Scientific Discovery, 1959) he accepted that Hume was right: induction can't be justified. But, he argued, science doesn't actually need it. The key is a logical asymmetry. No number of white swans proves "all swans are white." But a single black swan refutes it. Verification is impossible; falsification is possible. So science, on Popper's view, works by conjecture and refutation: propose bold hypotheses, deduce risky predictions, and try hard to knock them down. A theory that survives severe tests is corroborated, but never proved. It remains our best guess until something kills it. This gives a demarcation criterion: a theory is scientific if it's falsifiable, if there's some possible observation that would show it false. Popper wasn't saying unfalsifiable ideas are meaningless, as the positivists had. Metaphysics could be meaningful and even inspire science. It just isn't science. He applied the criterion with a political edge. Marxism, he argued, had started out scientific, since it made genuine predictions. When those failed, its defenders added ad hoc hypotheses to save it, and so a scientific theory degenerated into dogma. What Popper got right is hard to overstate. He turned the scientific attitude into something you can check: does this claim forbid anything? He also explained why piling up confirmations is cheap. It is, he argued, easy to obtain evidence in favour of virtually any theory, so confirmations should count only when they come from predictions that might really have failed. A good scientific theory tells you, in advance, what would prove it wrong. The problems Falsifiability is a powerful idea, and philosophers found serious problems with it as a complete account of science: No hypothesis is tested alone. Pierre Duhem had already argued that when a prediction fails, "not only is the questioned proposition put into doubt, but also the whole theoretical scaffolding used by the physicist." A failed prediction tells you that something in your package of theory, auxiliary assumptions and instruments is wrong, not which part. Quine generalised the point. Logic alone never forces you to abandon the theory rather than an assumption. Scientists rightly ignore refutations. For decades, Mercury's orbit didn't quite fit Newtonian mechanics. Astronomers didn't abandon Newton; they looked for unseen bodies and kept working, until Einstein's theory came along. By strict Popperian standards, they should have treated Newton as refuted. Most people think they were right not to. Probabilistic claims can't be falsified by single cases. "Smoking raises the risk of cancer" isn't refuted by a smoker who lives to 95. Corroboration sounds like induction in disguise. If a theory's surviving tests gives us no reason to expect it to keep surviving, why rely on it to build bridges? Critics argue that Popper's view needs a little induction after all. Part 4 — Kuhn: paradigms, puzzles and revolutions A historian looks at science Thomas Kuhn (1922–1996) began in physics, graduating from Harvard in 1943, then switched to the history of science, and eventually to philosophy. In 1965 an international colloquium at Bedford College, London, set up a now-famous confrontation between his ideas and Popper's; the papers, with contributions from Lakatos and Feyerabend, became the book Criticism and the Growth of Knowledge (1970). His The Structure of Scientific Revolutions (1962) is, the SEP says, "one of the most cited academic books of all time," and it gave the world the word paradigm in its modern sense. Kuhn's complaint was that philosophers described science as it ought to be, while history showed something different. Real science, he argued, moves in a cycle: Normal science → Anomalies → Crisis → Revolution → New normal science Normal science. A mature field works within a paradigm: a shared framework of theory, methods, instruments, standards and model examples. The paradigm supplies the puzzles and the tools to solve them. Most scientists, most of the time, aren't testing their foundations at all, Popper notwithstanding. They're solving puzzles the paradigm says are solvable. Anomalies. Some puzzles resist solution. At first they're set aside as problems for later. Crisis. When anomalies pile up, or one becomes impossible to ignore, confidence in the paradigm erodes. Revolution. A rival paradigm emerges, and the field eventually switches. Kuhn compared the switch to a gestalt shift, like the famous drawing that flips from a duck to a rabbit. After it, scientists see the same data differently. Before writing Structure, Kuhn had published The Copernican Revolution (1957), and the move from an Earth-centred to a Sun-centred astronomy is the classic illustration. Incommensurability Kuhn's most controversial claim was that rival paradigms can be incommensurable: they lack a common measure. Their key terms shift in meaning ("mass" in Newton and Einstein are not quite the same concept), they rank problems differently, and their defenders disagree about what counts as a good solution. So a paradigm switch can't be settled by a neutral algorithm. Critics heard this as a claim that science is irrational, or decided by fashion. The SEP notes that Kuhn's rejection of rules for theory choice "appeared tantamount to the claim that science is irrational." Kuhn didn't say science is irrational. He said scientific rationality looks more like trained judgement than like following an algorithm. Kuhn's clarifications Kuhn spent much of his later career saying he'd been misread. The philosopher Margaret Masterman pointed out in 1970 that he had used "paradigm" in a wide variety of senses. In the postscript to the 1970 second edition, Kuhn separated two of them: the disciplinary matrix, the whole constellation of a community's shared commitments, and the exemplar, the concrete model solution that students learn from and imitate. In a 1977 essay he also named five values that scientists share when choosing between theories: accuracy, consistency, scope, simplicity and fruitfulness. Everyone uses them, but they can pull in different directions and be weighted differently, which is why reasonable scientists can disagree during a revolution without anyone being irrational. What Kuhn got right, and what's still argued Kuhn's lasting contribution was to make science's social structure part of its epistemology. Science is done by communities with training, shared exemplars, incentives and institutions. Whether that structure makes science more reliable or less, and under what conditions, is a question you'll meet again in Part 7. His incommensurability thesis remains contested. Most philosophers think rival theories can be compared more than Kuhn's strongest statements allowed, but that theory choice involves judgement, not just logic. Part 5 — Lakatos and Feyerabend: rescuing method, and abandoning it Lakatos: research programmes Imre Lakatos (1922–1974) fled Hungary in 1956 after the Soviet suppression of the uprising. He had been taught by Popper at the London School of Economics, rose to become its Professor of Logic, and produced what the SEP calls a "radical revision" of Popper's demarcation criterion. He wanted to keep Popper's rationality while accepting Kuhn's history. His solution, set out in "Falsification and the Methodology of Scientific Research Programmes" (1970), changed the unit of evaluation from a single theory to a research programme: a sequence of theories sharing a common core. Part of the programme What it is Example (Newtonian mechanics) Hard core Central claims treated as untouchable by decision The laws of motion and gravitation Protective belt Auxiliary hypotheses that get adjusted when tests fail Assumptions about which bodies exist, their masses, measurement errors Negative heuristic The rule: don't aim refutations at the hard core Blame the auxiliary assumptions first When a prediction fails, scientists modify the belt, not the core. That's legitimate, Lakatos said, but there's a test of whether it's going well. A programme is progressive if its modifications predict novel facts that turn out true. It's degenerating if it only patches up failures after the fact. In his words, a research programme "is progressive if it is both theoretically and empirically progressive, and degenerating if it is not." The history of Uranus is a vivid illustration of the progressive kind. Uranus's orbit didn't match Newtonian predictions. Instead of abandoning Newton, the French mathematician Urbain Le Verrier adjusted the belt: he proposed an unseen planet and calculated its position. Johann Galle at the Berlin Observatory found Neptune in 1846 on his first night of searching. The anomaly became a triumph. Mercury's anomaly, by contrast, resisted such patching for decades, and the programme was eventually overtaken by Einstein's. A common objection is that there's no deadline. A degenerating programme might recover, so the method seems better at grading programmes than at telling you when to quit. Feyerabend went further: he argued that Lakatos's methodology either rests on "ungrounded value judgments" about good science or is "epistemological anarchism in disguise." Feyerabend: against method Paul Feyerabend (1924–1994) was Lakatos's friend and sparring partner. They planned a joint book, For and Against Method. After Lakatos died in 1974, Feyerabend's half became Against Method (1975), one of the most provocative books in the field. His argument was historical. Look closely at great episodes of science, he said, and you find that the heroes broke every rule philosophers have proposed. He portrayed Galileo as using rhetoric, propaganda and epistemological tricks to win support for the Sun-centred system, at a time when Aristotle's physics had strong empirical arguments on its side. If Galileo had obeyed the methodologists, science would not have progressed. Hence the notorious slogan, "anything goes." It's widely misread as "all ideas are equally good." Feyerabend's point, as the SEP explains, was narrower: if you insist on a universal methodology that never inhibits progress, the only rule it can contain is the useless one, "anything goes." In other words, every methodological rule has limited validity. Feyerabend did go further in later work. He defended the pursuit of alternative medicine and astrology and attacked the authority of science in public life, which made him a hero to some and a villain to others. Feyerabend's real target wasn't science. It was the belief that science owes its success to one method you could write down. Part 6 — Bayes: confirmation as degrees of belief The idea in one paragraph A different tradition sidesteps the debate over refutation by treating belief as a matter of degree. On the Bayesian view, evidence confirms a hypothesis when it makes the hypothesis more probable than it was before, and how much it does so depends on how much more likely the evidence is if the hypothesis is true than if it's false. This library already teaches the machinery. The Mathematics piece Bayes' Theorem: What a Positive Test Really Means builds it with a 1,000-person calculator, and Bayesian Thinking for Everyday Life, in Thinking & Learning, turns it into a habit. Here the question is what it does for the philosophy of science. It explains what Popper got right Bayes vindicates the power of risky predictions. Suppose a theory starts with a 10% chance of being right. It predicts something that would happen 95% of the time if it's true. Risky prediction: rarely happens otherwise (5%) Safe prediction: usually happens anyway (90%) Out of 1,000 possible worlds, theory true (100) 95 see the predicted result 95 see the predicted result Theory false (900) 45 see it anyway 810 see it anyway Chance the theory is true, after seeing the result 95 ÷ 140 ≈ 68% 95 ÷ 905 ≈ 10% The risky prediction moves the theory from 10% to about 68%. The safe prediction barely moves it at all. That's Popper's insight, recovered without his ban on induction: a theory that "explains everything," whose predictions would come true anyway, gains nothing from its successes. Bayesians argue it also helps with Duhem's problem. When a prediction fails, the probabilities flow back through the whole package, and the part that loses most credibility tends to be the one you were least sure of to begin with. That fits what astronomers did with Uranus: they doubted the assumption that there were no more planets, not Newton. What it doesn't solve The problem of the priors. Where do the starting probabilities come from? "Subjective" Bayesians allow any coherent prior; "objective" Bayesians look for rules that narrow the choice. Subjectivists point to merging-of-opinions theorems, which show that under certain conditions people with different priors should expect their beliefs to converge as shared evidence piles up, though only in the long run. The SEP's entry on Bayesian epistemology gives a telling example. You observe a hundred black ravens. Should your confidence that the next raven is black go up? It depends entirely on your prior distribution. With some priors, credence rises; with others, it stays put or even falls. Hume's problem hasn't been solved; it has moved into the choice of priors. Old evidence. In 1915, Einstein showed that general relativity explained the anomaly in Mercury's orbit, a fact known for over fifty years. That rightly raised confidence in the theory. But on the simplest Bayesian account, evidence you already know can't raise any probability, because you already had it. Clark Glymour pressed this problem in 1980, and it's still debated. The ravens paradox. Carl Hempel noticed in 1945 that "all ravens are black" is logically equivalent to "all non-black things are non-ravens." So a green apple seems to confirm that all ravens are black. Bayesian analyses can separate the cases: given plausible background assumptions (for instance, that the number of ravens doesn't depend on their colour), a black raven confirms the hypothesis and a green apple doesn't. But the result hangs on those assumptions, and the literature on getting them right is still growing. Part 7 — The replication crisis: the whole debate, live What happened In 2011, the Journal of Personality and Social Psychology, one of the field's top journals, published a paper by Daryl Bem reporting experimental evidence for precognition, or ESP. It had passed standard peer review. Three independent attempts to replicate it failed, and the journal that published the original declined to publish the replications, on the grounds that they weren't new research. They eventually appeared in PLoS ONE in 2012. The SEP's entry on reproducibility describes the Bem episode as one common origin story of psychology's "replication crisis." The same year, Joseph Simmons, Leif Nelson and Uri Simonsohn showed in Psychological Science that undisclosed flexibility in collecting and analysing data "dramatically increases actual false-positive rates." In many cases, they wrote, a researcher is more likely to find evidence for an effect that doesn't exist than to correctly find that it doesn't. Then came the large-scale checks: Project What was replicated What happened Amgen (Begley & Ellis, Nature, 2012) 53 "landmark" preclinical cancer studies Findings confirmed in 6 (11%) Reproducibility Project: Psychology (Open Science Collaboration, Science, 2015) 100 studies from three psychology journals 97% of originals were significant; 36% of replications were. Effects were about half as large. Social science in Nature and Science (Camerer et al., 2018) 21 experiments from 2010 to 2015 13 (62%) replicated; effects averaged about 50% of the original Reproducibility Project: Cancer Biology (Errington et al., eLife, 2021) 50 experiments from 23 papers 46% success by their main criterion; median effect size 85% smaller A Nature poll of about 1,500 researchers in 2016 found that more than half (52%) believed science was facing a reproducibility crisis. Every philosopher in this piece saw part of it coming This is where the abstract debate becomes concrete. Each idea above explains part of what went wrong. Hume. Every published result is an inductive leap from a sample to the world. Small samples are short runs of evidence, like the chicken's. The SEP notes that direct replications make up roughly 1% of the published psychology literature, so most leaps were never checked. Popper. Flexibility in analysis is the ad hoc rescue of the twenty-first century. If you can choose, after seeing the data, which outcomes to report, which participants to exclude and when to stop collecting, your hypothesis can't fail. It has become unfalsifiable without anyone deciding to make it so. Kuhn. Science is a social practice with incentives. The SEP documents journals that historically refused to publish replications, and a "publish or perish" culture that rewards novel, positive, surprising results. The community's shared standards, its paradigm of good practice, were part of the problem. Lakatos. A research programme that answers each failed replication by adding a new condition or moderator, without ever predicting anything new that turns out true, fits his definition of degeneration exactly. Bayes. In 2005 John Ioannidis published "Why Most Published Research Findings Are False" in PLoS Medicine. His argument is Bayesian at heart: the chance that a "significant" finding is true depends on how plausible the hypothesis was to begin with, the study's statistical power, and the amount of bias. Here's his logic with illustrative numbers (computed from the formula in Ioannidis's paper, not data from any particular field). Imagine 1,000 hypotheses tested, of which 200 are actually true. Conditions True effects found False positives Share of "significant" results that are true Strong studies (80% power), honest 5% false-positive rate 160 40 80% Weak studies (35% power), honest 5% rate 70 40 64% Weak studies, and analytic flexibility pushing the false-positive rate to 20% 70 160 30% Nothing in the bottom row requires fraud. Small samples and flexible analysis are enough to turn a literature mostly false. The replication crisis wasn't science failing. It was science's own methods, finally turned on itself, finding the failure. What changed The response has been, in a sense, applied philosophy of science. Reformers proposed pre-registration, publicly recording your hypothesis and analysis plan before collecting data, which is falsifiability made enforceable. In 2013 the journal Cortex launched Registered Reports, in which peer review happens before the results exist, and a study that passes gets an in-principle acceptance that can't be revoked because of what it finds. Data sharing, larger samples, and replication studies have become far more common, and new fields of "meta-science" study science itself. Vazire (2018) has called this a "credibility revolution," and in 2018 Nelson, Simmons and Simonsohn argued that experimental psychology's practices "have improved dramatically." A note of caution in both directions. Replication rates vary a lot by field and method, and the projects above sampled particular journals and kinds of study. A failed replication, as the cancer biology team put it, "does not disconfirm a finding, but it does suggest that additional investigation is needed." The crisis is a reason to trust single, surprising studies less. It is not a reason to distrust well-replicated science, such as the effectiveness of vaccines or the physics that keeps planes in the air. So how should you trust science? Putting the whole story together suggests a practical stance: Trust convergence, not single results. One study is a conjecture. Many independent studies, by different teams with different methods, are something close to knowledge. Prefer risky predictions. Ask whether the claim was stated in advance, in a way that could have failed. Weigh prior plausibility. A surprising result in a small study is, by Bayes, more likely to be a false positive. Extraordinary claims need extraordinary evidence. Look for the self-correction. The best evidence that a field is healthy isn't that it's never wrong. It's that it publishes its replications, shares its data and changes its practices when it finds out. The takeaway Hume showed that no amount of past evidence logically guarantees the future, and that our reliance on induction is a habit, not a proof. The logical positivists tried to make meaning itself depend on verification, and collapsed under their own criterion. Popper replaced verification with falsification: science advances by bold conjectures that survive attempts to refute them. Kuhn showed that real science runs on shared paradigms and changes through revolutions that involve judgement, not algorithms. Lakatos judged research programmes by whether they predict new facts, and Feyerabend argued that no single method explains science's success. Bayesian confirmation unites several of these insights but moves Hume's problem into the choice of priors. The replication crisis shows all of them at work: small samples, unfalsifiable flexibility, bad incentives and low priors produced a literature full of false positives, and science's own methods found and began to fix it. Trust science the way it deserves to be trusted: in proportion to replication, risk and convergence. Try this: pick a scientific claim you've seen in the news this month, and ask four questions. Popper: what result would have shown it false, and was that stated in advance? Bayes: how plausible was it before this study? Hume: how big was the sample, and has anyone else found the same thing? Kuhn: who funded or rewarded this result, and what would the field lose if it turned out wrong? Write down your confidence before and after. That's the whole history of the philosophy of science, in ten minutes. Sources & Further Reading Sources - David Hume, An Enquiry Concerning Human Understanding (1748), Sections IV, V and XII (Project Gutenberg); SEP, "The Problem of Induction" (Treatise 1739; Goodman's grue; Reichenbach; Strawson). - Bertrand Russell, The Problems of Philosophy (1912), ch. 6 (Project Gutenberg). - SEP, "Vienna Circle" (dates, manifesto, Schlick's murder, Passmore's "as dead as"); SEP, "Logical Empiricism" (Passmore 1967 quotation; the self-undercutting charge; Quine 1951). - SEP, "Karl Popper" (1919, Eddington, Logik der Forschung 1935, Conjectures and Refutations 1963 quotation, Marxism and ad hoc hypotheses); SEP, "Pierre Duhem" (Duhem's thesis quotation). - SEP, "Thomas Kuhn" (Structure 1962; paradigms; Masterman 1970; 1970 postscript; five values, 1977). - SEP, "Imre Lakatos" (research programmes; hard core and protective belt; Mercury); NASA, "Neptune: Facts" (Le Verrier and Galle, 1846). SEP, "Paul Feyerabend" (Against Method; "anything goes"; Galileo). - SEP, "Bayesian Epistemology" (priors; ravens example; old evidence and Glymour 1980); SEP, "Confirmation" and "Carl Hempel" (ravens paradox). Bayes table computed by the author. - SEP, "Reproducibility of Scientific Results" (Bem 2011; replication about 1% of psychology literature; publication bias; "credibility revolution"). Open Science Collaboration, Science 349 (2015), abstract via PubMed; Begley & Ellis, Nature 483 (2012); Camerer et al., Nature Human Behaviour 2 (2018); Errington et al., eLife (2021); Baker, Nature 533 (2016); Simmons, Nelson & Simonsohn, Psychological Science (2011); Nelson, Simmons & Simonsohn, Annual Review of Psychology (2018); Ioannidis, PLoS Medicine (2005), PPV formula; Chambers, Cortex (2013). Ioannidis table computed by the author from his formula. Go deeper - 📘 What Is This Thing Called Science? — Alan Chalmers. The classic, clear introduction to induction, Popper, Kuhn, Lakatos, Feyerabend and Bayesianism. - 📘 The Structure of Scientific Revolutions — Thomas Kuhn (1962). Short, readable and still provocative. Read the postscript too, where Kuhn answers his critics. - 📘 Conjectures and Refutations — Karl Popper (1963). The first chapter tells the Vienna story of Einstein, Freud and Marx in Popper's own words. - 📘 Science Fictions — Stuart Ritchie (2020). A lively account of the replication crisis, bias, hype and the reform movement, aimed at general readers. - 🌐 Stanford Encyclopedia of Philosophy — The Problem of Induction. Hume's argument and every serious attempt to answer it. - 🌐 Stanford Encyclopedia of Philosophy — Reproducibility of Scientific Results. The philosophy behind Part 7, with full references. For Your Vocabulary Problem of induction (Hume's point that no argument can justify expecting the future to resemble the past without assuming it) · Verification principle (the positivist claim that a statement is meaningful only if analytic or empirically verifiable) · Falsifiability (Popper's test: a scientific theory must rule out some possible observation) · Corroboration (Popper's term for surviving severe tests, short of proof) · Paradigm (Kuhn: a field's shared framework of theory, methods and model solutions) · Incommensurability (Kuhn: rival paradigms lack a fully common measure) · Research programme (Lakatos: a series of theories sharing a hard core, judged by whether they predict novel facts) · Prior probability (how plausible a hypothesis is before the new evidence) · Replication (repeating a study to see if its result holds) · Pre-registration (publicly recording hypotheses and analysis plans before collecting data)