Epidemiology & Evidence-Based Medicine: Out of How Many, Compared With What?

The headline says that a food "raises your risk of cancer by 20%". Should you worry? It depends on a number the headline left out: 20% of what? If the usual risk is 5 in 100, a 20% rise is 6 in 100. If the usual risk is 1 in 10,000, it is 1.2 in 10,000, which is two people in a hundred thousand more. Both sentences are true. Only one of them should change your dinner.

Epidemiology is the science of counting disease in populations and asking what causes it. Evidence-based medicine takes those methods to the bedside: before a treatment is used, ask who tested it, against what, and how big the effect really was. This explains how medicine works and is not medical advice. For a health question, see a clinician; in an emergency, call your local emergency number.

Rates and risks: always ask "out of how many?"

Four numbers carry most of public health. Prevalence is how many people have a condition now. Incidence is how many new cases appear in a period among people who could get it. Risk is the chance that one person gets the outcome over a stated time. A rate divides cases by person-time, so a person followed for two years counts twice as much as one followed for one. And odds are risk ÷ (1 − risk), the same information in a different form that behaves better in some statistics.

Prevalence and incidence are linked: in a steady state, prevalence ≈ incidence × average duration. A condition that is rare to catch but lasts forty years, such as some chronic diseases, can be common in the population; a flu that is caught by many but lasts a week can be rare on any given day. Every comparison starts with a denominator. A city with more cancer deaths than a village may simply have more people, or more older people. The reason epidemiologists "age-standardise" death rates is to stop an old country looking sicker only because it is old.

Study designs: from the case report to the randomised trial

Designs form a ladder of how well they rule out other explanations. A case report tells one story. A case-control study starts from people with a disease and picks similar people without it, then looks back at what they were exposed to: cheap and quick for rare diseases. In 1950 Richard Doll and Austin Bradford Hill used it in the British Medical Journal to show that people with lung cancer were far more likely to have smoked. A cohort study follows groups with and without an exposure forward in time; their 1951-onward study of British doctors added the evidence that smokers died younger. A randomised controlled trial (RCT) assigns the exposure by chance, so groups differ only by luck, which makes it the strongest design for testing a treatment. The first published RCT in medicine is usually taken to be the Medical Research Council's trial of streptomycin for tuberculosis in 1948, with Hill as its architect.

Each design gives a different number. A cohort gives a relative risk (RR: risk in the exposed ÷ risk in the unexposed). A case-control study cannot see risks, because the investigators chose how many cases and controls to include, so it gives an odds ratio (OR). For rare diseases the OR is almost the same as the RR. For common ones it overstates it:

At a baseline of 1% and a true RR of 2, the OR is 2.02: close enough that nobody would notice. At a baseline of 30%, the same RR of 2 gives an OR of 3.5. When a newspaper reports an odds ratio as if it were "times more likely", common outcomes are exaggerated. Rare outcomes are not.

Confounding and bias: why the obvious comparison lies

A confounder is a third thing that is linked to both the exposure and the outcome, and creates a link between them that is not causal. Coffee drinkers get more lung cancer? In some old data, yes, because more coffee drinkers also smoked. Hormone therapy for menopausal women looked, in observational studies, as if it protected the heart, partly because women who chose it tended to be healthier and richer already (the "healthy user" effect). A randomised trial breaks the link by assigning the exposure by chance. Without one, the main defence is to compare like with like (stratify or adjust), which only works for confounders you thought to measure.

Bias is different: a systematic error built into the way people were chosen or measured. Selection bias (who ended up in the study), recall bias (people with a disease remember their past differently), and lead-time bias (below) are three. Luck is a third source of error again, and shrinks as samples grow; bias and confounding do not.

With 40% of drinkers and 15% of non-drinkers smoking, drinkers have a 4.6% risk against 2.35% for non-drinkers: a crude relative risk of about 1.96, from a drink that does nothing. Split into smokers and non-smokers, the risk ratio inside each group is exactly 1. Set both sliders equal and the false effect vanishes. That is what a graduate sees in every "X is linked to Y" headline: what else travels with X?

Absolute and relative risk, and the number needed to treat

A treatment that "halves the risk" sounds great, but what it does for you is the absolute risk reduction (ARR): the baseline risk times the relative reduction. Its reciprocal is the number needed to treat (NNT): how many people must be treated for one to benefit.

The same 50% cut helps 10 in 1,000 people when the baseline is 2% (NNT 100), and 100 in 1,000 when it is 20% (NNT 10). A real example of the other side: the Women's Health Initiative stopped its oestrogen-plus-progestin arm in July 2002 because harms exceeded benefits. A hazard ratio of 1.24 for breast cancer sounds large, but it meant about 8 extra cases per 10,000 women per year (in the long-run analysis), while pulmonary embolism rose with a ratio of 2.13, about 10 extra per 10,000, and colorectal cancer fell with a ratio of 0.56, about 7 fewer. The trial overturned the earlier observational impression of heart protection. Later analyses found that the picture varies with age and with how soon after menopause treatment starts; the decision today is individual and belongs in a conversation with a clinician.

Screening and its harms

Looking for disease in people with no symptoms sounds always good. The 1968 WHO principles by Wilson and Jungner set out the conditions: an important condition, a recognisable early stage, a treatment that works better early, an acceptable test. Screening also has costs. False positives mean worry, further tests and sometimes needless procedures. Overdiagnosis finds cancers that would never have caused harm in the person's lifetime, which cannot be known for any one patient. And the numbers behave unintuitively because of prevalence:

At 1% prevalence, 90% sensitivity and 95% specificity, a screened thousand has 9 true positives and about 50 false positives: only about 15% of positive results are real. Drop prevalence to 0.1% and it is under 2%. A positive result in a low-risk population is mostly a false alarm, which is the same Bayes logic as in 12/17, and why screening is offered to groups where the condition is common enough and not to everyone for everything.

A second trap is lead-time bias. Screening finds disease earlier; survival is counted from diagnosis; so people appear to live longer even when death comes at the same time. Take ten invented patients who die 1, 2, 3, 3.5, 4, 4.5, 6, 8, 10 and 12 years after the point at which symptoms would have led to diagnosis. Without screening 4 of 10 (40%) survive five years from diagnosis. Detect everyone two years earlier and the same deaths give 8 of 10 (80%) without a single death delayed. The honest outcome for a screening test is deaths from the disease in a randomised comparison, not "five-year survival".

Systematic reviews and Cochrane: counting all the trials

One trial is a data point. Many trials, found and combined by explicit rules, make a systematic review; if their results are pooled numerically, a meta-analysis. The idea, pushed by the British epidemiologist Archie Cochrane (his 1972 book Effectiveness and Efficiency argued that health services should rest on randomised evidence), led to the Cochrane Collaboration, formed in 1993 under Iain Chalmers. Cochrane reviews follow written protocols that say in advance which trials count. Critics of the movement note unpublished studies, slow updates and many reviews ending in "inconclusive".

The simplest pooling gives each trial a weight of 1 ÷ its variance, so large, precise trials count most. A forest plot shows each trial's estimate and interval with a diamond for the pool. The pool is only as good as the trials found: if small studies with exciting results are published and small null ones are filed away (publication bias), the pool is rosy.

Pooled together the six trials give a risk ratio of about 0.85 (95% interval 0.76 to 0.96). Keep only the two large ones and it is 0.90, an interval of 0.79 to 1.03 that includes no effect. Keep only the three small trials that were "positive" (A, B and F) and the answer looks like a spectacular 0.62. Same drug, three answers, depending on which studies you let in. That is why reviewers search for unpublished work and register their protocols.

Landmark trials and failures

Medicine's history is a record of claims that felt true until someone counted.

Year	Event	The lesson
1747	James Lind compares remedies for scurvy on a ship; the citrus pair recovers	A fair comparison can be small and still clear; the lesson was ignored for decades
1847	Ignaz Semmelweis, Vienna: chlorinated-lime handwashing; deaths in the First Clinic fell from about 18% in April to about 2% in June	A before-and-after comparison with a dramatic effect, but no germ theory behind it, and not published by him until 1858–61. The tale that he was ignored out of pure stubbornness is simpler than the history
1948	MRC streptomycin trial in tuberculosis	The first published RCT in medicine: allocation by chance, not by the doctor's hunch
1950	Doll and Hill, smoking and lung cancer	A case-control study, then a cohort, each confirming the other
1957–62	Thalidomide, sold from 1957 in West Germany, withdrawn in Europe in November 1961 after birth defects in an estimated 10,000 to 20,000 children	Led to the 1962 Kefauver-Harris amendments in the US: drugs must show efficacy and safety
1989–91	CAST: drugs that suppressed abnormal heartbeats after heart attack	The drugs "worked" on the surrogate. In the trial 7.7% of the drug group died against 3.0% on placebo. A surrogate is not the outcome
2002	Women's Health Initiative stopped early	Observational studies had suggested a benefit that a randomised trial did not confirm

The Tuskegee study (1932–72) is a failure of ethics rather than design; see Medical Ethics & Law (35/60). Evidence-based medicine itself has critics, who say that average trial results do not always apply to the individual in the room, that trial populations differ from real patients, and that industry funding can shape which questions get asked. Its defenders reply that the answer is to improve the evidence and judgment, not abandon either.

What a graduate just sees

• Every percentage has a denominator; ask "out of how many?" before anything else.
• Relative changes sound large; absolute changes tell you what happens to a person.
• "Linked to" is a prompt to ask what else travels with the exposure.
• Only random assignment balances confounders you did not measure.
• An odds ratio is not a relative risk when the outcome is common.
• A positive result for a rare condition is more likely a false alarm than a case.
• Five-year survival can rise with no benefit; look for deaths in a randomised comparison.
• One trial is a data point; a review is only as good as the studies it finds.
• A good surrogate marker can still lead you wrong; check the outcome that matters.

Where it shows up next

• Clinical Reasoning & Diagnosis (35/59): sensitivity, specificity and likelihood ratios are the maths of a test in a clinic.
• Public & Global Health (35/61): vaccination programmes, screening policy and disease burden rest on these methods.
• Immunology (35/57) and Pharmacology (35/56): the vaccine and peanut trials, and every drug approval.
• Medical Ethics & Law (35/60): consent, equipoise and when a trial may stop early.
• In the world: health technology agencies (NICE in England, IQWiG in Germany and others), clinical guidelines, drug regulators, health journalism.

If you want to go deeper

• 📘 Textbook. Fletcher, Fletcher and Fletcher, Clinical Epidemiology: The Essentials (Wolters Kluwer). A short, clinical introduction used by many medical schools.
• 📘 Textbook. Trisha Greenhalgh, How to Read a Paper: The Basics of Evidence-Based Medicine and Healthcare (Wiley). The standard guide to appraising a published study.
• 📘 Gentler. Imogen Evans, Hazel Thornton, Iain Chalmers and Paul Glasziou, Testing Treatments (2nd edition 2011, foreword by Ben Goldacre), free to download at testingtreatments.org. Also Gerd Gigerenzer, Calculated Risks (2002).
• 🌐 Free course. Cochrane Interactive Learning: Module 1, introduction to conducting systematic reviews, is free with a login. The later modules need a subscription.