A machinist turns a hundred steel shafts to "10 millimetres". Measure them with a good micrometer and no two are the same: 9.981, 10.012, 10.004. Nothing is wrong. That scatter is a fact of every process, every material and every sensor.
Now stack five of those parts end to end in a gearbox. Will the stack still fit the housing? A pollster calls a thousand people and says 52% will vote yes, give or take three. A drug trial says "significant". A forecast says 30% chance of rain. Every one of these is the same question: given scatter, how much should I trust this number?
A graduate of probability and statistics doesn't see a single number any more. They see the shape of the scatter behind it. This piece builds that sight with eight things you can run: dice, tolerances, polls, a drug-trial trap, a belief that updates, and a gearbox stack.
Who takes it: almost every engineering and science student, usually in the second year; computer scientists meet it again before machine learning.
What it's for: deciding what to believe, and how far to trust a measurement, when outcomes are uncertain.
Comes after: Calculus I (34/01), which helps with continuous distributions, though the ideas here need little more than arithmetic.
Leads to: Machine Learning (34/32) needs it directly. It also shows up in Materials Science & Manufacturing (34/13: process control and defects), Systems Engineering (34/22: reliability) and Computer Security (34/33: randomness and guessing).
Honest line: you won't be able to design a clinical trial after this. You will see why "significant" doesn't mean "important", why a bigger sample beats a cleverer guess, and why most real quantities look like a bell curve. For easier first reads, see the library's Probability: How to Not Fool Yourself (12/03), Statistics for Reading the World (12/05) and Bayes' Theorem (12/17); this piece goes further into the engineering side.
Random variables: a shape with a centre and a spread
A random variable is a number that comes out of a process you can't predict one run at a time. The key move is to stop asking "what will it be?" and ask "what is the shape of what it could be?" Roll two dice and add them: 7 is six times as likely as 2 or 12, so the shape is a tent. Roll many times and the tent appears.
Without JavaScript: two dice add to 7 with probability 6/36 and to 2 or 12 with probability 1/36 each. The mean total is 7 and the standard deviation is about 2.42.
Two numbers summarise a shape. The mean is its balance point: 7 for two dice. The standard deviation (the "spread") says how far values typically stray from the mean: about 2.4 for two dice, which means totals usually land between roughly 5 and 9. The variance is the standard deviation squared, and it is the one that adds up neatly when you combine independent things.
Now the shape that matters most. Measured quantities (shaft diameters, heights, sensor noise) tend to form the normal curve, the bell. Its great convenience is a fixed rule: about 68.3% of values fall within one standard deviation of the mean, 95.4% within two, and 99.7% within three. Engineers use it every day. Say the shaft's spec is 10.00 mm ± 0.05 mm. Slide the process around and see what share of parts get thrown away.
Without JavaScript: a centred process with standard deviation 0.025 mm (the spec limits are exactly ±2 standard deviations) scraps about 4.55% of parts. Cut the spread to 0.01 mm and about 0.00006% are scrapped. Shift the average 0.02 mm off-centre with spread 0.02 mm and about 6.7% are scrapped.
Two lessons graduates keep. First, spread, not average, decides quality: a process centred perfectly but wide still scraps parts. Second, being off-centre costs a lot even when the spread is fine. That is why factories track both.
The central limit theorem: why the bell keeps appearing
Why should so many things be bell-shaped? The answer is one of the most useful theorems in mathematics. Take almost any random process, however lopsided, and average a few dozen independent draws. The averages pile up in a bell, centred on the true mean, with spread shrinking like 1/√n. Early forms are due to de Moivre in 1733 (for coin flips) and Laplace, whose general version dates from about 1810. Pick a lopsided source and drag the sample size.
Without JavaScript: for the lopsided source (mean 1, standard deviation 1) the average of n = 30 draws has standard deviation 1/√30 = 0.183, and the 3,000 averages form a close-to-normal bell around 1.
At n = 1 you see the raw source. By n = 30 even the lopsided one is a near-bell. The prediction (the curve) uses only two numbers from the source, its mean and its standard deviation. The shape of the source is forgotten. That is why a measurement made of many small independent errors is normal, and why so much of statistics can pretend things are bell-shaped.
In words: the average of n draws wobbles less than one draw, by the square root of n. To halve the wobble you need four times the data, not twice. It is the quietest expensive fact in all of experimental science.
Sampling and confidence intervals: how sure is a poll?
You can't ask everyone, so you ask n people and treat the share saying yes as an estimate. How far off might it be? The standard error from the last section gives the answer. For a share p the wobble is √(p(1−p)/n), and a 95% confidence interval is the estimate plus and minus about two of those. For n = 1,000 and a share near half, that is ±3.1 percentage points: the familiar "margin of error".
The wording is subtle, and the interactive makes it concrete. "95% confidence" is a statement about the method, not about one interval: if you ran the procedure again and again, about 95 intervals in 100 would capture the true value. Here the truth is secretly 52%. Run 100 polls.
Without JavaScript: with 1,000 people per poll the margin of error is about 3.1 points; run 100 polls and about 95 of the intervals cover the true value. With 4,000 people the margin halves to about 1.5 points.
Watch two things. Roughly five in a hundred intervals miss the truth even when everything is done correctly. And quadrupling the sample from 1,000 to 4,000 only halves the interval. A poll of a country and a poll of a town need the same sample size for the same precision, as long as the sample is random. The real danger is not the margin of error; it is a sample that isn't random at all, which no formula fixes.
P-values and Bayes: two ways to weigh evidence
A hypothesis test asks whether data are surprising if nothing is going on. The p-value is the probability, assuming nothing is going on, of seeing a result at least as extreme as yours. Flip a coin 100 times and get 60 heads: for a fair coin the chance of a split at least that lopsided in either direction is about 5.7%. It is not the probability that the coin is fair, and it is not the probability that the result was "due to chance". The American Statistical Association issued a rare formal statement on this in 2016, with six principles. Two of them: p-values do not measure the probability that a hypothesis is true, and a p-value does not measure the size or importance of an effect.
The classic trap is testing many things. At the usual cut-off of 0.05, a test of a true "nothing" comes out "significant" one time in 20 by luck. Test twenty unrelated things and the chance that at least one lights up is 1 − 0.9520, about 64%. Try it: each press tests twenty effects that are, by construction, nothing.
Without JavaScript: twenty tests of pure noise at the 0.05 level give at least one false "discovery" about 64% of the time, and one in 20 tests on average.
If you report only the one that lit up, you have a headline and no finding. This is part of why many published results fail to repeat: in the Open Science Collaboration's 2015 attempt to replicate 100 psychology studies, 97% of the originals had statistically significant results but only 36% of the replications did.
Bayes' rule answers a different question: given the data, how should my belief move? Start with a prior belief, multiply by how well each possibility explains the data, and renormalise. Say you test parts and want the defect rate. Know nothing at first (any rate from 0% to 100% equally likely) and add parts one at a time.
Without JavaScript: after 20 good parts and no defects, a flat starting belief gives a most-plausible defect rate near 0%, an average of 4.5%, and a 90% credible range of roughly a quarter of a percent to 13%. Twenty clean parts cannot rule out a defect rate near one in eight.
Frequentists and Bayesians disagree about the philosophy: one treats probability as long-run frequency and never assigns it to a hypothesis, the other as degrees of belief that data update. In practice both are used, often on the same project, and on big data sets they usually agree. The Bayesian version has one practical advantage here: after twenty clean parts it honestly says "we still can't rule out one in eight".
Correlation, regression and the trap of causation
Correlation (called r) measures how tightly two quantities move together along a straight line, from −1 to +1. Regression fits the best line through the cloud (by least squares, as in Linear Algebra, 34/04). Both are summaries, and any summary can hide the story. In 1973 the statistician Francis Anscombe made four data sets with, to two decimals, the same mean, the same spread, the same r (0.82) and the same line (y = 3 + 0.5x). Look at each.
Without JavaScript: all four Anscombe sets have mean x = 9, mean y = 7.50, correlation 0.816 and line y = 3.00 + 0.50x. Set 1 is a noisy line, set 2 a curve, set 3 a line with one outlier, and set 4 a single far point that creates the whole slope.
The numbers agree and the stories do not: one noisy line, one curve, one line with a rogue point, one vertical stack held up by a single point. Always plot first.
The other trap is the famous one. Ice-cream sales and drownings rise together, because summer heat drives both. A confounder (a hidden third factor) makes two things move together without either causing the other. The remedy graduates know is randomised assignment: if a coin decides who gets the treatment, nothing else can be systematically linked to it. Where you can't randomise, causal claims need much more than a line through a cloud.
Engineering: tolerance stacks, reliability and Monte Carlo
Five parts, each 10 mm ± 0.1 mm, line up in an assembly. What is the worst the total can be out? Worst-case stacking adds the tolerances: ±0.5 mm. It is safe and expensive, because it assumes every part is at its extreme in the same direction at once. If the parts vary independently, variances add, so the standard deviations add in quadrature: root-sum-square, √5 × 0.1 = ±0.224 mm. For five parts that is under half the worst-case figure.
When the formulas get hard, engineers use Monte Carlo: let the computer build the assembly ten thousand times with random parts and count how often it fails. Here each part's tolerance is treated as three standard deviations. Choose how many parts are stacked.
Without JavaScript: five parts at ±0.1 mm (three standard deviations) give a worst-case limit of ±0.5 mm, a root-sum-square limit of ±0.224 mm, and in simulation only about 0.3% of assemblies exceed the root-sum-square limit; none come close to 0.5 mm.
The honest caveat is that root-sum-square assumes the parts vary independently and are centred. A tool wearing in the same direction on every part breaks that assumption, and so does one supplier's batch. Engineers disagree about when to trust statistical stacking over worst-case; safety-critical designs often insist on worst-case.
Reliability follows the same arithmetic. A system that needs all of its parts is a series system, and probabilities multiply: ten components that each work 99% of the time give 0.9910 ≈ 90.4% for the whole. Add a backup in parallel and failure needs both to fail: two units that each work 90% of the time give 1 − 0.1² = 99%. The two rules drive why aircraft duplicate systems and why complicated machines need very reliable components.
What a graduate just sees
- Every number has a spread behind it; the first question is "how scattered?", not "what is it?".
- Spread, not the average, decides how many parts are scrapped, and centring matters too.
- Averages of many independent things are bell-shaped, whatever the source looked like.
- Halving the uncertainty costs four times the data.
- A 95% interval is a claim about a method: it misses one time in twenty by design.
- A p-value is "how surprising is this if nothing is going on", not "how likely is it true".
- Test twenty things and one will look significant; ask how many were tried.
- Plot the data; correlation is not causation, and only randomising breaks a confounder.
- Independent errors add in quadrature; series reliabilities multiply.
Where it shows up next
- Machine Learning (34/32): training is fitting a model to noisy data; ideas of overfitting, error bars and Bayes are this course made bigger.
- Materials Science & Manufacturing (34/13): process capability and scrap rates are the normal curve against a spec limit.
- Mechanics of Materials (34/11) and Machine Design (34/19): safety factors exist because loads and strengths are distributions, not numbers.
- Systems Engineering (34/22): reliability budgets multiply series probabilities and add redundancy to push them back up.
- Computer Security (34/33) and Algorithms (34/24): good random numbers, randomised algorithms, and the odds of guessing a secret.
If you want to go deeper
- 📘 Douglas Montgomery and George Runger, Applied Statistics and Probability for Engineers (Wiley; 7th edition). The standard engineering course text, with plenty of tolerance, quality and reliability problems.
- 📘 Joseph Blitzstein and Jessica Hwang, Introduction to Probability (CRC Press; 2nd edition, 2019). Best for deep intuition about probability itself, from Harvard's Stat 110.
- 📖 David Spiegelhalter, The Art of Statistics (2019): the most readable route for learning alone, built around real cases and honest about uncertainty.
- 🌐 MIT OpenCourseWare 18.05, Introduction to Probability and Statistics (Spring 2022): free notes and problem sets, with Bayesian and classical statistics side by side.
Nothing you measure is exact, so every number is a shape. Summarise a shape by its centre and its spread, trust averages more as the square root of the sample, read a confidence interval as a promise about a method, treat a p-value as a measure of surprise, and plot before you believe a correlation.
Try this: next time you see a poll or a "significant" headline, find the sample size and ask how many things were tested. Then, in the dice widget above, add four dice and watch the tent turn into a bell.
Sources. The normal-curve rule (68.3%, 95.4%, 99.7%), the central limit theorem, confidence intervals, the binomial test, the Beta posterior, root-sum-square stacking and series/parallel reliability are standard textbook results (Montgomery and Runger; MIT 18.05); every number shown was recomputed in code. Wasserstein and Lazar, "The ASA's Statement on p-Values: Context, Process, and Purpose", The American Statistician (2016). Open Science Collaboration, "Estimating the reproducibility of psychological science", Science (2015): 97% of originals significant against 36% of replications. Anscombe, "Graphs in Statistical Analysis", The American Statistician (1973). Early forms of the central limit theorem: de Moivre (1733) and Laplace (c. 1810), as surveyed in recent histories such as Ulyanov, "From classical to modern central limit theorems" (arXiv 2405.19828).
Go deeper. The books and course above; Wasserstein and Lazar's statement is free and short (doi.org/10.1080/00031305.2016.1154108).