Electrical Engineering A path through The Default Library ======================================== How electricity is made, moved, shaped and turned into information: from the grid to the chip to the radio in your pocket. Step 1. Calculus I: Slopes, Totals and the Bridge Between Them -------------------------------------------------------------- Why: Every signal is a function of time, and every circuit law is written with derivatives. Your car's speedometer is a derivative. Its odometer is an integral. Everything a first calculus course teaches is a way of moving between the two. Look at a dashboard. The odometer says how far you have gone in total. The speedometer says how fast the total is changing right now. Press the accelerator and the speedometer moves first; the odometer follows, a little behind, like a patient accountant adding up what the speedometer reported. Nobody gave the speedometer a ruler or a clock to look at the whole trip. It reads one instant. Yet it reports a rate, which is a statement about change, from a single moment. How can a single moment say anything about change? That puzzle is what the course exists to answer, and the answer turned out to run most of modern engineering and computing. This is the second time the library has met calculus. Calculus: Derivatives, Integrals, and the Language of Motion (12/11) is the easier first read, with the tangent line and the rectangle sum. Here we go a step further: what the course trains you to see, how a computer gets it wrong, and where the idea is used afterwards. The course in one box Who takes it: first-year students in Mechanical & Aerospace Engineering and in Computer Science, and almost every other science or engineering degree. What it is for: turning "how much?" into "how fast?" and back, so that change can be calculated instead of guessed. Comes after: nothing in this series. Algebra and a little trigonometry are all it assumes. Leads to: Multivariable Calculus (34/02), Differential Equations (34/03), Probability & Statistics (34/05), Physics I: Mechanics (34/06), Thermodynamics (34/14), and from them nearly everything else on both majors. Honest line: you will not be able to differentiate a hard formula by hand after this. You will see why a flat spot on a curve is where the best answer hides, and why the area under a rate is a total. A limit is where a calculation settles The speedometer puzzle is this. Speed is distance divided by time, but in an instant no time passes and no distance is covered, so you seem to be dividing zero by zero. The way out is to stop asking what happens at zero and ask what a sequence of better and better answers is settling towards. Measure over one second, then a tenth, then a hundredth, and watch the average speed home in on a number. That number is the limit. speed now = limit of (distance change ÷ time change) as the time change shrinks to 0 A graduate does not carry the formal definition around, but carries the habit: if a quantity looks like 0 ÷ 0, ask what it is approaching. Take sin(x) ÷ x as x goes to zero, with x in radians. At x = 0.1 it is 0.99833; at 0.01, 0.99998; at 0.001, 0.9999998. It never "is" 1 at zero, since it is undefined there, but it settles on 1. That one limit is why the derivative of sine turns out to be cosine. Interactive · Shrink the gap until the computer gives up We estimate the slope of ex at x = 1 by (e1+h − e) ÷ h. The true slope is e = 2.718281828459045. Make the gap h smaller. Gap h = 10−k The error falls as the gap shrinks, as the limit promises, until about h = 10−8, where it bottoms out near 7 × 10−9. Shrink further and it gets worse: the two values being subtracted agree in almost all of their roughly 16 stored digits, so the subtraction wipes out the information. At 10−16 the answer is exactly 0. This is a computer-science lesson hiding in a calculus course: the limit is a mathematical idea, and a machine can only approach it so far. The derivative is a slope, and a slope is a rate Zoom in on any smooth curve and it straightens out. The steepness of that straight piece is the derivative. In plain words: how much the output moves per unit nudge of the input. If the curve is distance against time, the derivative is speed. If it is temperature against position, it is how sharply it warms as you walk. Rate and slope are one thing seen two ways. f′(x) = limit of [f(x+h) − f(x)] ÷ h as h → 0 The graduate's real asset is not computing derivatives but reading them. Where the curve is rising, the derivative is positive. Where it is flat, the derivative is zero. Where the curve is steepest, the derivative is at its peak. Try it: the lower graph is the derivative of the upper one, drawn point by point. Interactive · A curve and its slope, side by side x³ − 3x sin x bell: e^(−x²) Position x Check the three curves. Every time the top curve peaks or dips, the bottom one crosses zero. Every time the top is climbing, the bottom is above the axis. For x³ − 3x the slope is 3x² − 3, which is zero at x = −1 and x = 1: the hill top and the valley floor. The chain rule: rates multiply down a chain Rates often come in chains. A bicycle's pedals turn the chainring, the chain turns the rear wheel, the wheel moves the bike. If the wheel turns 3 times per pedal turn, and the bike moves 2 metres per wheel turn, the bike moves 6 metres per pedal turn. You multiply the ratios. That is the chain rule, and it is the most used rule in the course. d(outer)/d(input) = d(outer)/d(middle) × d(middle)/d(input) The classic worked case is a balloon. Its volume depends on its radius, and the radius depends on time. At radius r the surface has area 4πr², so one extra centimetre of radius adds a shell of volume about 4πr² cm³: that is dV/dr. Multiply by how fast the radius is growing and you have how fast the volume is growing. At r = 10 cm, growing 2 cm each second, volume grows at 4π × 100 × 2 ≈ 2,513 cm³ per second. Pull the sliders and confirm it. Interactive · Inflate a balloon Radius now Radius is growing at The picture shows why it works: the shaded shell is what one second adds, a skin of area 4πr² and thickness equal to the radius growth. The same rule runs inside every neural network, where the "chain" is dozens of layers long and the multiplication is called backpropagation (Machine Learning, 34/32; Multivariable Calculus, 34/02). Where the slope is zero, something is best At the top of a hill you stop going up and have not yet started going down. The slope is zero. So to find the biggest or smallest value of anything smooth (strongest, cheapest, lightest, fastest) look for where its derivative is zero. That is optimisation, and it is why engineers love the derivative. Take a 30 cm by 20 cm sheet of card. Cut a square of side x from each corner and fold up the sides to make an open box. Its volume is V = x(30 − 2x)(20 − 2x). Cut too little and the box is shallow. Cut too much and it is narrow. In between there is a best x, and the slope tells you where: at x ≈ 3.92 cm the volume peaks at about 1,056 cm³. Interactive · The best box Corner cut x Jump to the flat spot Notice what the slope does as you drag. To the left of the peak the slope is positive, so a bigger cut helps. To the right it is negative, so a smaller cut helps. At the peak it is zero and nothing nearby is better. Gradient descent, the algorithm that trains machine-learning models, is exactly this reasoning repeated: look at the slope, step downhill, repeat. The integral adds up a rate into a total The odometer is the speedometer added up over time. Draw speed against time; the area between the curve and the time axis is distance. That is the integral. Anywhere a rate is known and a total is wanted (litres per minute into a tank, watts into a battery, newtons over a distance) the answer is an area. Try it before you read on. A car accelerates steadily from 0 to 20 m/s in 10 seconds, cruises at 20 m/s for 30 seconds, then brakes steadily to a stop in 10 seconds. How far has it gone? Interactive · Predict, then reveal 600 m 800 m 1,000 m 1,200 m Choose an answer. The shape is two triangles and a rectangle: ½ × 10 × 20 + 30 × 20 + ½ × 10 × 20 = 100 + 600 + 100 = 800 m. The common wrong guess, 1,000 m, uses the top speed for all 50 seconds and forgets that the car spends time slower than that. The Fundamental Theorem: two jobs, one idea Here is the surprise that makes the course a single subject. Build a running total from a rate: the amount in the tank after time T. Then ask how fast that total is growing at T. The answer is just the rate at T. Accumulating and differentiating undo each other, the way the odometer and speedometer do. Newton and Leibniz, working separately in the late 1600s, each saw this. if A(T) = area under the rate up to T, then A′(T) = the rate at T Its practical power is that you can find a total without adding anything: find a function whose slope is your rate and subtract its values at the two ends. Below, water flows into a tank at a rate that wobbles: 4 + 3 sin(t/2) litres per minute. Drag the time and watch the right-hand graph, the tank's total, steepen exactly when the rate on the left is high. Interactive · The tank and its rate Time T The readout checks the theorem two ways: area by adding 600 thin slices, and the exact formula 4T + 6(1 − cos(T/2)). At T = 12 minutes both give 48.24 litres. The slope of the total at that moment is 4 + 3 sin 6 ≈ 3.16 litres per minute, the rate at that moment. Every smooth thing is nearly a straight line The last habit is the most useful in daily engineering. Close to any point, a smooth curve is almost its tangent line, so you can replace it with a straight line and be nearly right. The small-angle rule sin x ≈ x is why a pendulum swings at a steady rhythm. The rule √(1 + x) ≈ 1 + x/2 gives mental square roots: √1.1 ≈ 1.05, true value 1.0488. f(a + x) ≈ f(a) + f′(a) · x   (good when x is small) How small is small? The error shrinks faster than the step. Halve the step and the error falls about four times for √(1 + x) and ex, and about eight times for sine. Interactive · How good is the straight line? sin x ≈ x √(1+x) ≈ 1 + x/2 eˣ ≈ 1 + x Step x from the point What a graduate just sees A rate and a slope are the same thing, and "how fast?" always means a derivative. A total is an area under a rate, and adding up thin slices is a calculation, not a mystery. Rates multiply along a chain: gear ratios, units, layers of a network. The best value of a smooth quantity sits where its slope is zero. Differentiating and accumulating are inverse jobs, so a total can often be found without adding. Up close, everything smooth is a straight line, and engineers exploit that constantly. Subtracting two nearly equal numbers on a computer destroys precision, so a naive slope estimate gets worse when the gap gets too small. Where it shows up next Physics I (34/06) and Dynamics (34/12): velocity is the derivative of position, acceleration of velocity, and Newton's law ties force to the second one. Differential Equations (34/03): when a law is written as a rate, solving it means integrating, which is the whole next course. Thermodynamics (34/14) and Heat Transfer (34/16): heat added is an integral of power; temperature profiles are read from slopes. Probability & Statistics (34/05): the chance of a range of outcomes is the area under a density curve. Algorithms (34/24) and Machine Learning (34/32): growth rates of running time are compared with limits, and models are trained by following slopes downhill. If you want to go deeper 📘 Textbook: James Stewart, Daniel Clegg and Saleem Watson, Calculus: Early Transcendentals (9th ed., Cengage). The most-assigned text; best for worked examples and a huge problem bank. 📘 Textbook: Hass, Heil, Bogacki and Weir, Thomas' Calculus (15th ed., Pearson). Strong on applications, popular in engineering departments. ▶️ Gentler: Grant Sanderson's Essence of Calculus series on 3Blue1Brown, a visual tour of the same ideas. Also Steven Strogatz, Infinite Powers, for the story. 🌐 Free course: MIT OpenCourseWare 18.01SC Single Variable Calculus, with video lectures, notes and problems. Khan Academy also covers it. The takeaway Calculus I teaches two moves and one bridge. The derivative turns a total into a rate (a slope); the integral turns a rate into a total (an area); and the Fundamental Theorem says they undo each other. From that come the habits of mind: chain rates together, look for zero slope to find the best, and treat smooth things as lines up close. Our slope estimate for ex was best near a gap of 10−8 and useless at 10−16, a reminder that the ideas are exact and the machines are not. Try this: next time you are in a car or on a bike, watch the speedometer and ask what the area under its history means. Then on a hill, find the spot where the road is flat. That is a derivative equal to zero. Sources & Further Reading Sources. Every number in the interactives was computed in node: the forward-difference error of ex at x = 1 for h = 10−1 to 10−16 (best 6.6 × 10−9 at 10−8, exactly 0 at 10−16, in IEEE double precision); sin(x)/x at 0.1, 0.01, 0.001; the box V(x) = 4x³ − 100x² + 600x, zero slope at x = (50 − √700)/6 = 3.9237, V = 1,056.3 cm³; the balloon 4π × 10² × 2 = 2,513.27 cm³/s; the car, 800 m; the tank, A(12) = 48.239 L; small-step errors of the three approximations. Book editions checked against the publishers' pages (Cengage, Pearson) and the course numbers against MIT OpenCourseWare. 🌐 MIT OpenCourseWare 18.01SC Single Variable Calculus ▶️ 3Blue1Brown: Essence of Calculus 🌐 Khan Academy: Calculus 1 📘 Stewart, Clegg and Watson, Calculus: Early Transcendentals, 9th ed. (Cengage) 📘 Hass, Heil, Bogacki and Weir, Thomas' Calculus, 15th ed. (Pearson) 📘 Steven Strogatz, Infinite Powers, a popular history of calculus Step 2. Physics I: Mechanics — Three Ways to Keep the Books ----------------------------------------------------------- Why: Energy, force and motion come before charge: the same ledgers, new currency. Why doubling your speed quadruples your stopping distance, why a skater spins up when she pulls her arms in, and the three questions an engineer asks before touching a calculator. A car doing 50 km/h stops in about 14 metres on a dry road. Most people guess that at 100 km/h it takes about 28. It takes about 56. The extra speed doesn't just cost twice the distance, it costs four times, and the same fact is why a crash at 100 km/h is not "twice as bad" as one at 50. Nobody needs a formula to be afraid of that, but a person who has done a first mechanics course sees it: the brakes have to get rid of the car's energy, energy grows with the square of speed, and the road can only take it away at a steady rate per metre. That one chain of thought is most of the course. Physics I is less a list of topics than a way of keeping accounts. There are three ledgers, force, energy and momentum, and a good physicist picks the one in which the problem is easiest. If you want an easier first read, the library has The Four Forces (13/01), Energy: The One Currency (13/02) and Orbits (13/11). This piece goes further: it makes you use the ledgers. The course in one box Who takes it: first-year students of Mechanical & Aerospace Engineering, and in many programmes Computer Science too, as the science requirement. What it's for: turning "what will this object do?" into something you can calculate. Comes after: Calculus I (34/01). Leads to: Physics II: Electricity & Magnetism (34/07), Statics (34/10), Dynamics (34/12), Materials Science & Manufacturing (34/13), Thermodynamics (34/14). Honest line: you won't be able to design a suspension after this. You will see why a car's tyres, not its engine, decide how fast it can take a corner. Newton's laws are one accounting system Isaac Newton set out his three laws in the Principia in 1687. Read as a bookkeeper would, they say: momentum (mass times velocity) is the thing being tracked; a force is a deposit or withdrawal; and every deposit has a matching withdrawal somewhere else. The first law says that with no net force the account doesn't change: things keep their velocity. The second says that a net force changes momentum at a steady rate, which for a fixed mass is the famous rule below. The third says forces come in pairs between two bodies: if you push a wall, the wall pushes you, and the pair never acts on the same body, which is why they don't cancel. Fnet = m · athe net force on a body is its mass times its acceleration Worked number: a 1,500 kg car getting from 0 to 100 km/h (27.8 m/s) in 8 seconds accelerates at 3.47 m/s², so the road must push it forward with about 5.2 kN. Two ice skaters of 50 kg and 70 kg who push apart from rest leave with equal and opposite momentum: if the heavier moves at 1.0 m/s the lighter goes at 1.4 m/s the other way. Total momentum stays at zero. Try the classic: you on a scale in an elevator. The scale doesn't read your weight, it reads the force pushing up on you. Interactive · The elevator scale (a 70 kg rider) Speeding up, going up Cruising Speeding up, going down Cable cut Elevator acceleration (up is positive) Accelerating upward at 2 m/s² the scale reads about 827 N, like 84 kg. At constant speed it reads your true weight, whether the elevator is moving or still. In free fall it reads zero. That is exactly the "weightless" astronaut of Orbits. Free-body diagrams, and friction as a force that only pushes as hard as it must The graduate's first reflex on any problem is to draw one object, alone, and every push on it: weight, the surface pushing back, friction, tension. Nothing else. That is a free-body diagram. Add the forces as vectors, split them along the directions that matter, and F = ma does the rest. Friction is the force beginners get wrong. Static friction is not a fixed number: it pushes exactly as hard as needed to stop sliding, up to a ceiling of μs times the normal force (how hard the surface pushes back). Only once the ceiling is passed does the block slide, and then friction drops to μk × normal force. f ≤ μs · Nfriction is at most a coefficient times how hard the surfaces press together On a ramp the weight splits into a part along the slope, mg sin θ, and a part pressing into it, mg cos θ. Sliding begins when the first beats μs times the second, which gives tan θ = μs. The mass cancels. Check that in the tool: change the mass and nothing about when it slips changes. (The coefficients are round textbook values; real surfaces vary a lot.) Interactive · A block on a ramp Wood on wood Rubber on concrete Steel on ice Ramp angle Mass Arrows are drawn as fractions of the weight. With μs = 0.5 the block holds until 26.6° and then slides. At 40° it accelerates down at about 4.05 m/s² (with μk = 0.3). This is why a pile of sand has a steepest slope, and why designers of ramps, screws and wedges care so much about one number. Energy: the balance that doesn't care about the path Force problems get painful when the path is complicated. Energy is the escape. The work a force does is force times distance moved along it, and the work–energy theorem says the net work on a body equals its change in kinetic energy. When only gravity or springs are involved, the total of kinetic plus potential energy is simply conserved, whatever the route. KE = ½ · m · v²speed counts squared; this is why speed is so costly A bobsled dropping 30 m ends at √(2 × 9.81 × 30) = 24.3 m/s (87 km/h) with no friction, however many bends the track has. Now back to the opening: the brakes do work μ × m × g × d against the car's ½mv², so the stopping distance is v² ÷ (2μg). Double v, quadruple d. Interactive · Brake, or hit Dry (μ 0.7) Wet (0.5) Snow (0.2) Ice (0.1) Speed when you brake Obstacle ahead Braking only, no reaction time. μ values are round, typical figures for tyres. The sharpest result: brake from 100 km/h with an obstacle 14 m away, the distance that stops a 50 km/h car on a dry road. You still hit it at about 87 km/h, with 75% of the energy you started with: 14 metres of hard braking has removed only a quarter of it. Momentum: what survives every crash Energy has a weakness: in a collision much of it turns to heat, sound and bent metal, so the energy ledger is no use. Momentum, though, is conserved in any collision, because the two forces are equal and opposite for the same short time (the third law again). The total before equals the total after, with direction as a sign. How bouncy the collision is gets one number, the restitution e: 1 for perfectly bouncy, 0 for stuck together. Momentum doesn't care about e. Kinetic energy does. Interactive · Two carts collide Equal masses, bouncy Stick together Light hits heavy Blue mass Blue velocity Gold mass Gold velocity Bounciness e Velocity to the right is positive; arrows show velocity. With equal masses and e = 1, the moving cart stops dead and the other leaves with its speed: the toy known as Newton's cradle. Stuck together, two 1,000 kg cars (one at 10 m/s, one parked) leave at 5 m/s and half the kinetic energy, 25 kJ, becomes damage. That lost half is the reason cars have crumple zones: they spend the energy deforming metal instead of people. Rotation: the same laws with new names Everything above has a spinning twin. Force becomes torque, force times the perpendicular distance from the pivot: 80 N on a 0.25 m wrench is 20 N·m, and a longer wrench gives more for the same push. Mass becomes moment of inertia I, which counts not just how much mass there is but how far from the axis it sits. Momentum becomes angular momentum L = I × ω. τ = I · α   ·   L = I · ωwith no outside torque, L is conserved Mass far from the axis counts for much more, because I grows with the square of distance. That's why a solid disc beats a hoop of the same mass and radius down a ramp: the hoop keeps all its mass at the rim, so it spends more of its energy on spinning (the disc's acceleration is two thirds of g sin θ, the hoop's one half, whatever the size or mass). And it's why a skater spins faster when she pulls her arms in. Interactive · The skater's weights Arms out Halfway Tucked in Weights' distance from the axis A model: body I = 0.9 kg·m², plus two 2 kg weights; start at one turn per second with arms out. From arms out to tucked in, I falls from 3.15 to 0.99 kg·m², so the spin rises from 1 to about 3.2 turns per second. The energy also rises, from 62 J to 198 J, and the difference is the work her muscles do pulling the weights inward. Circular motion and gravity: a force that points at the middle Going round a bend at constant speed is still accelerating, because the velocity keeps changing direction. The acceleration points to the centre and equals v²/r, so something must push sideways with force mv²/r. On a flat road that something is friction. No more friction, no more turn: the car continues in a straight line, which feels, from inside, like being thrown outward. vmax = √(μ · g · r)fastest speed a flat bend of radius r allows Notice the engine's power isn't in the formula. On a 50 m bend with μ = 0.5 (wet) the limit is 56 km/h; dry (0.7), 67 km/h; ice (0.1), 25 km/h. Interactive · Take the bend Dry (0.7) Wet (0.5) Ice (0.1) Speed Radius of the bend Replace friction with gravity and you get orbits. Set v²/r equal to GM/r² and the period T satisfies T² ∝ r³, Kepler's third law. Check it: the Space Station's lap of 92.6 minutes at 6,779 km from Earth's centre predicts, for the Moon at 384,400 km, a period of 27.5 days. The real month is 27.3. One law, from a bend in a road to the Moon. Which ledger? Choosing force, energy or momentum This is the real skill. A hard problem in one ledger is a one-liner in another. Three rough rules: if the question asks for a force or a time, use F = ma; if it asks for a speed between two positions, and the path is messy, use energy; if there's a collision or explosion, use momentum. If something spins, use angular momentum. Try five. Interactive · Pick the first tool Force Energy Momentum Angular momentum Next question What a graduate just sees Stopping distance goes with speed squared: double the speed, four times the distance. A scale reads the push on you, not your mass: it changes in an accelerating elevator. Static friction is only as large as it has to be, up to a limit that depends on μ and the pressing force. Energy ignores the path; use it when the route is complicated. Momentum survives every collision; kinetic energy rarely does. Mass far from the axis counts for much more in rotation, because distance is squared. A turn needs a sideways force; remove the grip and you go straight. Weightless means falling together, not "no gravity". Where it shows up next Statics (34/10) is F = ma with a = 0: the free-body diagram becomes the bridge or crane that stays put. Dynamics (34/12) and Vibrations (34/17) take rotation and the second law into machines and springs, usually as differential equations (Differential Equations, 34/03). Thermodynamics (34/14) is the energy ledger with heat added: the lost half of that crash is where it starts. Physics II (34/07) redoes the same accounting for charges, with Coulomb's force and voltage as energy per charge. In Programming Fundamentals (34/09), you'll turn F = ma into a simulation loop; in Computer Science game engines and robotics code, it's the physics step. If you want to go deeper 📘 Standard textbooks. Halliday, Resnick & Walker, Fundamentals of Physics (12th ed., Wiley, 2021), or Young & Freedman, University Physics (16th ed., Pearson); the usual first-year choices, with worked examples and thousands of problems. For a sharper, more mathematical route, Kleppner & Kolenkow, An Introduction to Mechanics (2nd ed., Cambridge). 📘 Gentler, free. The Feynman Lectures on Physics, Volume I, free online at feynmanlectures.caltech.edu; chapters 4 (energy) and 10 (momentum) are a good start. 🌐 Free course. MIT OpenCourseWare 8.01SC Classical Mechanics (Fall 2016): lectures, problem sets and solutions. The takeaway Mechanics is three ledgers kept over the same world. Force tells you what changes motion and when; energy lets you skip the path; momentum survives collisions; and each has a spinning twin. Doubling speed quadruples stopping distance because energy goes with v². The graduate's skill is choosing which ledger makes this problem easy. Try this: next time you're a passenger, estimate how far the car needs to stop at the speed you're doing, using "14 m at 50 km/h, four times that at 100". Then notice how far ahead the next car really is. Sources & Further Reading Sources. Newton's laws first published in Philosophiae Naturalis Principia Mathematica (1687). Textbook physics throughout. Friction coefficients are rounded from published tables, which vary widely between sources (for example Engineering Toolbox's "Friction coefficients for common materials", which gives rubber on dry concrete 0.6–0.85, on wet concrete 0.45–0.75). Every number in the interactives was computed in node and checked by hand: 14.05 m and 56.2 m braking distances at 50 and 100 km/h (μ = 0.7); 86.6 km/h residual speed; 5.21 kN; 24.3 m/s; skater I = 3.15 and 0.99 kg·m², 3.18 turns per second, 62 J and 198 J; 56.4, 66.7 and 25.2 km/h bend limits; Kepler scaling from the Space Station (92.6 min at 6,779 km) to the Moon (27.46 days from 384,400 km). 🌐 MIT OpenCourseWare: 8.01SC Classical Mechanics 📘 The Feynman Lectures on Physics, Vol. I (free online, feynmanlectures.caltech.edu) 📘 Kleppner & Kolenkow, An Introduction to Mechanics, 2nd ed. (Cambridge University Press) 🌐 Engineering Toolbox: friction coefficients Step 3. Linear Algebra: Machines That Move Space ------------------------------------------------ Why: A circuit with fifty nodes is fifty equations; matrices solve them. A grid of numbers is really a machine that moves all of space at once. Learn to see it, and circuits, bridges, search engines and AI all start to look alike. You pinch two fingers on a photo and it rotates and zooms smoothly under them. A game character turns to face you and every corner of its face swings round together. A search for "best pizza near me" ranks millions of pages in a fraction of a second. A bridge designer presses "solve" and a screen of colour shows how every beam is being squeezed. Underneath all four is the same object: a grid of numbers, and one habit of thought about what a grid of numbers does. A graduate of a linear algebra course no longer sees a table of numbers. They see a machine that grabs the whole plane, or the whole of space, and moves it. This piece builds that picture with seven things you can drag. You will not come out able to compute a determinant of a 5×5 by hand (nobody does that on purpose). You will come out seeing why a system of equations sometimes has no answer, why a bridge hums at particular pitches, and why the chips that run AI are built for one operation only. The course in one box Who takes it: nearly every engineering and computer science student, usually in the first or second year. It is the single most reused maths course in both majors. What it's for: handling many unknowns and many dimensions at once, with one set of tools. Comes after: only school algebra and a little geometry. (Calculus helps but is not required.) Leads to: Statics (34/10), Control Systems (34/20) and Machine Learning (34/32) list it as a prerequisite; Vibrations (34/17), Computer Architecture (34/25) and Multivariable Calculus (34/02) lean on it constantly. Honest line: you won't be able to run a finite element program after this piece. You will see why it spends nearly all its time solving one equation, Ax = b. If you want an easier first read, try the library's Vectors, Matrices and Linear Transformation (12/14); this one goes further into solving, eigenvectors, fitting and why hardware cares. A matrix moves all of space at once Start with the picture a graduate keeps. A 2×2 matrix is not four numbers. It is two arrows. The first column says where the arrow pointing one step right (call it i) ends up. The second column says where the arrow pointing one step up (j) ends up. Once you know where those two land, you know where everything lands, because every point is some amount of i plus some amount of j, and the machine is straight and even: grid lines stay straight, and evenly spaced lines stay evenly spaced. Ax = x1(column 1) + x2(column 2) Read that in words: the matrix times a point is "that many of the first column, plus that many of the second". Try it. The grey grid is the original plane; the blue grid is where the matrix sends it. Interactive · Move the plane Do nothing Rotate 30° Stretch Shear Mirror Column 1, right part (a) Column 2, right part (b) Column 1, up part (c) Column 2, up part (d) Without JavaScript: the rotate-30° matrix sends the arrow (1, 0) to (0.87, 0.5) and (0, 1) to (−0.5, 0.87). Every grid line turns 30° and the unit square keeps area 1. Notice what the shaded unit square does. In "Rotate" it keeps its size. In "Stretch" it doubles. In "Shear" it leans over but keeps its area. That area change is the next idea. The determinant is how much area gets scaled The determinant is one number that says how much the machine scales area. If the unit square becomes a shape with area 5, the determinant is 5. If it becomes a mirror image, the determinant is negative: area is kept (or scaled) but orientation is flipped. For a 2×2 matrix with columns (a, c) and (b, d) the rule is short: det = ad − bc Worked number: the matrix with columns (2, 1) and (1, 3) has det = 2×3 − 1×1 = 5, so it stretches every area in the plane by a factor of five. In three dimensions the determinant scales volume. The interesting case is zero. A determinant of zero means the machine flattens space: the plane is squashed onto a line (or a point), and the area becomes nothing. Squashing cannot be undone, because many different starting points land on the same spot. Drag the slider until the determinant reaches zero. Interactive · Squash the plane flat Tilt the second arrow: s Without JavaScript: the matrix with columns (1, 0.5) and (s, 1) has determinant 1 − 0.5s. At s = 2 the determinant is 0, the whole plane lands on one line, and the points (0, 0) and (2, −1) both land on the origin. The orange dot is a probe at (2, −1). At s = 2 it lands exactly on the origin, the same place as the point (0, 0). Two different inputs, one output: no machine could tell you which one you started from. That is the whole meaning of "this matrix has no inverse", and it is why a zero determinant is the alarm bell in every solver you will ever meet. Solving many equations at once: Ax = b Now use the machine backwards. Given the output b, which input x did the machine start from? That is the equation Ax = b, and it is the most common problem in applied mathematics. Every time you have several unknowns that are tied together by several straight-line relationships, you have one. A circuit is the cleanest example. Two loops share a resistor. Kirchhoff's voltage law says the voltage gained around each loop equals the voltage lost, which gives one equation per loop. With a 10 V battery, R1 = 2 Ω, R2 = 3 Ω (shared) and R3 = 4 Ω: 5I1 − 3I2 = 10−3I1 + 7I2 = 0 The determinant is 5×7 − 3×3 = 26, so there is exactly one answer: I1 = 2.69 A and I2 = 1.15 A. Change the resistors below and watch the matrix and the answer change together. Interactive · Solve a circuit with a matrix Battery V R1 (loop 1 only) R2 (shared) R3 (loop 2 only) Without JavaScript: with 10 V, 2 Ω, 3 Ω and 4 Ω the equations are 5I1 − 3I2 = 10 and −3I1 + 7I2 = 0, its determinant is 26, and the loop currents are 2.69 A and 1.15 A. The reason an engineer cares: nothing about this method knows it is a circuit. A truss is the same. At each joint of a bridge, the horizontal forces must add to zero and so must the vertical ones: two equations per joint. A simple triangle truss has three joints and so six equations, for three member forces and three support reactions, and the answer is one solve of Ax = b. A real structure has thousands of joints. Finite element programs routinely solve systems of many thousands of equations, and research codes have reported solving over a million unknowns at once. The method is the same one you just used; only the size has changed. (Statics, 34/10, is this idea applied to forces.) A zero determinant in a structure's equations is a warning: the structure may be able to move without any member pushing back. Eigenvectors: the directions a transformation only stretches Most arrows are turned by a matrix as well as stretched. But for many matrices there are special directions where the arrow comes out pointing the same way (or exactly backwards), only longer or shorter. Those are the eigenvectors, and the stretch factor along each is its eigenvalue. (Eigen is German for "own": the matrix's own directions.) Av = λv In words: feed the matrix the special arrow v and it hands back the same arrow multiplied by a plain number λ. For the matrix with columns (2, 1) and (1, 2) the special directions are 45° (stretched by 3) and 135° (stretched by 1). Every other arrow gets bent towards the stretchy one. Turn the dial and look for the moments the two arrows line up. Interactive · Find the directions that only stretch Stretch & mix Shear Rotation Two springs Direction of the input arrow Without JavaScript: for the matrix with columns (2, 1) and (1, 2), an arrow at 45° comes out at 45° three times longer, and an arrow at 135° comes out at 135° unchanged. A pure rotation has no such direction at all. Three things to notice. The shear has only one special direction (along the x-axis, where nothing happens at all). The rotation has none, because every arrow turns. And "Two springs" has two special directions with different stretch factors, 1 and 3; hold that thought. Eigenvectors at work: bridges that hum, pages that rank Vibration. Picture two equal masses on springs between two walls. Push them any way you like and the motion looks messy. But there are exactly two clean patterns: both swing together, or they swing in opposite directions. These are the modes, and they are the eigenvectors of the "two springs" matrix above, (1, 1) and (1, −1). The eigenvalues 1 and 3 fix how fast each mode vibrates: the squared frequency is the eigenvalue times k/m, so the opposite-motion mode vibrates √3 ≈ 1.73 times faster. Any messy motion is just a mix of the modes. That is how engineers find which pitches will shake a wing, a bridge deck or a drive shaft (Vibrations, 34/17). Ranking pages. Google's founders modelled a web surfer who follows a random link from each page, and now and then jumps to a random page instead. The long-run share of time spent on each page is its PageRank, and it is the eigenvector (with eigenvalue 1) of the matrix describing one step of that surfer. In their 1998 paper Brin and Page used a "damping" probability of 0.85 for following links. You can find the eigenvector without solving anything fancy: start with equal scores and apply the machine over and over. Press Step. Interactive · A four-page web finds its eigenvector Step onceRun 20 stepsReset Without JavaScript: A links to B and C, B links to C, C links to A, D links to C. After many steps with damping 0.85 the ranks settle to A 0.373, B 0.196, C 0.394, D 0.038. D, which nobody links to, keeps only the base share. After about twenty steps the numbers stop changing: the vector has become an eigenvector, because applying the machine again returns the same vector. Page C wins because two pages point at it and one of them (A) is itself well-ranked. Least squares: the best answer when there is no exact one Real data is never perfect. Six measurements that "should" lie on a straight line will not, so the equations Ax = b have more equations than unknowns and no exact solution. Do you give up? No. You choose the line whose total mistake is smallest. Measure each miss vertically, square it, and add up the squares. The least squares line makes that total as small as possible. In pictures: each squared miss is literally a square. Drag the line and watch the squares shrink and grow. Interactive · Make the squares as small as possible Slope Height at x = 0 Snap to the least-squares line Without JavaScript: for the six points (1, 1.2), (2, 2.9), (3, 3.1), (4, 4.8), (5, 5.2), (6, 7.1) the best line is y = 1.089x + 0.24, with total squared miss 0.798. The guess y = x gives 2.75. The graduate's picture is geometric: the right-hand side b is a point outside the flat sheet of everything the matrix can reach, and the best answer is the point on the sheet closest to it, found by dropping a perpendicular. That perpendicular condition gives the one equation to remember: ATA x̂ = ATb In words: multiply both sides by the matrix flipped on its side and you get a square system you can solve. The history is a priority fight: Legendre published the method in 1805, Gauss said he had used it since 1795 and published in 1809, and Gauss's use of it to predict where the asteroid Ceres would reappear in 1801 made it famous. Today it sits under every trend line, every calibration of a sensor, and the first model most machine-learning students ever train. Why GPUs and AI are matrix multiplication Multiplying two matrices is just many dot products: every cell of the answer is one row of the first matrix paired with one column of the second, multiplied and added. Click a cell below and see the row and column that make it. Interactive · Build a product cell by cell Tap a cell of C = A × B. Matrix size n (n × n times n × n) 1 billion multiplies per second 100 billion per second 10 trillion per second The speeds are round illustrative numbers, not any particular chip. Without JavaScript: multiplying two n×n matrices takes n×n×n multiplications. For n = 1,000 that is 1,000,000,000, and the 1,000,000 cells of the answer can all be computed independently. Two facts follow. First, the work is huge: n×n matrices need n³ multiplications, so 1,000×1,000 is a billion. Second, the cells are independent, so a thousand workers (or a million) can each take cells at the same time. That is exactly the shape of work a graphics chip was built for, because drawing a 3D scene means pushing every corner of every triangle through a matrix that rotates, moves and projects it. Modern neural networks turned out to be the same shape. A layer of a network is a matrix of learned numbers times a vector (or a matrix) of inputs. In their scaling-law paper, Kaplan and colleagues estimate that a transformer language model with N parameters needs roughly 2N arithmetic operations to process one token. GPT-3, with 175 billion parameters, therefore needs about 3.5×1011 operations for each token, nearly all of them inside matrix products. NVIDIA's Volta GPUs (2017) put dedicated "tensor core" circuits on the chip that do nothing but a small matrix multiply-and-add (4×4) per clock tick. When people say AI runs on GPUs, they mean it runs on hardware shaped like linear algebra. (Computer Architecture, 34/25, and Machine Learning, 34/32.) What a graduate just sees A matrix is a machine that moves all of space; its columns say where the basis arrows go. The determinant is the area (or volume) scale factor, and zero means the machine flattens space and cannot be undone. "Solve Ax = b" is the same question for a circuit, a truss, a fitted line and a game physics step. If there are as many independent equations as unknowns there is one answer; fewer, many answers; contradictory, none. Eigenvectors are the directions a machine only stretches; they simplify any problem the machine drives. A vibrating structure is a mix of modes, one per eigenvector; the eigenvalues are the pitches. When there is no exact answer, take the closest one: project onto what you can reach. Matrix multiplication is many independent dot products, which is why it parallelises and why hardware is built around it. Where it shows up next Statics (34/10) and Mechanics of Materials (34/11): joint and member forces are a linear system; stiffness matrices tie loads to deflections. Vibrations (34/17) and Dynamics (34/12): mode shapes and natural frequencies are eigenvectors and eigenvalues of mass and stiffness matrices. Control Systems (34/20): the eigenvalues of a system's matrix say whether a drone's wobble dies away or grows. Machine Learning (34/32): data is a matrix, a layer is a matrix, and training adjusts matrices; least squares is the model you meet first. Computer Architecture (34/25) and graphics: GPUs and tensor cores exist to multiply matrices fast. If you want to go deeper 📘 Gilbert Strang, Introduction to Linear Algebra (Wellesley-Cambridge Press; 6th edition, 2023). The standard first course for engineers; best for the four fundamental subspaces and for application. 📘 David C. Lay, Steven R. Lay and Judi J. McDonald, Linear Algebra and Its Applications (Pearson; 6th edition, 2021). Gentle, example-heavy, widely assigned. For proofs, Sheldon Axler's Linear Algebra Done Right (Springer; 4th edition, free to read online at linear.axler.net) is the elegant theoretical route. ▶️ 3Blue1Brown, Essence of Linear Algebra: the best visual series for learning alone; it builds exactly the "matrix as moving space" picture used here. 🌐 MIT OpenCourseWare 18.06, Linear Algebra (Gilbert Strang): full video lectures, free. The takeaway Linear algebra is the study of machines that move all of space at once. Read a matrix by its columns, read its determinant as the area scale, solve Ax = b when you need to run the machine backwards, look for the eigenvector directions when you want it to be simple, and project onto what you can reach when there is no exact answer. Try this: next time an app zooms, rotates or recolours a photo, picture the two column arrows moving. And for the fun of it, in the circuit widget raise R2 to 10: the shared resistor starts to decide what both loops do. Sources & Further Reading Sources. Strang, Introduction to Linear Algebra (6th ed., 2023) and MIT 18.06 for the standard course content. Kirchhoff loop equations and the spring-mass modes are textbook derivations; every number above was recomputed in code. Legendre's 1805 publication and Gauss's priority claim and Ceres prediction: Wikipedia, "Least squares"; MacTutor biography of Legendre. PageRank and the 0.85 damping factor: Brin and Page, "The Anatomy of a Large-Scale Hypertextual Web Search Engine" (1998). Volta tensor cores (4×4 matrix multiply-accumulate): Markidis et al., "NVIDIA Tensor Core Programmability, Performance & Precision" (2018). About 2N operations per token: Kaplan et al., "Scaling Laws for Neural Language Models" (2020); GPT-3 at 175 billion parameters: Brown et al. (2020). Million-unknown structural solves: NASA Technical Reports Server and OSTI conference papers on parallel sparse solvers. Go deeper. The books and courses above. Step 4. Programming Fundamentals: Instructions, State and Taking Things Apart ----------------------------------------------------------------------------- Why: Electrical engineers simulate, measure and program before they solder. A computer can't guess what you meant. Everything in a first programming course is a way of saying it exactly enough, and of working out why you didn't. Tap "sort by price" and a list of a thousand items rearranges before you've lifted your finger. Somewhere, a person wrote down steps so exact that a machine with no judgment, no memory of yesterday and no idea what a "price" is could do it perfectly, a thousand times over. That is the whole trade. A programmer isn't someone who knows a language, they're someone who can state a task so precisely that nothing is left to guess. And since people are bad at being precise on the first try, the other half of the skill is finding out what you actually said, as opposed to what you meant. Mechanical and aerospace engineers use the same habits to simulate a wing before it's built; computer scientists use them to build every other subject on the degree. Python and MATLAB are the common first languages for engineers, and this piece uses Python-style code. The ideas carry to every language. The course in one box Who takes it: first-year students of Computer Science (usually their very first course) and Mechanical & Aerospace Engineering (often in MATLAB or Python). What it's for: telling a machine what to do exactly, and finding out when you didn't. Comes after: nothing; it has no prerequisites. Leads to: Data Structures (34/23), Computer Architecture (34/25) and Software Engineering (34/31) on the computing side; Differential Equations (34/03), Dynamics (34/12) and Control Systems (34/20) on the engineering side, all of which are solved on a computer in practice. Honest line: you won't be able to ship an app after this. You will be able to read ten lines of code, predict what they do, and make a physics model in a loop. A program is instructions plus state Strip away the syntax and a program is two things: a list of instructions, and the state, the current values of all its named boxes. Each instruction reads the state and changes it a little. Running the program is just that, one step after another, and the answer is whatever state is left (or printed) at the end. The graduate habit is to run the program in their head: keep a small table of variable values and update it line by line. Below is a tiny interpreter that does it visibly. Press Step. Then edit the program: change the 6 in the Collatz example, or write your own with =, +=, if/else, while, for … in range(…) and print. Interactive · Step through a program Add 1 to 5 Collatz steps Count down by 3 The program (edit me) StepRun to the endStart again A real mini-interpreter, written for this page. It stops after 400 steps, in case you write a loop that never ends. The first program adds 1 + 2 + 3 + 4 + 5 and prints 15. The Collatz one halves even numbers and triples-plus-one odd ones until it reaches 1: starting from 6 it takes 8 steps. Nobody knows if every starting number reaches 1, and yet the program is five lines long. That's what "simple instructions, deep behaviour" feels like. Variables and types: what a number is made of A variable is a name for a box in memory. A type says what kind of thing is in the box, and therefore what you may do with it: you can add two numbers, join two pieces of text, but adding "3" and 4 is an error or a surprise. Types exist because the machine stores everything as patterns of bits, and the pattern means nothing without the type. The box has a size too. A signed 16-bit integer holds −32,768 to 32,767, and nothing more. Decimals are stored as floating point, binary fractions of limited precision, so 0.1 can't be stored exactly and 0.1 + 0.2 gives 0.30000000000000004. On 4 June 1996 the first Ariane 5 rocket veered off course and broke up about 37 seconds after lift-off, a loss of more than US$370 million. The investigation traced it to the guidance software, reused from Ariane 4: a 64-bit floating-point value (the horizontal bias) was converted to a 16-bit signed integer, and on the faster rocket it didn't fit. The failure shut down both inertial units. A number too big for its box was enough. Interactive · Does it fit? 8 bits 16 bits 32 bits Number to store Ariane's code reportedly raised an error on overflow; some systems wrap silently instead, which is shown here. Conditionals and loops: choosing and repeating With state and instructions, only two more ideas are needed to compute anything: choose (if something is true, do this, else that) and repeat (while something is true, or for each value in a range). Together they let ten lines do work that would take ten million lines written out. The classic trap is in the edges. In Python's range(1, 10) the loop starts at 1 and stops before 10, so it runs 9 times, not 10. Off-by-one errors are the "fencepost" mistake: a fence of 10 sections needs 11 posts. Try the bounds yourself. Interactive · Edit the loop Add up i Add up i² Just count Start Stop (never reached) Step With the defaults, range(1, 11) runs 10 times and adds up to 55. Make the step negative and the start bigger than the stop to count down. Set the start above the stop with a positive step and the body never runs at all, which is perfectly legal and a common source of silent bugs. Functions and recursion: naming a piece, then calling it again A function packages some instructions under a name, takes inputs, and gives back an output. That is decomposition, the central skill of engineering of any kind: break a big problem into pieces small enough to understand and test alone, give each a name, then build on the names. The tracer above could be one function; so could braking distance. Recursion is a function that calls itself on a smaller version of its problem, until it reaches a case it can answer directly. The Fibonacci numbers (0, 1, 1, 2, 3, 5, 8, …, each the sum of the previous two) have a one-line recursive definition. Run it naively and the machine re-solves the same sub-problems again and again. Interactive · Count the calls Naive Remember answers (memo) Compute fib(n), n = Time it on this device Naively, fib(5) makes 15 calls, fib(20) makes 21,891, and fib(30), whose answer is just 832,040, makes 2,692,537. Remembering answers brings fib(30) down to 59 calls. Same answer, a different algorithm, and the course after this one (Algorithms, 34/24) is the study of exactly that difference. Data: lists and dictionaries Most programs aren't about single numbers but collections. A list is an ordered row of values, found by position (the third item). A dictionary maps a key to a value (a word to its count, a student ID to a name) and finds an entry by key without scanning. Choosing the right container is the first lesson of Data Structures (34/23). The difference is speed. To find a word in a list of n you may have to look at all n. A dictionary (usually built as a hash table) jumps almost straight to it, in roughly constant time on average, however large it is. Paste some text below and look a word up. Interactive · Count the words Textthe cat sat on the mat and the dog sat on the cat Look up a word Debugging is the scientific method Programs don't do what you meant; they do what you wrote. Finding the gap is science: look at the result, form a hypothesis ("the loop is skipping the last item"), make a prediction ("if I print the count here it will be one short"), run the experiment (print it), and revise. Guessing and changing things at random is the opposite, and it's how you add new bugs. The strongest move is bisection. If the code is a chain of steps and the result is wrong, check the value halfway along. If it's right, the fault is in the second half; if wrong, the first. Each check halves the suspects. In 1947 operators of Harvard's Mark II computer taped a moth found stuck in a relay into the logbook with the note "First actual case of bug being found" — a joke that only works because engineers already called faults "bugs". Interactive · Find the bug in as few probes as you can Start with x = 5. Slide to a step and probe the value there against what you'd get by hand. Look at x Probe hereBlame this stepNew bug Twelve suspect lines need at most four probes when you halve each time, because 2⁴ = 16 is at least 12. Checking from the top, one line at a time, can take twelve. Modelling: a physics simulation in a dozen lines Engineers model long before they build. Most physical laws give a rate of change (acceleration from forces, as in Physics I), and a computer can follow a rate forward in small steps: update the position using the velocity, update the velocity using the acceleration, repeat. That is the Euler method, and the loop is the whole simulation. x, y = 0.0, 0.0 vx, vy = 21.2, 21.2 # 30 m/s at 45 degrees dt = 0.1 # time step in seconds while y >= 0: x = x + vx * dt y = y + vy * dt vy = vy - 9.81 * dt # gravity changes vy only print(x) # about 93.9 metres Smaller time steps are more accurate and cost more. For the cannonball at 30 m/s and 45°, the exact range without air is 91.7 m. With a step of 0.5 s the simulation says 102.1 m, 0.1 s gives 93.9 m, and 0.001 s gives 91.8. Now add air drag, for which no tidy formula exists: the same loop still works, which is why engineers simulate. Interactive · A cannonball in code No air With air drag Launch speed Launch angle Time step Drag is modelled as a quadratic term with a baseball-like coefficient (0.007 per metre); a model, not a measurement. Readable code is a letter to a person A program is run by a machine once and read by people many times: the next engineer, the reviewer, you in six months. The well-known line from the preface of Structure and Interpretation of Computer Programs is: "Programs must be written for people to read, and only incidentally for machines to execute." The machine will accept either version below. Only one of them can be checked by a human. def f(a, b): return (a / 3.6) ** 2 / (2 * b * 9.81) def braking_distance(speed_kmh, friction): speed = speed_kmh / 3.6 # convert to metres per second return speed ** 2 / (2 * friction * 9.81) Both give 14.05 m for 50 km/h and a friction of 0.7, the braking distance of Physics I. Only the second tells you what it means, and so lets a reader spot a wrong constant. Good names, small functions and comments that say why are the cheapest debugging tools there are. People disagree on how much structure is enough: some teams favour heavy up-front design, others short cycles of build, test and revise; Software Engineering (34/31) covers both camps. What a graduate just sees A program is instructions plus state; to understand it, track the variables line by line. Every value has a type and a size, and a number too large for its box is a real way to lose a rocket. Loops run until their stop value, not through it: watch the edges. Break a problem into named functions you can test one at a time. A function that calls itself can be elegant and wildly wasteful; counting calls shows which. The right container (list or dictionary) can be the difference between instant and never. Debugging means hypotheses and experiments; halve the suspects each time. Simulation is a rate of change plus a loop, and smaller steps cost more for less error. Code is read far more often than it is written. Where it shows up next Data Structures (34/23) and Algorithms (34/24): the list-versus-dictionary and naive-versus-memo contrasts above, measured properly. Computer Architecture (34/25): the fixed-size boxes and overflow are the hardware's registers; instructions plus state is literally how a processor works. Software Engineering (34/31): decomposition, testing and readable code, scaled to teams and years. Differential Equations (34/03), Dynamics (34/12), Control Systems (34/20): the Euler loop is the starting point of numerical solvers, vehicle simulators and the code that runs a controller. Machine Learning (34/32): training is a loop that nudges numbers until the error falls. If you want to go deeper 📘 Standard textbook. John Guttag, Introduction to Computation and Programming Using Python (3rd ed., MIT Press, 2021), the text behind MIT's introductory course, written for engineers and scientists, with modelling and data. 📘 Gentler, free. Allen Downey, Think Python (3rd ed., Green Tea Press / O'Reilly), free to read online at greenteapress.com. For the deeper "why", Structure and Interpretation of Computer Programs by Abelson and Sussman is free online from MIT Press. 🌐 Free course. MIT OpenCourseWare 6.0001 Introduction to Computer Science and Programming in Python (Fall 2016), or Harvard's CS50x (starts in C, then Python and SQL) and CS50P (Python only), free on OpenCourseWare. The takeaway A program is instructions acting on state. Variables, types, choices, loops and functions are how you say it exactly; lists and dictionaries are how you hold data; debugging is hypothesis and experiment; and a simulation is a rate of change inside a loop. The graduate's real skill is reading code, predicting it, and taking a big problem apart. Try this: take any small thing you do by hand, like splitting a bill or converting recipe units, and write it as exact steps for a friend to follow without any judgment. Then watch where they get stuck: those are your bugs. Sources & Further Reading Sources. Ariane 5 flight 501 (4 June 1996; 64-bit floating-point to 16-bit signed integer conversion overflow; about 37 seconds; more than US$370 million): Wikipedia, "Ariane flight V88", which summarises the Inquiry Board's report. The Mark II moth (9 September 1947, logbook note "First actual case of bug being found", now at the Smithsonian) from the Computer History Museum and JSTOR Daily, "The bug in the computer bug story", which notes the term was already in use. The SICP quotation from the preface of Abelson & Sussman, Structure and Interpretation of Computer Programs (MIT Press, 1985). CS50x language order from cs50.harvard.edu/x; Think Python 3rd ed. from greenteapress.com. Computed and checked in node, including by running the page's own interpreter on its example programs: 15; 8 Collatz steps; −2; 40,000 wraps to −25,536 in 16 bits; fib call counts 15, 177, 21,891 and 2,692,537 (and 2n−1 with a memo); projectile ranges 91.74 m exact and 102.1, 93.9, 91.8 m for steps of 0.5, 0.1 and 0.001 s; 14.05 m braking distance. 🌐 MIT OpenCourseWare: 6.0001 🌐 Harvard CS50x and CS50P 📘 Think Python, 3rd edition (free online) 📘 Guttag, Introduction to Computation and Programming Using Python, 3rd ed. (MIT Press) 🌐 JSTOR Daily: the bug in the computer bug story Step 5. Discrete Mathematics: Counting, Logic and Proof ------------------------------------------------------- Why: Logic and binary are the alphabet of every digital chip. Computers cannot handle the smooth infinite; they handle countable things. This is the maths of switches, steps, networks and clocks. Your phone adds two numbers in a few billionths of a second, using nothing but switches that are either on or off. A barcode scanner knows you mistyped one digit of a book's number before it looks anything up. A map app finds the quickest route among millions of roads without trying them all. And a password check takes the same instant whether the password is long or short. None of these runs on the smooth, continuous maths of calculus. They run on things you can count: bits, steps, roads, remainders. A graduate of a discrete mathematics course carries a small set of habits for reasoning about countable things, and they are the habits that separate "it seems to work" from "it must work". This piece gives you those habits through six ideas and eight things to try: build an adder from logic, count what you cannot list, watch an induction proof grow, search a graph, wrap numbers around a clock, and see why some programs take longer than the age of the universe. The course in one box Who takes it: every computer science student in the first year, and many engineers; it is the entry ticket to the theory side of computing. What it's for: reasoning exactly about finite and countable things: logic, collections, steps, networks. Comes after: nothing but school mathematics. (Programming Fundamentals, 34/09, helps but isn't required.) Leads to: Algorithms (34/24), Computer Architecture (34/25), Databases (34/28) and Theory of Computation (34/29) all list it as a prerequisite. Honest line: you won't be able to prove a hard theorem after this. You will be able to see why a loop must stop, why a network has a shortest path, and why one algorithm will never finish. For easier first reads, see the library's Graph Theory (12/18) and Cryptography (12/09); this piece goes further into logic, proof, counting and growth. Logic: truth tables become circuits A proposition is a statement that is either true or false. Three operations build all the rest: AND (true only when both are true), OR (true when at least one is) and NOT (flips it). A truth table lists every case, and for two inputs there are only four rows. Another useful operation is XOR ("one or the other, but not both"). ABA AND BA OR BA XOR B 00000 01011 10011 11110 Boole's 1854 book treated logic as algebra; the AND/OR/NOT algebra we use today was tidied up by his successors. The leap to hardware came in 1937, when Claude Shannon's MIT thesis showed that networks of switches could carry out exactly this algebra. Every gate in a processor since is a physical truth table. Look at the XOR column: it is adding. 1 + 1 = 10 in binary, so the digit is 0 (XOR) and there is a carry of 1 (AND). Chain that into a full adder (it also takes the carry from the column on its right) and line four of them up, and you have a circuit that adds numbers up to 15. Choose two numbers and watch the carries ripple. Interactive · Add with nothing but logic gates First number Second number Without JavaScript: 7 + 9 is 0111 + 1001 in binary. The carry ripples through all four columns and the result is 10000, which is 16. sum = A XOR B XOR carry-in    carry-out = (A AND B) OR (carry-in AND (A XOR B)) That is a complete circuit for one column: five gates. Everything a processor does (comparing, jumping, multiplying) is built the same way, so Computer Architecture (34/25) is largely this idea scaled up. Engineers also use the same logic on whole systems: a fault tree in reliability analysis is AND and OR gates over "this valve fails" and "this pump fails" (Systems Engineering, 34/22). Sets, functions and counting: sorting the world and counting it A set is a collection with no repeats; a function assigns each member of one set exactly one member of another. A function is one-to-one if no two inputs share an output, and onto if every possible output is hit. Remember these words: "the hash has collisions" means "the function is not one-to-one", and a compression scheme that shrinks every file would have to be one-to-one into a smaller set, which is impossible. The first counting rule is about overlap. If you add the sizes of two sets, anything in both gets counted twice, so subtract it once. This is inclusion–exclusion: |A ∪ B| = |A| + |B| − |A ∩ B| In a class of 40, 25 take computing, 18 take maths and 9 take both: 25 + 18 − 9 = 34 take at least one, so 6 take neither. Interactive · Count the overlap Take computing Take maths Take both Without JavaScript: in a class of 40 with 25 in computing, 18 in maths and 9 in both, 34 take at least one subject and 6 take neither. Next, counting arrangements. If order matters, choose the first item in n ways, the next in n−1, and so on: choosing and ordering k of n items gives n!/(n−k)! possibilities. If order doesn't matter, divide by the k! ways to shuffle the chosen ones: that is a combination, written C(n,k). C(n,k) = n! / ( k! (n−k)! ) Worked example: a five-card poker hand from 52 cards is C(52,5) = 2,598,960 hands, while the number of orderings of a whole deck is 52! ≈ 8.07×1067. A shuffled deck has almost certainly never been in that order before in history. Interactive · Orderings versus selections Items to choose from: n Items chosen: k Without JavaScript: choosing 3 of 10 gives 720 ordered ways and 120 unordered ways; choosing 5 of 52 gives 2,598,960 hands. The sneakiest counting idea has no formula. The pigeonhole principle says that if you put more than m items into m boxes, some box holds two. It sounds trivial, and it proves strong things: among any 13 people, two share a birth month; no compression method can shrink every file; any hash function that maps many inputs to few outputs must collide. Dirichlet used it in number theory in 1834. Drop random people into the seven days of the week: you are guaranteed a repeat by the eighth, and you will usually get one sooner (most often between the 3rd and 5th drop). Interactive · Pigeons into seven boxes Drop a pigeonDrop until a doubleClear Without JavaScript: with 7 boxes, the 8th pigeon is guaranteed to share a box. With random drops the first double typically comes earlier, most often between the 3rd and the 5th pigeon. Proof: induction and contradiction You can't test infinitely many cases, so mathematicians prove. Two tools do most of the work. Induction is the domino proof: show the first domino falls (the base case), and show that if any domino falls, the next one does (the step). Then they all fall. Take the claim that the first n odd numbers add up to n². Base case: 1 = 1². Step: if the first n odd numbers make an n×n square, the next odd number, 2n+1, is exactly an L-shaped border that turns it into an (n+1)×(n+1) square. Slide n and watch the border go on. Interactive · Squares grown by odd numbers Odd numbers added: n Without JavaScript: 1 + 3 + 5 + 7 + 9 = 25 = 5². Adding the next odd number, 11, makes 36 = 6²: a border of 2×5+1 = 11 squares. Induction is how you prove a loop does what it should: the loop invariant is the claim, the first pass is the base case, and each pass is the step. It is also the heart of recursion, which is a program that calls a smaller copy of itself. Contradiction works the other way: assume the claim is false and show that this leads to nonsense. Euclid's proof that there are infinitely many primes is usually told this way: suppose the list of primes were finite, multiply them all together and add one. That number leaves remainder 1 when divided by every prime on your list, so either it is itself a new prime or it has a prime factor you didn't list. Either way the list was incomplete. (Some historians point out Euclid's own argument can be read as a direct one; the contradiction telling is the modern standard.) A programmer uses the same move when they say "suppose the algorithm returned a wrong answer; follow that and it breaks the invariant". Graphs and trees: things and their connections A graph is dots (nodes) and links (edges); the library's Graph Theory piece (12/18) introduced it. Here is a deeper habit. Count the edge ends: each edge has two, so the degrees (links at each node) always add up to exactly twice the number of edges. This handshake lemma is why the number of guests who shook an odd number of hands is always even. The workhorse algorithm is breadth-first search: visit the start, then everything one step away, then everything two steps away, using a queue. It finds the fewest-hops route to every node. The edges it used to reach each new node form a tree: a connected graph with no loops, which always has exactly one fewer edge than nodes. Pick a start and press Step. Interactive · Breadth-first search, one step at a time StepRun to endRestart Without JavaScript: in the 8-node, 11-edge graph, the degrees add to 22, twice the edge count. Breadth-first search from A reaches every node in at most 3 hops, using exactly 7 edges, one fewer than the number of nodes. Graphs appear everywhere in the two majors. A truss is a graph whose nodes are joints and edges are members (Statics, 34/10). A circuit is a graph (the loop equations you meet in Linear Algebra, 34/04, come straight from it). A road map, a network of routers, a dependency chart for building a product and a social network are all graphs, and a file system is a tree (Operating Systems, 34/26). Routers and links are a graph too (Computer Networks, 34/27). Modular arithmetic: clocks, hashes and check digits On a clock, 10 o'clock plus 5 hours is 3 o'clock, not 15. Modular arithmetic wraps numbers around: a mod m is the remainder when you divide a by m. It is used wherever a quantity cycles or has to be squeezed into a fixed number of slots. Try the clock, and below it a tiny hash table: each word is turned into a number, and that number mod the table size picks its slot. Interactive · Clock and hash table The number Clock size m (also the number of table slots) Without JavaScript: 47 mod 8 = 7, because 47 = 5×8 + 7. With 8 slots, adding up each word's letter codes and taking the remainder sends "bolt" and "cam" to the same slot, and "shaft" and "valve" to the same slot: two collisions. Slide the clock size up and the collisions mostly disappear, but by the pigeonhole principle you can't remove them while there are more words than slots. Databases (Databases, 34/28) and compilers use hash tables everywhere. Remainders also catch mistakes. The last digit of a 13-digit book number (ISBN-13) is chosen so that the digits, weighted 1, 3, 1, 3 and so on, add to a multiple of 10. Take 978-0-306-40615-7: the first twelve digits give 93, and 10 − (93 mod 10) = 7. Change any one digit and the total is no longer a multiple of 10, so a scanner rejects it. Most swaps of neighbouring digits are caught too (those that differ by 5 slip through). Cryptography (12/09) is the same arithmetic taken to huge numbers, and it underlies Computer Security (34/33). Engineers meet it in machines. A shaft encoder reports its angle modulo one turn. And gear designers sometimes choose tooth counts with no common factor, such as 20 and 21, so that each tooth meets every tooth on the other gear before any pair repeats: the pattern repeats only after lcm(20, 21) = 420 meshes, against 60 for 20 and 30 teeth, which spreads wear evenly (Machine Design, 34/19). Big-O: growth as a counting idea How long does a program take? Count its steps, as a function of the input size n, and keep only the part that grows fastest: big-O notation. Searching an unsorted list takes about n steps, O(n). Searching a sorted list by repeatedly halving takes about log2n, which is 20 steps for a million items. Comparing every pair is about n². And trying every subset of n things is 2n, which is where counting becomes a wall. Pick a size and a machine speed. The shape of the growth matters more than the speed of the machine. Interactive · How long would it take? Problem size n Machine speed: one billion steps per second (an illustrative round number). Without JavaScript: at n = 1,000,000 an O(n) job takes 0.001 seconds, an n log n job 0.02 seconds, an n² job 1,000 seconds (about 17 minutes), and a 2n job would take far longer than the age of the universe. Even at n = 60, a 2n job needs about 36.5 years at a billion steps per second. That wall is the difference between an algorithm and a method that works in principle. Algorithms (34/24) is a whole course on getting from the right side of this table to the left; Theory of Computation (34/29) asks which problems can never be moved over. What a graduate just sees Any decision, however tangled, is AND, OR and NOT; a processor is a pile of truth tables. When you count two overlapping groups, subtract the overlap once. Order matters means n!/(n−k)!; order doesn't matter means divide by k!. More items than boxes means some box holds two: collisions are guaranteed, not unlucky. Induction is dominoes: base case plus step proves infinitely many cases; a loop invariant is the same. To prove something is impossible, assume it and chase the contradiction. Graphs are dots and links; edge ends add to twice the edges; a tree has one fewer edge than nodes. A remainder is a clock: mod wraps numbers into slots, and a check digit is a remainder that catches typos. Growth shapes (log, linear, n log n, squared, exponential) decide what is feasible far more than hardware does. Where it shows up next Algorithms (34/24) and Data Structures (34/23): induction proves them correct, big-O ranks them, graphs and trees are their raw material. Computer Architecture (34/25): gates, adders and the whole processor are applied logic. Databases (34/28): sets, functions and relations are the model; hashing makes lookups fast. Theory of Computation (34/29): proofs by contradiction show some problems are unsolvable. Statics (34/10), Systems Engineering (34/22) and Machine Design (34/19): trusses as graphs, fault trees as logic, tooth counts as remainders. If you want to go deeper 📘 Kenneth Rosen, Discrete Mathematics and Its Applications (McGraw Hill; 8th edition). The most-assigned university text, with huge problem sets. 📘 Susanna Epp, Discrete Mathematics with Applications (Cengage; 5th edition). Slower and gentler on logic and proof. 📖 Richard Hammack, Book of Proof (free from the author, who is at Virginia Commonwealth University): the gentle route for learning alone, focused on how to write proofs. For a free full text aimed at computer scientists, see the open textbook for MIT's 6.042J below. 🌐 MIT OpenCourseWare 6.042J, Mathematics for Computer Science (Spring 2015): lectures, problem sets and a free textbook by Lehman, Leighton and Meyer. The takeaway Discrete mathematics is exact reasoning about things you can count. Logic turns decisions into circuits, counting tells you what is possible, proof tells you what must be true for every case, graphs model connection, remainders give clocks and check digits, and growth rates tell you what will ever finish. Try this: check an ISBN on the back of a book yourself, weighting its digits 1, 3, 1, 3 and adding them. The total should end in 0. Then in the growth widget above, set n to 40 and then to 60, and watch the 2n job go from minutes to years while the n² job barely moves. Sources & Further Reading Sources. Logic, counting, induction, graphs and modular arithmetic are standard textbook content (Rosen; Lehman, Leighton and Meyer); every number shown was recomputed in code. Boole, An Investigation of the Laws of Thought (1854), and the later history of Boolean algebra: Wikipedia, "The Laws of Thought". Shannon, "A Symbolic Analysis of Relay and Switching Circuits" (MIT master's thesis, 1937). Euclid, Elements IX.20 (D. Joyce's commentary). Dirichlet and the pigeonhole principle (1834): Wikipedia, "Pigeonhole principle". ISBN-13 check digit: Wikipedia, "Check digit" and the ISBN agency rules. Age of the universe, about 13.8 billion years: NASA, from the Planck 2018 results. Go deeper. The books and course above. Step 6. Multivariable Calculus: Calculus on a Landscape ------------------------------------------------------- Why: Fields spread through space, so the maths has to spread too. A raindrop on a hillside picks the steepest way down without thinking. The arrow it follows is the same one that trains every neural network and drives heat through a metal plate. Stand on a hillside with a map. Height depends on two things at once: how far east you are and how far north. Single-variable calculus handles one input at a time. The hillside needs a new question: of all the directions I could step, which one rises fastest? Almost nothing real depends on one number. Temperature in an engine block depends on position in three dimensions. The error of a language model depends on billions of adjustable weights. Air pressure depends on place and time. This course takes the slope-and-area ideas of Calculus I (34/01) and lets them live in more than one dimension. Surprisingly little new is needed: one new arrow, the gradient, and one new accounting rule, the idea that what flows out of a region is what is made inside it. The course in one box Who takes it: second-year Mechanical & Aerospace Engineering students; Computer Science students meet it in year 4 when machine learning needs it. What it is for: calculating change and totals when many things vary at once. Comes after: Calculus I (34/01). Leads to: Physics II: Electricity & Magnetism (34/07), Fluid Mechanics (34/15), Machine Learning (34/32), and through them Heat Transfer (34/16) and Propulsion (34/21). Honest line: you will not be able to grind through a surface integral after this. You will see why a gradient points uphill, why training an AI is "just" rolling downhill, and what the big theorems say in plain words. Two inputs make a landscape, and a partial derivative is one slope A function of two inputs is a surface: height above each point of a map. Engineers draw it the way hikers do, with contour lines: closely spaced lines mean steep ground, widely spaced lines mean gentle ground. To ask "how steep?" you must say in which direction. Walk due east and note the slope: that is the partial derivative with respect to x, written ∂f/∂x. Hold east still and walk north: that is ∂f/∂y. The recipe is simple: treat every other variable as frozen and differentiate as before. ∂f/∂x = slope of f as only x changes, with y held still The landscape below has a tall hill near (1, 1) and a lower one near (−1.2, −0.8). Drag the point on the map (or use the sliders). The cyan arrow shows the slope eastward and the green arrow the slope northward; a leftward or downward arrow means the ground falls in that direction. Interactive · Walk the map East–west position x North–south position y At the starting point (−0.2, 0.3) the height is 0.684, the eastward slope +0.621 and the northward slope +0.226: the ground rises both ways, more steeply towards the east. The gradient points straight uphill Put the two partial slopes together as one arrow, east component first: that is the gradient, written ∇f. It has a beautiful property: it points in the direction of steepest ascent, and its length is how steep that is. Step in any other direction and the slope is the gradient's length times the cosine of the angle between your step and the arrow. Straight along it, cosine is 1: the most you can get. Sideways to it, cosine is 0: you walk along a contour and stay level. slope in direction u = ∇f · u = |∇f| cos(angle) Two consequences follow. The gradient is always perpendicular to the contour line through that point, which is why water runs across contours, not along them. And the same arrow governs heat: in Fourier's law, heat flows down the temperature gradient, from hot to cold, at a rate proportional to its steepness. Many diffusion laws have this shape: flow follows the negative gradient. Interactive · Which way is uphill? Direction you step (0° = east, 90° = north) Turn to the gradient Turn the pink direction until the slope is largest: it lines up with the orange arrow. Turn it 90° from the arrow and the slope is zero. At the very top of a hill the gradient shrinks to nothing, which is the multivariable version of "zero slope means a peak" from Calculus I. Rolling downhill is how a network learns Now flip the sign. A machine-learning model has adjustable numbers (weights), and a loss measures how wrong it is on the training data. The loss is a landscape over the weights, with one dimension per weight. To learn, take a small step against the gradient, so downhill, then recompute and repeat. This is gradient descent, which Augustin-Louis Cauchy proposed in 1847 for solving equations, long before computers. The chain rule from Calculus I is what lets the gradient be computed layer by layer, in the method called backpropagation, published by Rumelhart, Hinton and Williams in Nature in 1986. A model like GPT-3 (2020) has 175 billion weights, so its landscape has 175 billion dimensions, yet the rule is the same arrow. new position = old position − (step size) × ∇(loss) The step size matters, and you can feel why. In the bowl below, one direction is twelve times steeper than the other. Too small a step crawls; just right is fast; too large and each step overshoots the valley floor by more than it started and the loss explodes. For this bowl, steps above 1/6 (about 0.167) diverge. Interactive · Learning rate Too small Good Zig-zag Too big Step size (learning rate) Steps taken Real training uses refinements (momentum, adaptive step sizes, random mini-batches), but all of them are ways of coping with exactly this picture: a long narrow valley where the steepest direction is not the direction of the destination. Double and triple integrals add up a landscape's worth Integrals extend the same way. A single integral adds thin slices along a line. A double integral adds tiny patches over an area: if each patch has a density (kilograms per square metre), the sum is the plate's mass. A triple integral does the same over a solid, with density in kilograms per cubic metre. The recipe is slice, multiply, add, repeated once per dimension. mass = sum over all pieces of (density × piece size)  =  ∬ ρ dA The same sums also give the centroid, the balance point: weight each position by its piece's mass and divide by the total. Engineers use it constantly: the centre of mass of a rocket, a car or an aircraft decides whether it is stable. Try a plate 2 m long and 1 m wide whose density rises along the length. Interactive · Where does the plate balance? Density rises along the length: a (kg per m² per m) Density rises across the width: b The density here is 1 + ax + by kg/m². With a = 3 and b = 0, the plate's mass is 2 + 2×3 = 8 kg and it balances at x = 1.25 m, shifted from the middle (1 m) towards the heavy end. A vector field puts an arrow on every point Wind on a weather map, water in a pipe, the pull of a magnet: each attaches an arrow (a direction and strength) to every point in space. That is a vector field. Two numbers computed from the field's partial derivatives tell you most of what it is doing locally. Divergence: is the field spreading out from here, or converging? A tap under water is a source; a drain is a sink. A positive divergence means a small blob of fluid is expanding. Curl: is the field spinning? Put a tiny paddle wheel in the flow. If it turns, the curl is not zero, and the sign tells you which way. divergence = ∂vx/∂x + ∂vy/∂y    curl = ∂vy/∂x − ∂vx/∂y Pick a field and let a round blob of fluid (grey dashes show where it started) be carried along. You will see that a source inflates the blob without turning it, a whirlpool turns it without changing its area, and a saddle stretches it one way and squeezes it the other while keeping its area. Interactive · Drift a blob through a field Source Whirlpool Shear Saddle Time elapsed Divergence and curl are the two words in Maxwell's equations of electromagnetism (Physics II, 34/07) and in the equations of fluid flow (Fluid Mechanics, 34/15). Spreading and spinning are the two things a field can do locally, besides moving along as a whole. What flows out of a region is what is made inside Here is the idea behind the three big theorems, in the language of a bathtub. If water pours out through the boundary of a region, then some taps inside must be producing it (or the level inside must be falling). Add up every tiny source and sink inside and you get the net flow out through the boundary. Nothing more is needed. net flow out through the boundary = total divergence inside That is the divergence theorem, the three-dimensional version of the statement. Green's theorem is the flat version (the name honours George Green, whose Essay of 1828 contained the related surface-to-volume formula), and Stokes's theorem relates the spinning of a field inside a surface to the flow around its edge. All three say the same kind of thing: a sum over a boundary equals a sum of local changes inside. It is why one can measure a rotating machine from outside and why conservation laws can be written either as "this region's total changes by what crosses its walls" (integral form) or as an equation about each point (differential form). Interactive · Count the flow two ways Source Cubic Source in, sink out Steady wind Circle radius Circle centre, left or right The first number adds up the field's push through the circle itself. The second never looks at the circle's edge: it adds the divergence in every patch inside. They agree for every field, radius and position. For the plain source (x, y), the divergence is 2 everywhere, so a unit circle encloses 2 × π = 6.283 units of production, and exactly 6.283 flows out through its rim. What a graduate just sees A slope needs a direction; a partial derivative is the slope with every other input held still. The gradient points straight uphill, its length is the steepness, and it is perpendicular to the contour. "Downhill" is the whole idea behind training a model: step against the gradient, and mind the step size. Heat, and many other things, flow down a gradient from high to low. A double or triple integral is slice, multiply, add, once per dimension; mass and balance point come from it. A field can only spread (divergence) or spin (curl), locally. What leaves a region through its wall is exactly what its insides produce. Where it shows up next Physics II: Electricity & Magnetism (34/07): Maxwell's equations are statements about the divergence and curl of the electric and magnetic fields. Fluid Mechanics (34/15): conservation of mass is a divergence equation, and the divergence theorem turns "what crosses the walls of this control volume" into "what happens inside it". Heat Transfer (34/16): conduction is heat flowing down the temperature gradient; the heat equation is the gradient and divergence together. Machine Learning (34/32): training is gradient descent on a loss landscape with millions or billions of dimensions. Mechanics of Materials (34/11) and Dynamics (34/12): centroids and moments of inertia are integrals of exactly the kind you added up above. If you want to go deeper 📘 Textbook: Stewart, Clegg and Watson, Calculus: Early Transcendentals (9th ed.), chapters 14 to 16: the standard multivariable text, rich in examples. 📘 Textbook: Jerrold Marsden and Anthony Tromba, Vector Calculus (6th ed., W. H. Freeman): the classic for the geometry of divergence, curl and the big theorems. ▶️ Gentler: Khan Academy's Multivariable Calculus course, which has videos with a strong visual feel, and Grant Sanderson's multivariable lessons. 🌐 Free course: MIT OpenCourseWare 18.02SC Multivariable Calculus, with lectures, notes and exercises. The takeaway With many inputs, a slope needs a direction. The partial derivatives are the slopes along the axes; the gradient joins them into one arrow that points uphill and measures how steep it is. Walk against it and you are doing gradient descent, which trains models with billions of weights and also describes how heat flows. Integrals extend to areas and volumes, giving mass and balance points, and for flows the whole subject rests on one bookkeeping rule: what leaves a region through its boundary is what its interior produces. Try this: on any hill, a park slope or a street, find the line along which a ball would roll: that is the negative gradient. Then notice that the ball's path crosses the contours at right angles. Sources & Further Reading Sources. Gradient descent: A.-L. Cauchy, "Méthode générale pour la résolution des systèmes d'équations simultanées", Comptes rendus 25 (1847), pp. 536–538; Claude Lemaréchal, "Cauchy and the gradient method", Documenta Mathematica, Extra Volume ISMP (2012). Backpropagation: Rumelhart, Hinton and Williams, Nature 323 (1986), pp. 533–536. GPT-3's 175 billion parameters: Brown et al., "Language Models are Few-Shot Learners" (2020, arXiv 2005.14165). Green's Essay of 1828 containing the theorem named after him: MacTutor biography and the essay itself. Computed in node: the landscape f = 2e−((x−1)²+(y−1)²)/1.5 + 1.2e−((x+1.2)²+(y+0.8)²)/1.0 and its analytic gradient (checked against finite differences, agreement to 3 × 10−10); gradient descent on ½(x² + 12y²), stable for step sizes below 2/12; plate mass and centroid by 3,200-cell sums against the exact formulas; blob area against ediv·t; circle flux against the divergence integral for four fields (agreement to 4 decimals). 🌐 MIT OpenCourseWare 18.02SC Multivariable Calculus 🌐 Khan Academy: Multivariable calculus 🌐 Documenta Mathematica, ISMP volume, which contains Lemaréchal's short history of gradient descent 📘 Stewart, Clegg and Watson, Calculus: Early Transcendentals, 9th ed. (Cengage) 📘 Marsden and Tromba, Vector Calculus, 6th ed. Step 7. Differential Equations: Laws Written as Rates ----------------------------------------------------- Why: Capacitors and inductors make every circuit a differential equation. A cup of coffee cools, a capacitor charges, a car bounces over a bump, a fish stock collapses. Each follows a rule that says how fast things change, not where they will be. Solving it is the whole course. Pour coffee and watch it cool. It loses heat fast at first, when it is far hotter than the room, and ever more slowly as it approaches room temperature. Nobody wrote down the temperature at every minute in advance. All you need is a rule: the hotter the cup is above the room, the faster it cools. That sentence, turned into an equation, is a differential equation, and it predicts the whole curve. This is the secret structure of physics. Nature rarely says "the position will be this". It says "the rate of change is that, given the state right now". Newton's second law is such a rule. So are the equations of electrical circuits, chemical reactions, populations, heat, and the flow of air over a wing. A differential equations course teaches how to read these rules and how to turn them into predictions, either exactly or step by step on a computer. The course in one box Who takes it: second-year Mechanical & Aerospace Engineering students. It is not on the Computer Science route in this series, but CS students meet the same ideas whenever they simulate, animate or train anything that evolves in time. What it is for: predicting how a system evolves when you only know the rule for its rate of change. Comes after: Calculus I (34/01). Leads to: Dynamics (34/12), Fluid Mechanics (34/15), Heat Transfer (34/16), Vibrations (34/17), Control Systems (34/20). Honest line: you will not learn the solution tricks that fill the textbook. You will see the three behaviours that recur everywhere (settling, oscillating, tipping) and why a simulation can lie to you. An equation about a rate hands you a direction at every point An ordinary differential equation, an ODE, is an equation that mentions a quantity and how fast it is changing. The simplest useful kind says the rate depends on how far the quantity is from a target: dy/dt = k (target − y) Read it aloud: the further I am from the target, the faster I move towards it. Coffee towards the room's temperature, a capacitor towards the supply voltage, a medicine level towards zero. The picture is a slope field: at every point of the plane the equation fixes the direction a solution must go through, drawn as a tiny segment. A solution is any curve that follows those segments the whole way, like a leaf carried along a stream. Different starting points give different curves, all flowing toward the target. Interactive · Follow the arrows Pull strength k Target value Starting value Notice that the segments get flatter as you approach the dashed line: the rate shrinks with the distance. That is why the curve approaches the target but never quite arrives. The exact solution is the target plus the starting gap times a decaying exponential, e−kt. The time constant: how fast a first-order system forgets The rule dy/dt = k(target − y) has one number that sums it up: the time constant, τ = 1/k. In one τ the system covers about 63.2% of the remaining gap; after about 4.6 τ it has covered 99%. Engineers read response speed from that single number. fraction of the gap covered after time t = 1 − e−t/τ In an electrical circuit, a resistor R in series with a capacitor C charges towards the supply with τ = R × C. A 10 kΩ resistor and a 100 µF capacitor give τ = 10,000 Ω × 0.0001 F = 1 second: after 1 s the capacitor holds 63.2% of the supply (3.16 V of 5 V), after 4.6 s, 99%. A cooling coffee follows the same curve; only the story and the units change. Interactive · Charge a capacitor Resistance R Capacitance C Notice what the sliders do: double either one and the curve stretches twice as wide, but keeps exactly the same shape. The same shape is why a thermometer takes a while to read a fever, and why a control engineer (Control Systems, 34/20) asks first: "what is the time constant?" Solving it step by step: Euler's method Most differential equations have no neat formula. The fall-back is to walk along the slope field in small steps. At the current point read the slope, move a short time h in that direction, and repeat. This is Euler's method, set out by Leonhard Euler in his Institutionum calculi integralis (1768), and it is the ancestor of every physics engine, weather model and circuit simulator. ynext = y + h × (the rate at y) Test it on a rule we can also solve exactly, dy/dt = −y, starting from 1. Each Euler step multiplies the value by (1 − h). Small steps follow the true curve e−t closely. But raise h and something stranger happens. Interactive · Euler versus the exact answer Step size h At h = 1 the first step lands exactly on zero. Beyond 1 the steps overshoot, and past h = 2 each step makes the number bigger in size, so the "answer" explodes even though the real one decays. That is numerical instability: a stepping rule has its own stability limit, and a simulation that ignores it will happily print nonsense. At h = 0.5 Euler gets 0.000244 at t = 6, where the true value is 0.00248: right shape, ten times too small. Smaller steps, or smarter methods (the Runge–Kutta family), fix that. Second order: the spring, the mass and the damper Newton's second law says force equals mass times acceleration, and acceleration is the rate of change of the rate of change of position. So a mass on a spring obeys a second-order equation: the second derivative of position x depends on position and velocity. m x″ + c x′ + k x = 0 The three terms are inertia (mass m), a damper that resists velocity (c) and a spring that pulls back with force proportional to displacement (k). Two numbers decide everything: the natural frequency, ω = √(k/m), and the damping ratio, ζ = c ÷ (2√(km)). ζ below 1, underdamped: it rings, swinging past rest and fading. ζ = 1, critically damped: the fastest return without overshoot. ζ above 1, overdamped: no swing, but a sluggish crawl home. Engineers tune for it. A door closer is aimed near critical damping so the door shuts without banging or bouncing. A shock absorber is a compromise: enough damping to stop the bouncing, not so much that the wheel can't follow the road. Interactive · Tune the damper No damping Light Critical Heavy Mass m Spring stiffness k Damping c With m = 1 kg and k = 25 N/m the natural frequency is 5 rad/s, or about 0.8 Hz. Setting c = 10 N·s/m gives ζ = 1, critical. With c = 2, ζ = 0.2 and the first swing past rest reaches 52.7% of the starting displacement on the other side. This same equation, with different symbols, describes the RLC circuit, a building in an earthquake and a robot arm: that is the next courses (Vibrations, 34/17, and Control Systems, 34/20). Equilibria: where things settle, and where they tip Often you do not need the whole curve, only the places where change stops: the equilibria, where the rate equals zero. Two questions follow. Where are they? And is each one stable (nudge it and it comes back) or unstable (nudge it and it runs away)? The answer is the slope of the rate curve at that point, from Calculus I: negative slope means stable, positive means unstable. A toy model shows why this matters. Fish regrow according to dy/dt = y(1 − y/100) per year (a standard logistic model), and a fleet takes a fixed catch H every year. For H = 20 there are two equilibria, at about 27.6 and 72.4 fish: the upper is stable, the lower unstable. Start just above 27.6 and the stock recovers to 72.4. Start just below it and the stock collapses to zero in a few years. Raise the catch above 25, the most the stock can regrow, and no equilibrium remains at all. Interactive · A fishery on the edge Catch 20, start 30 Catch 20, start 25 Catch 26 Fixed catch H per year Starting population The lesson generalises: a system can look healthy while it sits near an unstable tipping point, and a slow change in a parameter (here the catch) can make the stable state vanish suddenly. Bridges, power grids, aircraft and ecosystems are all analysed this way. The model is a classroom toy, not a description of any real fishery. Why physics is written in differential equations Why do laws come as rates? Because change is local. What a system does next depends only on what is happening here, now: the forces, the gradients, the current state. Newton's second law (second order in position), Maxwell's equations for electric and magnetic fields, the equations of heat flow and the equations of fluid motion all have this shape. Give the rule and the starting conditions and the future follows. Two kinds appear. An ordinary equation (ODE) has one changing variable, usually time: a pendulum, a circuit, a rocket's trajectory. A partial differential equation (PDE) has several, such as time and position: the temperature along a rod, the pressure in the air around a wing. The Navier–Stokes equations of fluid flow (Fluid Mechanics, 34/15) are PDEs so hard that the Clay Mathematics Institute named a question about them one of its seven Millennium Prize problems in 2000. Most real ones are solved on computers using the stepping idea above. A glimpse of PDEs: the heat equation Joseph Fourier wrote the heat equation in his Théorie analytique de la chaleur (1822). It says: the temperature at a point changes at a rate proportional to how much its surroundings, on average, are hotter or cooler than it is, which is the curvature of the temperature curve. ∂u/∂t = α ∂²u/∂x² A peak has negative curvature, so it cools; a dip has positive curvature, so it warms. Heat flows from hot to cold and the profile smooths out. The rod below is insulated at both ends, so the average temperature, the total heat, never changes, only its distribution. Fourier's great discovery (see the library's piece on the Fourier transform, 12/12) is that smooth wiggles of different sharpness die away at different rates: a wiggle five times sharper dies 25 times faster, because it decays like e−(n²π²)t. Interactive · Heat spreading along a rod Hot spot Half hot Gentle wave Sharp wave Time Time step, as a fraction of the stability limit Try the two waves at the same time. The gentle wave 1 + 0.8 cos(πx) has faded to 0.298 high at t = 0.1, matching the exact 0.8e−π²t. The sharp wave 1 + 0.8 cos(5πx) is gone almost at once, 25 times faster in the exponent. Then drag the time-step slider above 0.5 and watch the simulation explode: Euler's lesson again, now in a PDE. Real heat solvers (Heat Transfer, 34/16) must respect the same limit or use methods built to avoid it. What a graduate just sees A law about a rate is a direction at every point; a solution is a curve that follows those directions. First-order systems approach their target by a time constant: 63% in one τ, 99% in about 4.6 τ. Cooling, charging and decay are the same curve with different labels. A step-by-step simulation has a stability limit: too big a step creates growth that is not in the physics. Mass, spring and damper give three behaviours: ring, crawl, or the sweet spot at ζ = 1. Equilibria matter, and slope tells stable from unstable. Systems tip. Sharp features smooth out first in diffusion, and total heat is conserved. Where it shows up next Dynamics (34/12): every motion problem becomes an equation of the form above. Vibrations (34/17): the damped spring is the building block of everything that shakes, from engines to aircraft wings. Control Systems (34/20): a thermostat, cruise control or autopilot is a differential equation you shape with feedback; time constants and damping ratios are its vocabulary. Heat Transfer (34/16) and Fluid Mechanics (34/15): the heat equation and Navier–Stokes are PDEs; the stepping and stability ideas carry straight over. Computer science: physics engines in games, climate and weather models, and circuit simulators are loops of Euler-like steps; neural ODEs even treat a network as a flow. If you want to go deeper 📘 Textbook: William Boyce, Richard DiPrima and Douglas Meade, Elementary Differential Equations and Boundary Value Problems (12th ed., Wiley): the classic course text. 📘 Textbook: Steven Strogatz, Nonlinear Dynamics and Chaos (3rd ed., CRC Press): an unusually readable treatment of equilibria, stability and tipping. ▶️ Gentler: 3Blue1Brown's differential equations series, and Strogatz's Infinite Powers for the story of why calculus describes nature. 🌐 Free course: MIT OpenCourseWare 18.03SC Differential Equations, with video lectures, notes and problem sets. The takeaway Physics speaks in rates. A differential equation gives a direction at every point and a solution follows it. First-order systems approach a target with a time constant (63% in one τ). Second-order systems, from springs and masses, ring, crawl or sit at the sweet spot of critical damping. Equilibria are where change stops and slope tells stable from unstable. When no formula exists, you step along on a computer, but only small enough steps tell the truth: past a limit, a simulation blows up even though nature does not. Try this: take a hot drink and a thermometer (or just your hand) and note how long it takes to lose half its excess heat over the room. It will take about the same time again to lose half of what is left. That constant "half-life" is the time constant in disguise. Sources & Further Reading Sources. Euler's method: Institutionum calculi integralis, vol. 1 (1768); Encyclopedia of Mathematics, "Euler method". Newton's law of cooling originates in Newton's anonymous 1701 paper "Scala graduum caloris"; the modern differential form is not in the original (Wikipedia, "Newton's law of cooling", and history-of-science references). Fourier's Théorie analytique de la chaleur (1822). Millennium Prize problems announced by the Clay Mathematics Institute in 2000, Navier–Stokes among them. Computed in node: RC charging (τ = RC = 1 s for 10 kΩ and 100 µF, 63.2% at τ, 99.0% at 4.605τ); Euler at h = 0.5 gives 2.44 × 10−4 at t = 6 against the exact 2.48 × 10−3; spring closed-form solutions for the three regimes checked against a step-by-step integration (agreement to 5 decimals) and the first-swing overshoot e−πζ/√(1−ζ²) = 52.7% for ζ = 0.2; harvested logistic equilibria 50(1 ± √(1 − H/25)) = 27.64 and 72.36 for H = 20, with collapse after 4.3 years from a start of 25; the explicit heat scheme (100 cells, stable below 0.5) against the exact decay 0.8e−π²t = 0.2982 (numerical 0.2981) at t = 0.1. 🌐 MIT OpenCourseWare 18.03SC Differential Equations ▶️ 3Blue1Brown: differential equations 📘 Boyce, DiPrima and Meade, Elementary Differential Equations and Boundary Value Problems, 12th ed. (Wiley) 📘 Steven Strogatz, Nonlinear Dynamics and Chaos, 3rd ed. (CRC Press) 🌐 Newton's law of cooling (with the history of the law) Step 8. Physics II: Electricity & Magnetism — Charge, Fields and Light ---------------------------------------------------------------------- Why: Charge, fields, induction and light: the physics the whole degree stands on. A rubbed balloon beats the whole Earth's gravity, electrons crawl through your wiring slower than a snail, and light turns out to be a field vibrating. The one course that explains your phone, your motor and your Wi-Fi. Rub a balloon on your hair and stick it to the wall. It hangs there. Gravity, the pull of an entire planet, is trying to drop it, and a few billion stray electrons on its surface are winning. Between a proton and an electron the electric attraction is about 2.3 × 10³⁹ times stronger than their gravitational attraction. Gravity only seems strong because the Earth is enormous and its charge is balanced out. The second surprise is quieter. When you flip a switch, the light comes on at once, yet the electrons in the copper behind the wall drift at a fraction of a millimetre per second. What travels at nearly light speed is not the electrons. It is a field. Physics II is the course where those two sentences stop being puzzling. The library has an easier first read, How Electricity Reaches Your Wall (13/04), about the grid and Ohm's law. This piece goes further: into fields, capacitors, induction, and Maxwell's discovery that light is made of the same stuff. The course in one box Who takes it: second-term first-year or second-year students of Mechanical & Aerospace Engineering and, in many programmes, Computer Science. What it's for: predicting what charges and currents do, from a battery to a radio wave. Comes after: Physics I: Mechanics (34/06) and Multivariable Calculus (34/02). Leads to: the motors, sensors and actuators of Mechanical & Aerospace Engineering, then Control Systems (34/20); for Computer Science, Computer Architecture (34/25) and Computer Networks (34/27), which are electricity at tiny and large scales. Honest line: you won't design a power supply after this. You will know why a thin wire gets hot and a thick one doesn't, and why a transformer hums. Charge and the electric field Matter carries electric charge, positive or negative. Like charges push apart and opposite charges pull together, with a force that falls with the square of the distance, the same shape as gravity. A charge doesn't need to "reach" another: it fills the space around it with a field, and any other charge placed there feels it. E = F / q = k · Q / r²the field is the force a small test charge would feel, per unit of its charge (k ≈ 8.99 × 10⁹ N·m²/C²) Worked number: 1 nC (a billionth of a coulomb) makes a field of 900 V/m at 10 cm. Drag the gold probe below. Put a positive and a negative charge side by side and the field curves from one to the other. Interactive · Drag the probe through the field A positive and a negative Two positives One alone Left charge Right charge Charges are 30 cm apart; the grid shows field direction. Equal and opposite charges: the voltage along the midline between them is exactly 0. Voltage: energy per charge Lift a book and it gains energy; the same book on a higher shelf has more. Voltage is the electrical version of height: the energy a charge gains per coulomb moved. A 1.5 V battery gives every coulomb that crosses it 1.5 joules, whatever the battery's size. V = energy ÷ charge  ·  1 volt = 1 joule per coulomban electron falling through V volts gains eV of energy That makes electrons easy to launch. Accelerate one through 100 V and it leaves at about 5.9 × 10⁶ m/s, two per cent of the speed of light. At 10,000 V its speed is 0.195 c, and the ordinary formula (½mv²) overestimates it by about 1.5%. The old cathode-ray TV worked this way, aiming such electrons at a screen. Interactive · An electron falls through a battery Voltage it falls through Computed with the relativistic energy formula, and compared with the ordinary one. Current and resistance: the slow river Current is charge flowing past a point each second (1 ampere = 1 coulomb per second). Push a voltage across a wire and a current flows; the ratio is the resistance. For most metals it's constant, which is Ohm's law, and a wire's resistance grows with its length and shrinks with its thickness. Resistance turns electrical energy into heat at a rate P = I²R. V = I · R  ·  R = ρ · L / A  ·  P = I²Rρ is the material's resistivity: for copper about 1.68 × 10⁻⁸ Ω·m Now the surprise. The electrons' average drift speed is I ÷ (n·A·e), with n about 8.5 × 10²⁸ free electrons per cubic metre of copper. Ten amps through a 2.5 mm² house wire gives a drift of 0.29 mm/s. An electron takes nearly an hour to cover one metre. The lamp lights at once because the wire is already full of electrons and the field pushes them all together, like water in a pipe that is already full. Interactive · A copper wire under load Current Wire cross-section Wire length With those defaults the wire wastes 13 W as heat. Halve the thickness and the heat doubles. That is why extension cords are rated by thickness, and why power lines run at very high voltage: a given power needs a smaller current, and heat goes as current squared. Circuits: two rules Every circuit, however tangled, obeys two bookkeeping rules from Gustav Kirchhoff. The junction rule: current flowing into a junction equals current flowing out, because charge doesn't pile up. The loop rule: going around any closed loop, the voltage rises and drops add to zero, because energy is conserved. They are conservation of charge and of energy, again. From them follow the two shapes. Resistors in series share one current, so their resistances add. Resistors in parallel share one voltage, so their currents add, and the total resistance gets smaller as you add branches. That's why plugging more appliances into one socket pulls more current from the wall, not less. Interactive · Three resistors, a 12 V battery All in series All in parallel R1, then R2 ∥ R3 Battery R1 R2 R3 With 100, 200 and 300 Ω: in series the battery supplies 20 mA; in parallel the total resistance falls to 54.5 Ω and the current rises to 220 mA, eleven times as much. Capacitors and the RC clock A capacitor is two plates close together that store charge, and with it energy in the field between them. Its capacity C is how much charge it holds per volt. Connect one through a resistor to a battery and the plates fill, but ever more slowly, because the more it holds the less voltage is left to push more in. V(t) = V₀ (1 − e−t/RC)  ·  τ = R · Cthe time constant τ sets the pace; the exponential is the solution of Differential Equations (34/03) After one τ the capacitor is 63% full, after three τ 95%, after five 99%. A 10 kΩ resistor with a 100 µF capacitor makes τ = 1 second. Camera flashes, phone touch screens, power-supply smoothing and the blink of a turn signal all use this clock. Interactive · Charge it, discharge it Charging Discharging Resistor Capacitor Time elapsed A 5 V supply. Magnetic fields: the push on moving charge A current makes a magnetic field that circles the wire. Ten amps at 5 cm makes about 40 µT, the same size as Earth's own field (about 25 to 65 µT at the surface). A magnetic field pushes only on moving charge, and sideways: perpendicular to both the motion and the field. A sideways push with constant speed is the circular motion of Physics I. F = q · v · B  ⇒  r = m · v / (q · B)the radius is bigger for faster or heavier particles, smaller for stronger fields A wire carrying current in a field gets the same push: that is a motor. And a particle's lap time, 2πm/(qB), doesn't depend on its speed, which is how cyclotron accelerators work. Interactive · A charge in a magnetic field Electron Proton Speed Field Induction: change a field, get a voltage In 1831 Michael Faraday showed that a changing magnetic field through a loop of wire drives a voltage around it. A steady field does nothing; only change counts. Faster change, more loops, and a stronger field all give a bigger voltage. ε = − N · dΦ/dtvoltage = number of turns × how fast the magnetic flux changes; the minus sign says the induced current opposes the change This one rule explains three machines. Spin a coil in a field and you have a generator: the voltage rises and falls as a sine wave, peak N·B·A·ω. Run the same machine backwards and you have a motor. Wind two coils on one iron core and an alternating current in the first induces an alternating voltage in the second, with output voltage in the ratio of the turns: a transformer. It works only with alternating current, since only change induces, and it is why the grid uses AC and why a charger hums. Interactive · Generator, then transformer Turns in the coil Magnet field Spin speed Transformer input Output coil (input has 1,000) Coil area 10 cm × 10 cm. Ideal transformer, no losses. At the defaults the generator gives 31.4 V peak at 10 Hz, and the transformer steps 240 V down to 12 V: 20 times fewer turns, 20 times the current. Maxwell: four rules and the speed of light In the 1860s James Clerk Maxwell gathered everything known into four equations. In words: (1) electric charges make electric fields that spread outward; (2) there are no isolated magnetic charges, so magnetic field lines never end; (3) a changing magnetic field makes a circulating electric field (Faraday); (4) an electric current, and, Maxwell's own addition, a changing electric field, make a circulating magnetic field. Put (3) and (4) together and something remarkable happens: a changing electric field makes a changing magnetic field, which makes a changing electric field, and the disturbance can sustain itself and travel. Maxwell calculated its speed from two measured constants of electricity and magnetism, 1/√(μ₀ε₀), and got the measured speed of light, 299,792,458 m/s. In his 1865 paper he concluded that light is such a wave. Heinrich Hertz produced and detected radio waves in the late 1880s, and the rest of the spectrum followed. c = 1 / √(μ₀ ε₀) = 299,792,458 m/sthe speed of every electromagnetic wave in empty space Radio, microwaves, infrared, visible light and X-rays differ only in frequency, and the wavelength is c ÷ f. A 100 MHz FM station has a wavelength of 3 m; Wi-Fi at 2.4 GHz, 12.5 cm; red light at 700 nm, 430 THz. This also explains antenna sizes: a quarter wavelength works well, so an FM antenna is about 75 cm. Interactive · Slide along the spectrum Mains 60 Hz AM radio FM radio Wi‑Fi Red light Frequency (log scale) What a graduate just sees Electric force is absurdly strong; things look neutral because charges balance. A voltage is energy per charge; a current is charge per second; power is their product. A thin wire heats up because P = I²R, and that's why the grid uses high voltage. Electrons drift slowly; the signal is the field, and it moves near light speed. Series adds resistance, parallel adds current. Any capacitor and resistor make a clock, and 63% in one τ is the number to remember. Magnetism acts only on moving charge, and always sideways. Only change induces: no change in the field, no voltage. So transformers need AC. Light, radio and X-rays are the same wave at different frequencies. Where it shows up next Motors, generators and sensors in Mechanical & Aerospace Engineering are the F = qvB and induction sections above; Control Systems (34/20) then closes the loop around them. Vibrations (34/17) and Differential Equations (34/03): the RC curve is the same equation as a cooling cup of coffee, and an inductor with a capacitor oscillates like a mass on a spring. Computer Architecture (34/25): a logic gate is a switch, and the delay of a wire is an RC time constant, which sets how fast a chip can be clocked. Computer Networks (34/27): Wi-Fi and cellular are Maxwell's waves with data on them; the wavelength rule sets antenna size. Thermodynamics (34/14) picks up the I²R heat; a chip's heat is why laptops have fans. If you want to go deeper 📘 Standard textbooks. Halliday, Resnick & Walker, Fundamentals of Physics (12th ed.) or Young & Freedman, University Physics (16th ed.), the usual engineering-year texts. Purcell & Morin, Electricity and Magnetism (3rd ed., Cambridge, SI units) is the elegant one for stronger students. 📘 Gentler, free. The Feynman Lectures on Physics, Volume II, free at feynmanlectures.caltech.edu. 🌐 Free course. MIT OpenCourseWare 8.02 Physics II: Electricity and Magnetism (Spring 2007), with lecture notes and problem sets. The takeaway Charges make fields; a voltage is energy per charge; a current is charge per second, resisted into heat. Two rules, conservation of charge and of energy, solve any circuit. A changing magnetic field makes a voltage, which gives generators, motors and transformers, and Maxwell showed that a changing electric field and a changing magnetic field can sustain each other as a wave travelling at the speed of light. That wave is light. Try this: look at the charger of a laptop or phone. Find the voltage and amperes printed on it, multiply them, and you have the power it can deliver. Then feel how warm the cable and the brick get, and work out which part is I²R. Sources & Further Reading Sources. Constants from the NIST CODATA 2022 listing (elementary charge 1.602 176 634 × 10⁻¹⁹ C and speed of light 299 792 458 m/s are exact; electron and proton masses; G; μ₀; ε₀). Copper's resistivity (1.68 × 10⁻⁸ Ω·m at 20 °C) from the tabulation in Wikipedia's "Electrical resistivity and conductivity"; free-electron density 8.5 × 10²⁸ m⁻³ and the drift-velocity worked example from Wikipedia's "Drift velocity". Earth's field 25–65 µT: Wikipedia's "Earth's magnetic field". Faraday's 1831 demonstration: Wikipedia's "Faraday's law of induction" (Joseph Henry independently, 1832). Maxwell's paper read to the Royal Society 8 December 1864 and published 1865: Wikipedia's "A Dynamical Theory of the Electromagnetic Field". Hertz produced and detected radio waves in 1886 and published his evidence in 1887–88: Wikipedia's "Heinrich Hertz". Computed in node and checked: electric to gravitational force ratio 2.27 × 10³⁹; drift speed 0.294 mm/s and 13.4 W; electron speeds (5.93 × 10⁶ m/s at 100 V; 0.195 c at 10 kV); 54.5 Ω and 220 mA; 63.2%, 95.0%, 99.3% RC fractions; electron orbit radius 5.7 cm and period 35.7 ns at 10⁷ m/s and 1 mT; 31.4 V peak; 1/√(μ₀ε₀) = 299,792,458 m/s to 8 digits. 🌐 MIT OpenCourseWare: 8.02 Physics II, Spring 2007 📘 Purcell & Morin, Electricity and Magnetism, 3rd ed. (Cambridge University Press, 2013) 📘 The Feynman Lectures on Physics, Vol. II (free online, feynmanlectures.caltech.edu) 🌐 NIST: fundamental physical constants (CODATA 2022) Step 9. Circuit Analysis: Why the Lights Dim When the Kettle Starts ------------------------------------------------------------------- Why: Ohm, Kirchhoff and the tricks that turn any circuit into a few numbers. Ohm, Kirchhoff and a handful of tricks turn any tangle of wires into a few numbers. Why chargers warm up, why supply wires matter, and why a radio can pick one station. You switch the kettle on and, for an instant, the kitchen lights dip. Nothing is wrong. The kettle draws about 10 amps, those amps flow through the same wires that feed the lights, and every wire has a little resistance, so a few volts quietly go missing on the way. The lights see the leftover. Feel your phone charger after an hour: warm. Some of the energy it handles never reaches the battery; it turns into heat in the charger's own resistances. If a 20 W charger is 90% efficient, 2 W of heat is the price. Both mysteries have the same answer, and it fits on a postcard. That postcard is circuit analysis. The course in one box Who takes it: every electrical and computer engineer, usually in year 2, and many mechanical and aerospace students in a shorter form. What it's for: predicting the voltage and current anywhere in a circuit, before building it. Comes after: Physics II: Electricity & Magnetism (34/07) and Differential Equations (34/03). Leads to: Electronics (34/35), Signals & Systems (34/37) and Power Systems & Electric Machines (34/39). Computer scientists meet it again in Computer Architecture (34/25), where every gate is a circuit. Honest line: you won't be able to design a power supply after this. You will see why a thin wire carrying a lot of current gets hot, why a sensor reading sags when you connect it to something, and why every radio has a coil and a capacitor in it. Ohm and the heat in a wire Picture water in a pipe. Voltage is the pressure pushing, current is how much flows past a point each second, and resistance is how narrow and rough the pipe is. A narrow pipe needs more pressure for the same flow. Georg Ohm published in 1827 that for metals the relation is proportional. V = I × Rvolts = amps × ohms P = V × I = I² × Rpower in watts: what the voltage does to the current, and what current does to a resistance The second line is the one a graduate carries: heat in a resistance grows with the square of the current. Double the current and you get four times the heat. A 60 W load at 5 V needs 12 A. Push that through a cable of 0.2 Ω and the cable burns 12² × 0.2 = 28.8 W, nearly half the power you meant to deliver. The same 60 W at 20 V needs 3 A and wastes 3² × 0.2 = 1.8 W, 3%. This is why fast chargers negotiate their voltage upward: a fixed power, a lower current, less heat in the cable. Interactive · Same power, different voltage Power the load needs Cable resistance (there and back) 60 W at 5 V needs 12 A and the cable burns 28.8 W. Here the voltage at the load is 5 V; the cable heat is I² × R. The last line of the readout shows the same job at 9, 12, 20 and 48 V. Kirchhoff's laws, series and parallel, and the divider Gustav Kirchhoff published two rules in 1845, and every circuit calculation is built from them. Both are bookkeeping. Current law: charge does not pile up at a junction, so the current in equals the current out. Voltage law: go once around any loop and the voltage rises and falls add up to zero, because a point can only have one voltage. Two resistors in series share one current, so their resistances add and the voltage splits in proportion. Two in parallel share one voltage, so the currents add, and the combined resistance is always smaller than either. Series is a queue; parallel is extra checkout lanes. Vout = Vin × R2 ÷ (R1 + R2)the voltage divider: the share of the voltage is the share of the resistance With 5 V across two equal 10 kΩ resistors, the middle is 2.5 V. That is how a sensor, a volume knob or a battery-level gauge turns a resistance into a voltage. Now the catch every beginner meets once: connect something to the middle and it joins in parallel with R2. A 10 kΩ load turns the lower half into 5 kΩ, and the output falls to 1.67 V, a third lower. Measuring or loading a node changes it. Interactive · The divider that sags R1 (top) R2 (bottom) Load on the output (far right = none) Unloaded: 2.5 V. Input 5 V. Slide the load toward R2's size and watch the second bar sink. A load ten times larger than R2 costs only a few percent. Node analysis: one equation per junction Real circuits are not one divider. For the rest, the method is mechanical and it is the one every program uses. Pick a ground, call it 0 V. Give every other junction a voltage, an unknown. Then write Kirchhoff's current law at each: the currents leaving through every resistor add to zero (or to whatever a current source pushes in). Each current is a voltage difference divided by a resistance, so every equation is linear. That gives one equation per unknown, which is a system you solved in Linear Algebra (34/04). Fifty nodes means fifty equations in a matrix, and the circuit simulator that designs a chip solves exactly that, millions of times. The matrix is symmetric, with each node's total conductance on the diagonal and the minus-conductances between neighbours off it: you can write it by looking at the picture. G × v = iconductances (1/R) times node voltages = injected currents In the circuit below, with Vs = 12 V and a 2 mA source, the two equations (in mS and mA) are 2A − 0.5B = 12 and −0.5A + 1.5B = 2. They give A = 6.91 V and B = 3.64 V. The readout checks that the currents leaving each node cancel. Interactive · Two nodes, two equations Source voltage Vs Current source Is into node B Bridge resistor R3 R1 = 1 kΩ, R2 = 2 kΩ, R4 = 1 kΩ. Make R3 tiny and A and B become one node; make it huge and they separate. Thévenin, Norton and maximum power Here is the idea that makes big circuits manageable. Take any pile of sources and resistors with two terminals sticking out. From outside, the whole pile behaves like one ideal voltage source Vth in series with one resistance Rth. Léon Thévenin published this in 1883 (Hermann von Helmholtz had it earlier, in 1853), and the Norton version, a current source in parallel, is the same thing turned over. You find Vth by asking what the terminals read with nothing attached, and Rth by asking how much the voltage falls when you draw current. The wall socket is the clearest example. Its Thévenin voltage is 230 V (or 120 V) and its Thévenin resistance is the wiring back to the transformer, plausibly a few tenths of an ohm. Flicker standards test appliances against a reference loop impedance of about 0.4 Ω plus a small reactive part (IEC 61000-3-3, as listed by test-equipment makers); here we use 0.5 Ω. A 2.4 kW kettle is 22 Ω, so it draws 230 ÷ 22.5 = 10.2 A. The share of the 230 V that reaches the kettle is 22 ÷ 22.5, so the kettle sees about 225 V and everything else on that wire loses the missing 5 V while it runs. That is the dimming lights. There is a second lesson hiding here. How much power can a source with internal resistance give to a load? Too small a load and the current is large but the load's voltage is tiny; too large and the voltage is high but the current vanishes. The best is the middle: Pmax = Vth² ÷ (4 Rth) when RL = Rthmaximum power transfer: match the load to the source At that match half the power is wasted inside the source. A radio receiver or a speaker amplifier chasing every last bit of a weak signal accepts this trade. A power station never does: it wants a load far larger than its own resistance so that almost nothing is lost, which is also why the mains is built with the lowest resistance the money allows. Interactive · Match the load, or don't Kettle on the mainsOld battery, matched Open-circuit voltage Vth Source resistance Rth Load resistance (relative to Rth) The curve is power as a share of the maximum, against the load size from one hundredth to one hundred times Rth. Capacitors, inductors and the RC transient Resistors have no memory. A capacitor stores charge, and its current is proportional to how fast its voltage changes (I = C dV/dt). An inductor stores energy in a magnetic field, and its voltage is proportional to how fast its current changes (V = L dI/dt). Put one of them with a resistor and Kirchhoff's law becomes a differential equation, the first one you solved in Differential Equations (34/03). The answer is the exponential everyone learns once: v(t) = V × (1 − e−t/τ), τ = R × Ccharging an RC circuit; for a coil and a resistor, τ = L ÷ R The capacitor reaches 63% of the way in one τ, 95% in three, 99% in five. A 10 kΩ resistor with 100 µF has τ = 1 second. A capacitor charged to 5 V stores ½CV² = 1.25 mJ, and the resistor turns exactly as much into heat while charging it, whatever the resistance. That half-and-half loss is also why switching chips cost power (see Electronics, 34/35). Feed the same RC a square wave and it becomes a filter. When τ is much shorter than a half-period the output follows the wave. When τ is much longer the capacitor never catches up and you get a nearly flat level: slow changes pass, fast ones are averaged away. Interactive · Charging, and a filter Step chargeSquare wave Resistance R Capacitance C Time since the switch closed Square-wave frequency Input is 5 V. In square-wave mode the output settles to a steady swing between V·a/(1+a) and V/(1+a), where a = e−half period ÷ τ. AC, phasors and resonance The mains and every radio signal are sine waves, and a sine wave through an RLC circuit comes out a sine wave of the same frequency, just scaled and shifted in time. That changes the problem: you no longer need a differential equation, only the size and the shift. Engineers write that pair as a single arrow, a phasor, and the arrow's behaviour has a name, impedance. ZR = R, ZL = jωL, ZC = 1 ÷ (jωC)the j means "a quarter turn": current in a coil lags, in a capacitor it leads Every resistor, coil and capacitor now obeys Ohm's law with a complex number. A coil's reactance ωL grows with frequency; a capacitor's 1 ÷ ωC shrinks. In a series RLC they point in opposite directions, so at one frequency they cancel completely. There the impedance is only R, the current peaks, and the circuit rings at its natural frequency: f0 = 1 ÷ (2π √(L C)), Q = (1 ÷ R) √(L ÷ C)resonance, and the quality factor: how sharp the peak is (bandwidth ≈ f0 ÷ Q) With L = 10 mH and C = 1 µF, f0 is 1,592 Hz. Make R only 10 Ω and Q is 10: the circuit answers loudly within about 160 Hz of that note and ignores everything else. This is a radio tuner, and it is also a filter: take the output across the resistor and you have a band-pass (the tuner); across the capacitor, a low-pass; across the coil, a high-pass. It is the mechanical mass-and-spring of Vibrations (34/17) in another medium, with Q playing the part of the damping ratio. Interactive · Tune the circuit Across R (band-pass)Across C (low-pass)Across L (high-pass) Jump to f0LowHigh Frequency Resistance R Fixed: L = 10 mH, C = 1 µF. In the phasor picture VR points right (in phase with the current), VL up, VC down; the dashed arrow is the source. At f0 the up and down arrows are equal and the dashed arrow lies flat. What a graduate just sees Heat is I² × R. Doubling the current quadruples the loss, so power travels at high voltage and low current. A source is never ideal: every battery, socket and sensor has an internal resistance, so loading it lowers its voltage. Series shares current, parallel shares voltage; parallel resistance is always smaller than the smallest branch. A circuit with n unknown voltages is n linear equations: write conductances down the diagonal. Any two-terminal mess is a Thévenin source. Matching the load to it gives maximum power and 50% efficiency. RC means τ = RC: 63% in one, 95% in three, 99% in five time constants. A coil and a capacitor trade energy back and forth at f0; Q says how long it rings and how narrow the filter is. Impedance is resistance that depends on frequency, which is why the same circuit can pass bass and block treble. Where it shows up next Electronics (34/35): diodes and transistors are added to the toolbox; the analysis is the same, with an extra source whose value depends on a voltage somewhere else. Signals & Systems (34/37): the phasor idea, one number for size and shift, grows into the Fourier and Laplace transforms; an RC filter is the first example of a system. Power Systems & Electric Machines (34/39): three-phase power and power factor are phasors with a bigger vocabulary; the kettle's voltage dip is how a grid engineer thinks about weak versus strong supplies. Vibrations (34/17) and Control Systems (34/20): the RLC circuit is the same differential equation as the mass-spring-damper, so what you learned there transfers in both directions. The real world: phone chargers, speaker crossovers, guitar tone knobs, radio tuners, the pull-up resistor next to every microcontroller pin, and the fuse in your plug. If you want to go deeper 📘 Textbook: James Nilsson and Susan Riedel, Electric Circuits (Pearson; 12th edition): a long-standing classroom text, thorough on node and mesh methods. 📘 Textbook: Charles Alexander and Matthew Sadiku, Fundamentals of Electric Circuits (McGraw-Hill): many worked examples, widely used internationally. 📘 Gentler: Paul Horowitz and Winfield Hill, The Art of Electronics (Cambridge University Press; 3rd edition, 2015), whose first chapters are the clearest practical treatment of these ideas. Also Ben Eater's free video series on building circuits from scratch. ▶️ Free course: MIT OpenCourseWare 6.002 Circuits and Electronics (Spring 2007): the course the electrical engineering major usually opens with. 🌐 Easier first read in this library: Physics II: Electricity & Magnetism (34/07) for what charge and voltage are. The takeaway Circuit analysis is bookkeeping with two laws: charge does not pile up at a node, and voltage adds to zero around a loop. From them come dividers, node equations and the Thévenin trick that reduces any two-terminal mess to a source and a resistor. Heat grows with the square of the current, which is why power moves at high voltage and why the lights dim when the kettle starts. Add a capacitor and the answer is an exponential with a time constant; add a coil and a sine wave and it is impedance, resonance and filters. Try this: next time a charger or power brick feels warm, touch the cable too. Then watch a lamp on the same circuit as a kettle, a hair dryer or a toaster when it switches on. If you can see a dip, you have just measured a Thévenin resistance with your eyes. Sources & Further Reading Sources. Ohm's law, Kirchhoff's laws (1845), node analysis, Thévenin's and Norton's theorems, RC and RLC behaviour are standard textbook material (Nilsson and Riedel; Alexander and Sadiku; MIT 6.002). The usual textbook attributions are used: Ohm 1827, Kirchhoff 1845, Helmholtz 1853, Thévenin 1883. The supply reference impedance (about 0.4 Ω + j0.25 Ω for a 230 V single-phase test) is from IEC 61000-3-3 as listed by test-equipment makers (Kikusui and Pacific Power datasheets); other documents list different network values, so the 0.5 Ω used here is illustrative. Computed in node: every number in the readouts and worked examples (the 28.8 W and 1.8 W cable losses, the 2.5 V and 1.67 V divider, node voltages 6.909 V and 3.636 V, the 225 V kettle, maximum power and 50% efficiency, 3.11 V and 1.89 V RC swing, f0 = 1,591.5 Hz and Q = 10). 🌐 MIT OpenCourseWare 6.002 Circuits and Electronics 🌐 Ben Eater: building circuits and computers from scratch 📘 Nilsson and Riedel, Electric Circuits 📘 Horowitz and Hill, The Art of Electronics Step 10. Digital Logic Design: Gates That Decide, Flip-Flops That Remember -------------------------------------------------------------------------- Why: Gates, flip-flops and state machines: how a chip makes decisions and remembers. Every digital chip is built from two kinds of circuit: ones that decide from what they see now, and ones that remember what happened before. Learn to tell them apart and you can read a crossing light, a counter or a processor. Press the button at a pedestrian crossing. Nothing happens. The cars keep going, and you wait, and then some seconds later the light changes for you. Between your press and the light, something held on to the fact that you had pressed. You are not holding the button down; the press was a moment, and the circuit kept it. That act of remembering is what separates a calculator from a pile of switches. A circuit whose output depends only on its inputs right now is combinational: it decides. A circuit whose output also depends on what happened earlier is sequential: it has memory, so it has a past. Strip any digital chip down and you find only those two kinds, joined in loops, with a clock keeping the loops honest. The course in one box Who takes it: electrical and computer engineering students, usually in the second year; computer scientists meet a lighter version inside architecture courses. What it's for: designing circuits that make decisions and keep state, and knowing how fast they can be clocked. Comes after: Discrete Mathematics (34/08). Leads to: Embedded Systems (34/41), and it is the hardware layer under Computer Architecture (34/25). Honest line: you won't be able to lay out a chip after this piece. You will be able to simplify a small truth table by eye, step a state machine by hand, and say why a faster-looking circuit can have a slower clock. For easier first reads, see the truth-table section of Discrete Mathematics (34/08) and the ripple-carry adder in Computer Architecture (34/25); this piece goes further than both, and Processors: How a CPU Thinks (29/11) shows where it all ends up. Boolean algebra: three gates, and NAND builds the rest A wire is high or low, 1 or 0. A gate combines wires: AND outputs 1 only if both inputs are 1, OR outputs 1 if either is, NOT flips one. George Boole published the algebra of such true-or-false quantities in 1854, in An Investigation of the Laws of Thought. In 1937 Claude Shannon's MIT master's thesis showed that the same algebra describes networks of relay switches, which is the day logic became engineering. The rule graduates carry is De Morgan's law: NOT (A AND B) equals (NOT A) OR (NOT B), and swapping AND and OR gives the other half. In words: "not both" means "at least one is missing". It lets you push inverters through a circuit and change one kind of gate into another. NOT(A · B) = A′ + B′   ·   NOT(A + B) = A′ · B′ A consequence surprises almost everyone: one gate type is enough. NAND (AND followed by NOT) can be wired as NOT, AND, OR and XOR. NOT is a NAND with both inputs tied together. AND is a NAND followed by that NOT. OR comes straight from De Morgan: NAND the two inverted inputs. XOR takes four. Interactive · Build a gate from NANDs only Choose XOR and flip A and B. Each NAND in the list shows the value on its output wire; the last one is the answer. The check line runs all four input combinations against the real XOR. Karnaugh maps: simplifying by finding neighbours Many circuits that give the right answer are bigger than they need to be. The identity behind all shrinking is small: A·B + A·B′ = A. If a result is 1 both when B is 1 and when B is 0, then B does not matter, and the wire for B can be cut. A Karnaugh map turns hunting for such pairs into looking. Lay the truth table out as a grid whose rows and columns are numbered 00, 01, 11, 10 (one bit changes at each step, so neighbours differ in exactly one variable). Then any block of 1s that is 2, 4 or 8 cells, rectangular, can be written as one short product term; each time the block doubles, one variable drops out. The grid also wraps around its edges. Veitch drew the chart in 1952 and Karnaugh refined it in 1953. A worked case: the carry out of a full adder is 1 when at least two of A, B, C are 1 (the majority). Its map groups into three pairs, giving AB + AC + BC, instead of four terms of three variables each. Cells you will never see in practice can be marked don't care (X): the minimiser may treat them as 1 if that makes a bigger block. The program here is a real Quine–McCluskey minimiser. Interactive · Karnaugh map (4 variables) Tap a cell: 0 → 1 → X (don't care). Columns are A B, rows are C D. Tap a term below to outline its cells. Try "Majority of A, B, C": four terms collapse to three pairs. Try "Odd number of 1s": the 1s sit on a checkerboard, no two touch, and nothing simplifies: parity genuinely costs 8 terms. Last, "BCD digit is 5 or more" uses X for the six codes (10 to 15) that a decimal digit never takes; with them marked X the answer drops to A + BC + BD, three terms, against A′BC + A′BD + AB′C′ if you insist they be 0. Free choice in the inputs buys a smaller circuit. Combinational blocks: a multiplexer is a lookup table Three blocks do most of the deciding in a chip. A decoder takes an n-bit number and lights exactly one of 2n output lines; it is how a memory finds one row out of thousands. A multiplexer (mux) does the reverse job: n select wires choose which of 2n data inputs reaches the output. All are combinational: the output is a pure function of the inputs. The idea worth keeping is hidden in the mux. Take an 8-to-1 mux and use A, B, C as the select wires. Now tie each of the eight data inputs to a fixed 0 or 1. The output is whatever you wrote on the data inputs, for each of the eight input combinations: the data inputs are the truth table. Change the eight bits and the same hardware computes a different function. That is a lookup table (LUT), and a k-input LUT holds 2k bits and can be any of 2(2^k) functions. functions of k inputs = 22k   ·   k = 3: 256  ·  k = 4: 65,536 Interactive · An 8-to-1 mux as a programmable gate Data inputs (the stored truth table), one per row of A B C: Select lines (the inputs to the function): Press "Majority" and set A B C to 1 1 0: the output is 1, because two of three are high. Press "Parity" and the same select values give 0: nothing was rewired, only the stored bits changed. This is exactly what an FPGA is: a sea of such LUTs, each with a flip-flop beside it, joined by wiring that can also be programmed. The first commercial FPGA, Xilinx's XC2064 of 1985, had 64 configurable blocks, each computing a function of four inputs; modern ones commonly use 4 to 6. A 6-input LUT can be any of 264, about 1.8 × 1019, functions. Timing: gates take time, and the slowest path sets the clock Gates in diagrams answer instantly. Real gates take a short time, the propagation delay, because a transistor must charge a wire. Chain gates together and the delays add. Two things follow. First, the answer is wrong for a moment while signals race through different paths. Second, whatever reads the answer must wait until the slowest path has finished: the critical path. The wrong-for-a-moment effect is a glitch (or hazard). Take A·B + A′·C with B = C = 1: the output should stay 1 whether A is 0 or 1. But when A flips, the inverter that makes A′ is a little late, so for an instant both terms are 0 and the output dips. On a Karnaugh map it is two blocks that touch without overlapping. Add the redundant term B·C to bridge them and the dip vanishes. The critical path sets how fast the clock can run. Between two flip-flops the signal needs the flip-flop's own delay to launch, then the logic, then a small setup margin before the next edge: Tclock ≥ tlaunch + tlogic + tsetup   ·   fmax = 1 ÷ Tclock Take the adder from Computer Architecture (34/25). In an N-bit ripple adder the last sum bit sits 2N gate levels from the inputs. The cure is carry-lookahead. For each column, g = a·b says "this column makes a carry by itself" and p = a⊕b says "this column passes a carry along". Then each carry unrolls into one flat expression, for example c₃ = g₂ + p₂g₁ + p₂p₁g₀ + p₂p₁p₀c₀, which is two gate levels no matter how wide the adder. The price is fan-in: the carry into the top bit needs a gate with about N inputs, and the carry logic grows roughly with the cube of N. Real designs look ahead in groups of four. Interactive · Spend gates, buy clock speed Adder width Delay of one gate (illustrative) Model: depth counted in gate levels from inputs to the last sum bit; launching and setup assumed to cost 3 gate delays in total; lookahead idealised with gates of unlimited fan-in. Gate delay is a made-up round number, not a process figure. At 32 bits and 50 ps per gate, the ripple adder is 64 levels deep and its clock can be no faster than about 300 MHz; the lookahead version is 4 levels and could reach about 2.9 GHz, but needs a 32-input gate that does not exist. Slide the width down to 4 and the two nearly tie. Fast digital design is that trade: more hardware, shallower paths. Flip-flops and the clock: how a wire remembers Memory comes from a loop. Cross-couple two NOR gates, each one's output feeding the other's input, and the pair has two stable states: Q = 1 or Q = 0. Pulse Set and it goes to 1; pulse Reset and it goes to 0; release both and it stays, because each gate is holding the other in place. That is the SR latch, the smallest memory. Raise Set and Reset together and both outputs drop to 0; release them at the same moment and the result depends on which gate is faster, so the latch lands unpredictably. A latch changes whenever its inputs do, which makes big circuits hard to reason about. The fix: let memory change only at one instant, an edge of a shared clock. A D flip-flop copies its D input to Q at the rising edge and ignores D at all other times (two latches back to back, open on opposite clock phases). Now the whole chip is a set of snapshots: between ticks the logic settles, at the tick every flip-flop captures its input together. That is why timing, above, is the clock's only constraint. The catch is the setup window. If D changes too close to the edge, the flip-flop can end up halfway: neither 0 nor 1 for an unpredictable time, called metastability. It cannot be designed out, only made very unlikely; signals crossing between unrelated clocks pass through two flip-flops in a row for that reason. Interactive · A latch, then a flip-flop SR latch from two NOR gates. Switch Set and Reset on and off. D flip-flop. Set D, then give it a clock edge. Try Set on, then off: Q stays 1, which is memory. Then switch both on, then both off, and the latch reports an unknown state. In the flip-flop, change D freely and watch Q refuse to move until the edge. Then press "Change D just before the edge" and Q becomes "?". Put several flip-flops side by side on one clock and you have a register: n bits captured at once. Feed a register's output through an incrementer and back into its input and you have a counter: each tick adds one. Feed each flip-flop from its neighbour and you have a shift register, which moves bits sideways. A counter is also a frequency divider: the lowest bit toggles every tick, the next every two, so each bit runs at half the speed of the one below. A watch crystal oscillates at 32,768 Hz, which is 215, so fifteen divide-by-two stages leave exactly one pulse per second. Interactive · A 4-bit register, as a counter or a shift register Clock tick8 ticksShift-in bit: 1 Divide a 32,768 Hz crystal by 2, this many times State machines: a vending machine you can step through A register plus combinational logic that computes its next value from its current value and the inputs is a finite-state machine (FSM). The register holds the state (the machine's whole memory of its past). Each tick, logic reads the state and the inputs and decides the next state and the outputs. A crossing light is one: Green, Yellow, Red, and the button press moves it from "waiting" to "scheduled to change". Lifts, microwaves, protocol handlers and the control unit inside a processor are all FSMs. Design recipe: list the situations (states), draw an arrow for every input that leaves each one, write the table, then pick a binary code for each state and turn the table into equations, which is what the Karnaugh map was for. n states need at least ⌈log₂ n⌉ flip-flops. Our machine sells one item for 15 cents. It accepts nickels (N, 5¢) and dimes (D, 10¢), one coin per tick. It needs to remember how much money is in: three situations, 0¢, 5¢, 10¢, so two flip-flops. At 10¢ a dime makes 20¢, so it also returns one nickel. Outputs here depend on the state and the input (a "Mealy" machine); a crossing light whose outputs depend on state alone is a "Moore" machine. Interactive · Vending machine FSM Insert nickelInsert dimeReset StateNickelDime 0¢→ 5¢→ 10¢ 5¢→ 10¢→ 0¢, item 10¢→ 0¢, item→ 0¢, item, 5¢ change Code the states 0¢ = 00, 5¢ = 01, 10¢ = 10, and call the two flip-flops S1 and S0. The code 11 never occurs, and a nickel and a dime never arrive together: those are don't-cares. Run the Karnaugh minimiser on the table (the same one in the map above) and the whole machine collapses to four short equations. Next state: S1′ = S1′·S0′·D + S1·N′·D′ + S0·N and S0′ = S1′·S0′·N + S0·N′·D′. Outputs: item = S1·N + S1·D + S0·D and change = S1·D. Writing hardware: Verilog, VHDL and FPGAs Nobody draws a million gates. Engineers write a hardware description language and let a tool, the synthesizer, turn it into gates, LUTs and flip-flops. The two main languages are Verilog (created by Phil Moorby at Gateway Design Automation in the mid-1980s, an IEEE standard since 1995) and VHDL (born from a US Department of Defense programme, an IEEE standard since 1987); SystemVerilog is Verilog's later extension. The mental switch is hard: a program runs line by line, but a hardware description is a circuit, and all of it exists at once. Here is a free-running counter that drives an LED, the "hello world" of FPGA boards: module blink(input clk, output led); reg [25:0] count = 0; always @(posedge clk) count <= count + 1; assign led = count[25]; endmodule reg [25:0] asks for 26 flip-flops. The always @(posedge clk) block means "capture on the rising edge"; the <= means every register updates at the same instant; the adder after it is combinational, as in the sections above. Bit k of a counter toggles every 2k ticks, so the LED, on bit 25, changes state every 225 ticks and completes a full blink in 226. Interactive · How fast does the LED blink? Predict first. A 50 MHz board clock, led = count[25]. How long is one full blink (on and off)? LED wired to bit The answer is 226 ÷ 50,000,000 = 1.34 seconds per blink. With a 12 MHz clock the same code takes 11 seconds; pick a lower bit to blink faster. An FPGA is not the only target. On an ASIC (a custom chip) the same code becomes fixed gates: much faster and smaller per function, and cheap per chip only when millions are made, because the masks cost a fortune. An FPGA is reprogrammable by loading a new bit-pattern, so it suits prototypes, low volumes and algorithms that may change. What a graduate just sees A circuit either decides from the present (combinational) or also uses the past (sequential); every chip is both, joined by a clock. NAND alone is enough, and De Morgan lets you trade ANDs for ORs by moving inversions. If an output is 1 for both values of a variable, that variable is not needed: look for neighbours. Any truth table is a memory: a mux with the answers stored on its data inputs. That is a LUT, and an FPGA is a sea of them. Delays add along a path; the slowest path sets the clock; a wide, shallow circuit buys speed with fan-in. A flip-flop samples at the clock edge; if the input changes inside the setup window the result can be neither 0 nor 1. A counter divides frequency by two per bit; a register plus next-state logic is a state machine. State count sets flip-flop count: n states need ⌈log₂ n⌉ bits, and unused codes are free don't-cares. Hardware code describes a circuit that exists all at once, not a sequence of steps. Where it shows up next Embedded Systems (34/41): the timers, counters and serial ports inside a microcontroller are the registers, dividers and state machines from this piece, fixed in silicon. Computer Architecture (34/25): the processor's arithmetic unit is combinational, its registers are flip-flops, and its control unit is a state machine; the pipeline stages are separated by registers exactly because of the clock rule above. Signals & Systems (34/37) and Communication Systems (34/40): filters and modems are often built on FPGAs, using the same LUTs, counters and shift registers. If you want to go deeper 📘 Textbook: Sarah Harris and David Harris, Digital Design and Computer Architecture, RISC-V Edition (Morgan Kaufmann, 2021). Covers gates through to a processor, with SystemVerilog and VHDL side by side. 📘 Textbook: M. Morris Mano and Michael Ciletti, Digital Design: With an Introduction to the Verilog HDL, VHDL, and SystemVerilog (6th ed., Pearson). A standard second-year course text, strong on state machines and Karnaugh maps. 📘 Gentler: Charles Petzold, Code (2nd ed.), and Ben Eater's free video series building an 8-bit computer from logic gates on breadboards (eater.net/8bit). 🌐 Free course: MIT OpenCourseWare 6.111 Introductory Digital Systems Laboratory (Spring 2006): lectures, a traffic-light controller lab, and Verilog on an FPGA. 🌐 Free, hands-on: Nand2Tetris: build a computer from a NAND gate upwards. The takeaway A digital chip is decisions and memory. Gates decide, and a few rules (De Morgan, neighbours on a map) shrink them; flip-flops remember, and the clock lets thousands of them agree on when. Add a rule for how state changes and you have a machine, which a hardware language can describe and a lookup-table chip can become. Try this: pick a machine with a button and a light (a crossing, a lift, a microwave). Write down its situations and what moves it from one to the next; count them, and work out how many flip-flops it needs. Sources & Further Reading Sources. Boole, An Investigation of the Laws of Thought (1854). Shannon, "A Symbolic Analysis of Relay and Switching Circuits", MIT master's thesis (1937), published in Transactions of the AIEE (1938). Veitch, "A Chart Method for Simplifying Truth Functions" (1952); Karnaugh, "The Map Method for Synthesis of Combinational Logic Circuits", Transactions of the AIEE 72, Part I (1953). XC2064 (1985, 64 configurable logic blocks): IEEE Spectrum Chip Hall of Fame and the ETHW milestone entry "The Field Programmable Gate Array (FPGA), 1984". Verilog: Computer History Museum profile of Phil Moorby; IEEE 1364-1995; VHDL: IEEE 1076-1987. The 32,768 Hz = 215 watch-crystal divider is a textbook design choice, documented in watch-divider patents. Propagation delay, setup time and metastability: textbook material (Harris & Harris; Mano & Ciletti). All interactive numbers were computed in code. 🌐 MIT OpenCourseWare: 6.111 🌐 nand2tetris.org ▶️ Ben Eater, "Building an 8-bit breadboard computer" (eater.net/8bit) 📘 Code, Charles Petzold (Microsoft Press, 2nd ed., 2022) Step 11. Probability & Statistics: Reasoning with Uncertainty ------------------------------------------------------------- Why: Noise, tolerances and bit errors are probability. Nothing you measure is exact, and no part comes out the same twice. Statistics is the discipline of knowing how much to trust a number. A machinist turns a hundred steel shafts to "10 millimetres". Measure them with a good micrometer and no two are the same: 9.981, 10.012, 10.004. Nothing is wrong. That scatter is a fact of every process, every material and every sensor. Now stack five of those parts end to end in a gearbox. Will the stack still fit the housing? A pollster calls a thousand people and says 52% will vote yes, give or take three. A drug trial says "significant". A forecast says 30% chance of rain. Every one of these is the same question: given scatter, how much should I trust this number? A graduate of probability and statistics doesn't see a single number any more. They see the shape of the scatter behind it. This piece builds that sight with eight things you can run: dice, tolerances, polls, a drug-trial trap, a belief that updates, and a gearbox stack. The course in one box Who takes it: almost every engineering and science student, usually in the second year; computer scientists meet it again before machine learning. What it's for: deciding what to believe, and how far to trust a measurement, when outcomes are uncertain. Comes after: Calculus I (34/01), which helps with continuous distributions, though the ideas here need little more than arithmetic. Leads to: Machine Learning (34/32) needs it directly. It also shows up in Materials Science & Manufacturing (34/13: process control and defects), Systems Engineering (34/22: reliability) and Computer Security (34/33: randomness and guessing). Honest line: you won't be able to design a clinical trial after this. You will see why "significant" doesn't mean "important", why a bigger sample beats a cleverer guess, and why most real quantities look like a bell curve. For easier first reads, see the library's Probability: How to Not Fool Yourself (12/03), Statistics for Reading the World (12/05) and Bayes' Theorem (12/17); this piece goes further into the engineering side. Random variables: a shape with a centre and a spread A random variable is a number that comes out of a process you can't predict one run at a time. The key move is to stop asking "what will it be?" and ask "what is the shape of what it could be?" Roll two dice and add them: 7 is six times as likely as 2 or 12, so the shape is a tent. Roll many times and the tent appears. Interactive · Roll dice, grow a shape Dice added together Roll 10 timesRoll 1,000 timesClear Without JavaScript: two dice add to 7 with probability 6/36 and to 2 or 12 with probability 1/36 each. The mean total is 7 and the standard deviation is about 2.42. Two numbers summarise a shape. The mean is its balance point: 7 for two dice. The standard deviation (the "spread") says how far values typically stray from the mean: about 2.4 for two dice, which means totals usually land between roughly 5 and 9. The variance is the standard deviation squared, and it is the one that adds up neatly when you combine independent things. standard deviation = √( average of (value − mean)² ) Now the shape that matters most. Measured quantities (shaft diameters, heights, sensor noise) tend to form the normal curve, the bell. Its great convenience is a fixed rule: about 68.3% of values fall within one standard deviation of the mean, 95.4% within two, and 99.7% within three. Engineers use it every day. Say the shaft's spec is 10.00 mm ± 0.05 mm. Slide the process around and see what share of parts get thrown away. Interactive · How many parts fail the spec? Average diameter Spread (standard deviation) Without JavaScript: a centred process with standard deviation 0.025 mm (the spec limits are exactly ±2 standard deviations) scraps about 4.55% of parts. Cut the spread to 0.01 mm and about 0.00006% are scrapped. Shift the average 0.02 mm off-centre with spread 0.02 mm and about 6.7% are scrapped. Two lessons graduates keep. First, spread, not average, decides quality: a process centred perfectly but wide still scraps parts. Second, being off-centre costs a lot even when the spread is fine. That is why factories track both. The central limit theorem: why the bell keeps appearing Why should so many things be bell-shaped? The answer is one of the most useful theorems in mathematics. Take almost any random process, however lopsided, and average a few dozen independent draws. The averages pile up in a bell, centred on the true mean, with spread shrinking like 1/√n. Early forms are due to de Moivre in 1733 (for coin flips) and Laplace, whose general version dates from about 1810. Pick a lopsided source and drag the sample size. Interactive · Average anything, get a bell Flat (0 to 1) Very lopsided Coin (0 or 1) Draws averaged together: n Without JavaScript: for the lopsided source (mean 1, standard deviation 1) the average of n = 30 draws has standard deviation 1/√30 = 0.183, and the 3,000 averages form a close-to-normal bell around 1. At n = 1 you see the raw source. By n = 30 even the lopsided one is a near-bell. The prediction (the curve) uses only two numbers from the source, its mean and its standard deviation. The shape of the source is forgotten. That is why a measurement made of many small independent errors is normal, and why so much of statistics can pretend things are bell-shaped. standard error of an average = σ / √n In words: the average of n draws wobbles less than one draw, by the square root of n. To halve the wobble you need four times the data, not twice. It is the quietest expensive fact in all of experimental science. Sampling and confidence intervals: how sure is a poll? You can't ask everyone, so you ask n people and treat the share saying yes as an estimate. How far off might it be? The standard error from the last section gives the answer. For a share p the wobble is √(p(1−p)/n), and a 95% confidence interval is the estimate plus and minus about two of those. For n = 1,000 and a share near half, that is ±3.1 percentage points: the familiar "margin of error". The wording is subtle, and the interactive makes it concrete. "95% confidence" is a statement about the method, not about one interval: if you ran the procedure again and again, about 95 intervals in 100 would capture the true value. Here the truth is secretly 52%. Run 100 polls. Interactive · 100 polls, one truth People asked in each poll Run 100 polls Without JavaScript: with 1,000 people per poll the margin of error is about 3.1 points; run 100 polls and about 95 of the intervals cover the true value. With 4,000 people the margin halves to about 1.5 points. Watch two things. Roughly five in a hundred intervals miss the truth even when everything is done correctly. And quadrupling the sample from 1,000 to 4,000 only halves the interval. A poll of a country and a poll of a town need the same sample size for the same precision, as long as the sample is random. The real danger is not the margin of error; it is a sample that isn't random at all, which no formula fixes. P-values and Bayes: two ways to weigh evidence A hypothesis test asks whether data are surprising if nothing is going on. The p-value is the probability, assuming nothing is going on, of seeing a result at least as extreme as yours. Flip a coin 100 times and get 60 heads: for a fair coin the chance of a split at least that lopsided in either direction is about 5.7%. It is not the probability that the coin is fair, and it is not the probability that the result was "due to chance". The American Statistical Association issued a rare formal statement on this in 2016, with six principles. Two of them: p-values do not measure the probability that a hypothesis is true, and a p-value does not measure the size or importance of an effect. The classic trap is testing many things. At the usual cut-off of 0.05, a test of a true "nothing" comes out "significant" one time in 20 by luck. Test twenty unrelated things and the chance that at least one lights up is 1 − 0.9520, about 64%. Try it: each press tests twenty effects that are, by construction, nothing. Interactive · Test twenty things that do nothing Run 20 testsReset the tally Without JavaScript: twenty tests of pure noise at the 0.05 level give at least one false "discovery" about 64% of the time, and one in 20 tests on average. If you report only the one that lit up, you have a headline and no finding. This is part of why many published results fail to repeat: in the Open Science Collaboration's 2015 attempt to replicate 100 psychology studies, 97% of the originals had statistically significant results but only 36% of the replications did. Bayes' rule answers a different question: given the data, how should my belief move? Start with a prior belief, multiply by how well each possibility explains the data, and renormalise. Say you test parts and want the defect rate. Know nothing at first (any rate from 0% to 100% equally likely) and add parts one at a time. Interactive · Learn a defect rate from parts Add a good partAdd 10 good partsAdd a defective partReset Without JavaScript: after 20 good parts and no defects, a flat starting belief gives a most-plausible defect rate near 0%, an average of 4.5%, and a 90% credible range of roughly a quarter of a percent to 13%. Twenty clean parts cannot rule out a defect rate near one in eight. Frequentists and Bayesians disagree about the philosophy: one treats probability as long-run frequency and never assigns it to a hypothesis, the other as degrees of belief that data update. In practice both are used, often on the same project, and on big data sets they usually agree. The Bayesian version has one practical advantage here: after twenty clean parts it honestly says "we still can't rule out one in eight". Correlation, regression and the trap of causation Correlation (called r) measures how tightly two quantities move together along a straight line, from −1 to +1. Regression fits the best line through the cloud (by least squares, as in Linear Algebra, 34/04). Both are summaries, and any summary can hide the story. In 1973 the statistician Francis Anscombe made four data sets with, to two decimals, the same mean, the same spread, the same r (0.82) and the same line (y = 3 + 0.5x). Look at each. Interactive · Four data sets, one set of numbers Set 1Set 2Set 3Set 4 Without JavaScript: all four Anscombe sets have mean x = 9, mean y = 7.50, correlation 0.816 and line y = 3.00 + 0.50x. Set 1 is a noisy line, set 2 a curve, set 3 a line with one outlier, and set 4 a single far point that creates the whole slope. The numbers agree and the stories do not: one noisy line, one curve, one line with a rogue point, one vertical stack held up by a single point. Always plot first. The other trap is the famous one. Ice-cream sales and drownings rise together, because summer heat drives both. A confounder (a hidden third factor) makes two things move together without either causing the other. The remedy graduates know is randomised assignment: if a coin decides who gets the treatment, nothing else can be systematically linked to it. Where you can't randomise, causal claims need much more than a line through a cloud. Engineering: tolerance stacks, reliability and Monte Carlo Five parts, each 10 mm ± 0.1 mm, line up in an assembly. What is the worst the total can be out? Worst-case stacking adds the tolerances: ±0.5 mm. It is safe and expensive, because it assumes every part is at its extreme in the same direction at once. If the parts vary independently, variances add, so the standard deviations add in quadrature: root-sum-square, √5 × 0.1 = ±0.224 mm. For five parts that is under half the worst-case figure. When the formulas get hard, engineers use Monte Carlo: let the computer build the assembly ten thousand times with random parts and count how often it fails. Here each part's tolerance is treated as three standard deviations. Choose how many parts are stacked. Interactive · Monte Carlo tolerance stack Parts in the stack Build 10,000 assemblies again Without JavaScript: five parts at ±0.1 mm (three standard deviations) give a worst-case limit of ±0.5 mm, a root-sum-square limit of ±0.224 mm, and in simulation only about 0.3% of assemblies exceed the root-sum-square limit; none come close to 0.5 mm. The honest caveat is that root-sum-square assumes the parts vary independently and are centred. A tool wearing in the same direction on every part breaks that assumption, and so does one supplier's batch. Engineers disagree about when to trust statistical stacking over worst-case; safety-critical designs often insist on worst-case. Reliability follows the same arithmetic. A system that needs all of its parts is a series system, and probabilities multiply: ten components that each work 99% of the time give 0.9910 ≈ 90.4% for the whole. Add a backup in parallel and failure needs both to fail: two units that each work 90% of the time give 1 − 0.1² = 99%. The two rules drive why aircraft duplicate systems and why complicated machines need very reliable components. What a graduate just sees Every number has a spread behind it; the first question is "how scattered?", not "what is it?". Spread, not the average, decides how many parts are scrapped, and centring matters too. Averages of many independent things are bell-shaped, whatever the source looked like. Halving the uncertainty costs four times the data. A 95% interval is a claim about a method: it misses one time in twenty by design. A p-value is "how surprising is this if nothing is going on", not "how likely is it true". Test twenty things and one will look significant; ask how many were tried. Plot the data; correlation is not causation, and only randomising breaks a confounder. Independent errors add in quadrature; series reliabilities multiply. Where it shows up next Machine Learning (34/32): training is fitting a model to noisy data; ideas of overfitting, error bars and Bayes are this course made bigger. Materials Science & Manufacturing (34/13): process capability and scrap rates are the normal curve against a spec limit. Mechanics of Materials (34/11) and Machine Design (34/19): safety factors exist because loads and strengths are distributions, not numbers. Systems Engineering (34/22): reliability budgets multiply series probabilities and add redundancy to push them back up. Computer Security (34/33) and Algorithms (34/24): good random numbers, randomised algorithms, and the odds of guessing a secret. If you want to go deeper 📘 Douglas Montgomery and George Runger, Applied Statistics and Probability for Engineers (Wiley; 7th edition). The standard engineering course text, with plenty of tolerance, quality and reliability problems. 📘 Joseph Blitzstein and Jessica Hwang, Introduction to Probability (CRC Press; 2nd edition, 2019). Best for deep intuition about probability itself, from Harvard's Stat 110. 📖 David Spiegelhalter, The Art of Statistics (2019): the most readable route for learning alone, built around real cases and honest about uncertainty. 🌐 MIT OpenCourseWare 18.05, Introduction to Probability and Statistics (Spring 2022): free notes and problem sets, with Bayesian and classical statistics side by side. The takeaway Nothing you measure is exact, so every number is a shape. Summarise a shape by its centre and its spread, trust averages more as the square root of the sample, read a confidence interval as a promise about a method, treat a p-value as a measure of surprise, and plot before you believe a correlation. Try this: next time you see a poll or a "significant" headline, find the sample size and ask how many things were tested. Then, in the dice widget above, add four dice and watch the tent turn into a bell. Sources & Further Reading Sources. The normal-curve rule (68.3%, 95.4%, 99.7%), the central limit theorem, confidence intervals, the binomial test, the Beta posterior, root-sum-square stacking and series/parallel reliability are standard textbook results (Montgomery and Runger; MIT 18.05); every number shown was recomputed in code. Wasserstein and Lazar, "The ASA's Statement on p-Values: Context, Process, and Purpose", The American Statistician (2016). Open Science Collaboration, "Estimating the reproducibility of psychological science", Science (2015): 97% of originals significant against 36% of replications. Anscombe, "Graphs in Statistical Analysis", The American Statistician (1973). Early forms of the central limit theorem: de Moivre (1733) and Laplace (c. 1810), as surveyed in recent histories such as Ulyanov, "From classical to modern central limit theorems" (arXiv 2405.19828). Go deeper. The books and course above; Wasserstein and Lazar's statement is free and short (doi.org/10.1080/00031305.2016.1154108). Step 12. Electronics: The Switch That Became Everything ------------------------------------------------------- Why: Diodes, transistors and op-amps: the parts that make a circuit do something. A one-way valve, a voltage-controlled tap and a feedback trick. Three ideas, billions of copies, and the chip in your phone. Open a phone charger, a radio or a laptop and you find the same few parts over and over: tiny black boxes with three legs, small glass beads with two, and one chip with a hundred. Everything the machine does, from playing a song to deciding that a screen should light up, is done by those parts. There are really only three ideas. A part that lets current through in one direction only. A part where a small voltage on one leg controls a large current between the other two. And a trick, feedback, for making the second kind behave. Learn these and you can read the inside of nearly any device. They are also the bottom layer of everything in Computer Architecture (34/25). The course in one box Who takes it: electrical and computer engineers, usually in year 3, often as a pair of courses (devices, then circuits). What it's for: designing circuits that make something happen: switch a load, amplify a signal, convert power, hold a voltage steady. Comes after: Circuit Analysis (34/34). Leads to: Digital Logic Design (34/36) and Embedded Systems (34/41), where the transistors are hidden behind gates and code; Communication Systems (34/40), whose radios are amplifiers and filters; and Power Systems & Electric Machines (34/39), where the inverters that connect solar panels to the grid are transistor switches. Honest line: you won't be able to lay out a circuit board after this. You will see why a diode needs a resistor, why an amplifier's gain is set by two resistors rather than by the chip, and why a faster processor runs hotter. Silicon, the p–n junction and the diode Pure silicon barely conducts: its electrons are locked into bonds. Add a sprinkle of phosphorus and each atom donates a spare electron: you get n-type silicon, with mobile negative charge. Add boron and each atom is one electron short, leaving a hole that behaves like a mobile positive charge: p-type. Neither is interesting. Join them and something happens at the seam. Electrons wander across into the p side, holes into the n side, and they cancel. What is left near the seam are fixed charged atoms that set up an electric field, which pushes back. The result is an empty zone with a built-in voltage of roughly 0.6 to 0.7 V in silicon. Push the p side positive (forward bias) and the zone shrinks and the carriers pour through; push it the other way (reverse bias) and the zone widens and almost nothing flows. That is the diode: a one-way valve with a rule that every graduate recites. I = Is × (eV ÷ VT − 1)VT = kT ÷ q ≈ 25.9 mV at room temperature. The current is exponential in the voltage. Exponential means a tiny voltage change is a huge current change. Each extra 60 mV multiplies the current by ten, which is why a silicon diode always seems to sit at "about 0.6 to 0.7 V" whether it carries a milliamp or a hundred. That is also why you never connect a diode straight across a battery. Put an LED, which needs about 2 V, on a 5 V supply and you choose the current with a resistor: (5 − 2) V ÷ 10 mA = 300 Ω. An LED is a junction in which falling electrons give off light, at the energy of the material's gap. Silicon is a poor light emitter, so LEDs use other compounds; efficient blue ones, which made white LED lighting possible, took until the early 1990s and earned the 2014 Nobel Prize in Physics (Akasaki, Amano and Nakamura). See Screens (29/10) for what they do next. The commonest job of a diode is rectifying. A diode (or four, in a bridge) turns the alternating mains into bumps of one polarity. A big capacitor smooths the bumps: it charges at each peak and feeds the load between peaks. The sag between peaks is the ripple, roughly I ÷ (2 f C) for a bridge. A 100 mA load on a 1 mF capacitor at 50 Hz gives about 1 V of ripple. Interactive · The junction and its curve Voltage across the diode Temperature Ideal Shockley model with a silicon-like saturation current; real diodes also have series resistance and break down in reverse. Heat the diode and it conducts at lower voltage. Interactive · Rectifier and smoothing capacitor Half-wave (1 diode)Full-wave (bridge) 50 Hz60 Hz Smoothing capacitor Load current A 12 V rms transformer secondary (17 V peak), silicon diodes with a 0.7 V drop each, a constant-current load. Computed step by step, not from the formula. The transistor as a switch Now the idea that changed the century. A transistor has three terminals, and a voltage or current on one controls a large current between the other two. Bell Labs demonstrated the first working one in December 1947 (Bardeen, Brattain and Shockley; the 1956 Nobel Prize in Physics). The version inside every chip is the MOSFET: a metal gate, a thin glass insulator and the silicon underneath. Picture a tap with no handle but an electric field. Put a positive voltage on the gate and it pulls electrons to the surface beneath it, forming a conducting channel between the other two terminals, called source and drain. Below a threshold voltage nothing flows: the switch is open. Above it the switch closes, and the more the gate exceeds the threshold the lower the channel's resistance. Because the gate sits on an insulator it draws no steady current at all. A signal can control a thousand such switches without being loaded down. ID = ½ k (VGS − Vth)²in saturation; for small drain voltages the channel is a resistor, R ≈ 1 ÷ (k × (VGS − Vth)) Add a resistor from the drain up to the supply and you have the simplest logic gate. Gate low: the transistor is off, no current flows through the resistor, and the output sits at the supply, a logic 1. Gate high: the transistor conducts, pulls the output toward 0 V, and the output is a logic 0. High in, low out: an inverter. Notice the output is clean at both ends and wobbly only in the middle, where the input is near the threshold. Digital design is the art of never living in the middle, and that is why noise in a digital circuit does not accumulate: each stage restores the level. Interactive · A transistor as a switch (an inverter) Gate voltage Pull-up resistor Supply 5 V, threshold 1 V, k = 2 mA/V². A textbook square-law model; real short-channel transistors depart from it, but the shape is the same. The transistor as an amplifier The same part can be used in between on and off, as a variable tap. This is how weak signals from a microphone, an antenna or a heart sensor get strong enough to use. Take the older bipolar transistor (BJT): its collector current depends on the voltage across its base and emitter exactly as the diode's does, exponentially. A graduate remembers one number from that, the transconductance: gm = IC ÷ VT, gain = −gm × RCa BJT with a collector resistor: set by the bias current and one resistor, and the minus sign means the output is upside-down At 1 mA the transconductance is 38.7 mS, and with a 4.7 kΩ resistor the voltage gain is −182. A 5 mV wiggle at the input becomes about 0.9 V at the output. The idea behind this is calculus: near its resting point (the bias or Q-point) the exponential curve looks like its tangent line, and a straight line is an amplifier. Move far from that point and the curve bends: output peaks get squashed, which is distortion, and when they hit the supply they flatten, which is clipping. Choosing the bias so the output rests mid-way between the rails, leaving room to swing both ways, is most of the practical art. Interactive · Biasing an amplifier Bias current IC Collector resistor RC Input signal (peak) Supply 12 V, exponential BJT model at 300 K, output limited to between 0.2 V and the supply. Dashed grey is what a perfectly straight-line amplifier would give. The op-amp, feedback and the golden rules A single transistor stage has a gain that wanders with temperature, with the individual part and with the supply. Engineers do not fight that. They pile on enormous gain, then use feedback to throw most of it away in a controlled way. An operational amplifier is a block of a dozen transistors with two inputs and one output whose gain is huge (100,000 or more) and utterly unreliable. Wrapped with two resistors it becomes a precision amplifier. Two approximations make op-amp circuits almost mental arithmetic. Horowitz and Hill call them the golden rules: (1) with feedback, the output does whatever it must to make the two inputs equal; (2) the inputs draw no current. Take an inverting amplifier with an input resistor Rin and a feedback resistor Rf. The "+" input is at 0 V, so the "−" input is held at 0 V too, a virtual ground. The input current is Vin ÷ Rin and, since none enters the op-amp, it all goes through Rf: Vout = −Vin × Rf ÷ Rinnon-inverting version: Vout = Vin × (1 + Rf ÷ Rin) The gain is set by two resistors, which are cheap and stable, and the op-amp's own huge gain drops out. The price is bandwidth: an op-amp's gain falls as frequency rises, and the product of gain and bandwidth stays roughly constant. A part with a gain-bandwidth product of 1 MHz can give a gain of 10 up to about 100 kHz, or a gain of 100 up to about 10 kHz. This is exactly the loop-gain argument of Control Systems (34/20), written in resistors. Interactive · Gain from two resistors InvertingNon-inverting Rf ÷ Rin Op-amp open-loop gain Single-pole op-amp model with a gain-bandwidth product of 1 MHz. Drag the open-loop gain down to 100 to see where the golden rule stops being true. Noise: the floor under every signal Every resistor, even one carrying no current, has a tiny random voltage across it. Electrons jiggle with heat, and the jiggle adds up to noise. John Johnson measured it at Bell Labs and Harry Nyquist explained it, both in 1928. The result is a number you can compute, and it is the same for every material of that resistance: vn = √(4 k T R B)rms noise voltage: k is Boltzmann's constant, T the temperature in kelvin, R the resistance, B the bandwidth you listen over A 10 kΩ resistor at room temperature observed over 20 kHz, the audio range, has about 1.8 µV of noise. That is far below anything you would hear, but a microphone, a radio front end or a heart monitor deals in microvolts, and there the noise is the whole story. The practical consequences are all in the formula: listen over less bandwidth (filter), use lower resistances where the signal allows, and, if you must, cool the thing. Independent noise sources add in power, not in amplitude, so two equal noise sources give only about 41% more voltage, not double. That is the same square-root rule you met in Probability & Statistics (34/05). Interactive · Hunting a signal in the noise Source resistance Bandwidth listened to Temperature Signal size The noise trace uses a fixed random sequence scaled to the computed rms, so you see the same wiggles grow and shrink. Only thermal noise is modelled; real circuits add shot and amplifier noise on top. From one transistor to a chip Now pair two MOSFETs, one that conducts when its gate is high (n-type) and one that conducts when it is low (p-type), with their drains joined. For either input, exactly one is on and the other is off: the output is pulled firmly to the supply or to ground, and no steady current flows. That is CMOS, the logic family of every processor, and its great virtue is that a gate that is not changing costs almost nothing. Power is drawn only at the moment of switching, when the output's capacitance is charged or discharged. You have already met that moment: it is the RC transient of Circuit Analysis (34/34), in which charging a capacitor from a supply costs ½CV² in the capacitor and the same amount in heat. P ≈ α × C × V² × fdynamic power: the fraction α of switched capacitance C that toggles each clock tick, the supply voltage squared, and the clock frequency The square on the voltage is the most important term in computing. Halve the supply and the power drops to a quarter. For decades each generation of smaller transistors let engineers lower the voltage and raise the clock together (this is Dennard scaling, described by Robert Dennard and colleagues in 1974). In the mid-2000s the voltage could not be lowered further without the transistors leaking current while off, and clocks stopped climbing. That is the power wall, and it is why modern chips add more cores instead of racing a single one, and why they switch off whatever they are not using. The scale is staggering. NVIDIA's H100 data-centre GPU, announced in 2022, has 80 billion transistors; each one is a switch like the one above. Chips this size are designed with software that solves node equations like those of the last piece, over and over, long before anything is built. The next step up, from transistors to gates, flip-flops and instructions, is exactly where Digital Logic Design (34/36) and Computer Architecture (34/25) begin. For the history of the processor itself, read Processors (29/11) first. Interactive · What a billion switches cost Supply voltage Clock frequency Activity: share of gates that switch each tick Illustrative round numbers, not a real chip: 1 billion gates of 0.3 fF each. The speed limit uses the common alpha-power model, with the clock anchored at 3 GHz for 1 V and a threshold of 0.3 V. The shape matters here, not the watts. What a graduate just sees A diode's current is exponential in voltage: about 60 mV per factor of ten, so a diode always sits near 0.6 to 0.7 V and always needs a resistor. A rectifier plus capacitor is a bucket refilled at every peak; the ripple is the load current divided by frequency and capacitance. A MOSFET is a switch controlled by a voltage and drawing no steady gate current; that is why switches can be chained by the billion. An amplifier is a transistor biased on the straight part of its curve; gain is gm × RC, and bias decides whether the output has room to swing. With feedback, gain depends on two resistors, not on the op-amp: make the inputs equal, assume they draw no current, and solve. Gain and bandwidth trade: more feedback gain means less speed. Noise is a floor: √(4kTRB), so narrower bandwidth and lower resistance buy you quiet. Digital means refusing to live in the middle; CMOS burns power only when it switches, as α × C × V² × f. Where it shows up next Digital Logic Design (34/36): an inverter, a NAND gate and a flip-flop are the transistors above, drawn as symbols. Computer Architecture (34/25): the power wall is why processors are multi-core, why they have caches and why they scale their clock to the load. Signals & Systems (34/37) and Communication Systems (34/40): amplifiers, filters and mixers are how a radio picks one signal out of the air, and noise decides how far it can reach. Power Systems & Electric Machines (34/39): a solar inverter or an electric-car motor drive is transistors switching hundreds of amps, many thousands of times a second. The real world: LED lamps and their drivers, phone chargers, audio amplifiers, every sensor, and every device with a battery that lasts a day. If you want to go deeper 📘 Textbook: Adel Sedra, Kenneth Smith, Tony Chan Carusone and Vincent Gaudet, Microelectronic Circuits (Oxford University Press; 8th edition, 2020): the standard course text for devices and analog circuits. 📘 Textbook and reference: Paul Horowitz and Winfield Hill, The Art of Electronics (Cambridge University Press; 3rd edition, 2015): it reads like a conversation with a working designer and is the book people keep on their desks for decades. 📘 Gentler: Ben Eater's free video series building circuits and a small computer from transistors and gates, and Charles Platt's Make: Electronics (Make Community), a learn-by-doing book. ▶️ Free course: MIT OpenCourseWare 6.012 Microelectronic Devices and Circuits (Fall 2009). 🌐 Easier first reads in this library: Processors (29/11), Screens (29/10) and Batteries (29/08). The takeaway Electronics is three ideas repeated. A p–n junction passes current one way, with a current that grows exponentially with voltage. A transistor is a voltage-controlled valve: used as a switch it is the atom of every digital chip, used biased on its straight section it is an amplifier whose gain is gm times a resistor. Feedback around an op-amp trades most of its huge gain for precision set by two resistors, and costs bandwidth. Noise sets the floor, and in a chip, power grows as α × C × V² × f, which is the reason clocks stopped climbing. Try this: take any phone charger or LED bulb that has a visible printed circuit board and find four things: the bridge of four diodes (or a single module), the biggest capacitor (the smoothing bucket), the transistor on a heat sink, and a small resistor next to an LED. If you can identify the job each one does, you can read the page. Sources & Further Reading Sources. Shockley diode equation, MOSFET square law, BJT transconductance, op-amp golden rules (Horowitz and Hill, The Art of Electronics), and the CMOS dynamic-power formula are standard textbook material (Sedra and Smith; Horowitz and Hill). The first transistor (Bell Labs, December 1947) and the 1956 Nobel Prize in Physics to Shockley, Bardeen and Brattain: Computer History Museum and Wikipedia's "Transistor" article (reported dates within December 1947 vary by source, so no day is given). Johnson (1928) and Nyquist (1928) thermal noise: standard attribution. The 2014 Nobel Prize in Physics for blue LEDs (Akasaki, Amano, Nakamura): Nobel Foundation. Dennard scaling: R. H. Dennard et al., IEEE Journal of Solid-State Circuits (1974); end of voltage scaling around the mid-2000s is the usual textbook account. NVIDIA H100: 80 billion transistors, announced 2022 (TechPowerUp and press coverage of the launch; pre-launch rumours of 140 billion were wrong). Computed in node: every reading in the interactives, including 38.7 mS and a gain of −182 at 1 mA and 4.7 kΩ, 1.82 µV of noise for 10 kΩ over 20 kHz at 300 K, the 1 V ripple of the 1 mF example, and the inverter's three regions. 🌐 MIT OpenCourseWare 6.012 Microelectronic Devices and Circuits 🌐 Ben Eater: building circuits and computers from scratch 📘 Sedra, Smith, Chan Carusone and Gaudet, Microelectronic Circuits 📘 Horowitz and Hill, The Art of Electronics Step 13. Signals & Systems: Every Signal Is a Recipe of Sines ------------------------------------------------------------- Why: Every sound, image and sensor reading as a sum of frequencies. Why a room has an echo, an equaliser can fix a song, a movie wheel spins backwards, and your phone can hear a note inside a chord: one idea (a system's reply to a single click) and one trick (break everything into sines). Clap once in an empty stairwell. What you hear afterwards is not the clap: it is the stairwell. The echo, the ring, the way the sound dies away over a second or two, all of that is a fingerprint of the room, and if you recorded it you could, in principle, make any voice sound as if it were speaking from inside that stairwell. Recording studios do exactly this with a recorded "impulse response" of a cathedral. Now watch a car's wheel in a film as it speeds up. At some speed the spokes seem to slow, stop, and turn backwards. The wheel has not changed. The camera takes only 24 snapshots a second, and a fast wheel looks, in 24 snapshots, exactly like a slow one. The same thing, called aliasing, decides how fast your phone must sample a sound and why the music on a CD tops out at 20 kHz. A signals-and-systems graduate sees both at once. A signal is a quantity that changes with time (a voltage, a pressure, a pixel row). A system is anything that turns one signal into another (a room, an amplifier, a filter, a bridge, a phone's microphone). The course gives you a very small number of ideas that handle all of them. The course in one box Who takes it: electrical and computer engineers, usually in year 3; mechanical and aerospace engineers meet a version of it inside control, and data scientists reinvent it under other names. What it's for: describing, predicting and designing anything that takes a signal in and gives a signal out, using the fact that sines pass through "linear, time-invariant" systems unchanged in shape. Comes after: Circuit Analysis (34/34), Linear Algebra (34/04) and Differential Equations (34/03). Leads to: Communication Systems (34/40), which is this subject applied to radio, plus Control Systems (34/20), which uses the Laplace half of it, and Embedded Systems (34/41), where the filters run on a chip. Machine Learning (34/32) uses convolution inside image and audio networks. Honest line: you won't be able to design a filter to a specification after this piece. You will see why the output of any such system is a sum of echoes, why every signal has a spectrum, why sampling needs a speed limit, and what a spectrogram really draws. For a first, gentler look at the transform itself, read The Fourier Transform (12/12) in this library; this piece goes further. A signal is a function; a system is a machine for functions A sound is air pressure at your ear as a function of time. A photograph is brightness as a function of position. A signal is any such function, and a system takes one in and gives one out: x(t) goes in, y(t) comes out. An amplifier, a guitar pedal, a spring and a mass, a cheap phone microphone are all systems. Most of engineering rests on a lucky class of systems: linear and time-invariant, "LTI". Linear means that doubling the input doubles the output, and the output for two signals added together is the two outputs added. Time-invariant means that if you play the input a second later, you get the same output a second later. Springs, resistors, capacitors, air in a room and most electronics at small signal sizes behave like this, and for them the whole behaviour compresses into something astonishingly small. output = input ⁎ hthe input "convolved" with h, the system's reply to a single sharp click That h is the impulse response: what comes out when you put in one instant of signal (a click). Everything else follows from the two properties above. Any input is a long row of clicks of different sizes at different times. The system answers each click with a scaled, shifted copy of h, and by linearity the output is all those copies added up. Convolution: an echo you can build The sum of shifted, scaled copies has a name, convolution, and in discrete form it is only this: y[n] = Σk x[k] · h[n − k]each input sample launches a copy of h, scaled by that sample, starting when the sample arrives The picture to keep is the echo. A hand clap in a canyon has h = "the clap, then a quieter clap 1.2 seconds later". Shout a word and the output is the word plus a quieter copy of the word 1.2 s later: convolution did nothing mysterious, it only laid down one copy of the input for each spike in h. A room with many reflections has many spikes, and the sound blurs into reverb. A moving-average smoother has a box for h, and it blurs the signal for the same reason. A worked number: the input below is eight samples. The echo filter has h[0] = 1 and h[d] = g. With d = 10 and g = 0.5, the output is 18 samples long (8 + 11 − 1), the first eight are the input untouched, and starting at sample 10 there is a half-size copy. The tallest part of the copy is 0.5 × 1.0 = 0.5, at sample 12. Interactive · Build an echo EchoReverbSmoother Delay between copies Strength of each copy Echo: the output is the input plus a half-size copy 10 samples later. Computed live by the convolution sum above. Top: input (8 samples). Middle: the system's impulse response h. Bottom: the output y. The smoother ignores the sliders (a fixed 5-sample average). Fourier: every signal is a sum of sines Convolution is easy to understand and tedious to do. The second big idea is a change of viewpoint that turns it into multiplication. A pure sine wave has a remarkable property: put it through any LTI system and a sine of the same frequency comes out, only scaled and delayed. Never a different frequency, never a different shape. Sines are the system's natural shapes, like the pure modes of a guitar string. Now add Fourier's claim (1807, for heat flow in a solid): nearly any repeating signal, however jagged, is a sum of sines at multiples of its base frequency, each with its own strength and its own delay. So to know what a system does to any signal, break the signal into sines, scale and delay each by what the system does at that frequency, and add them back. The "what the system does at each frequency" is its frequency response, written H(f). Y(f) = X(f) · H(f)convolution in time becomes plain multiplication in frequency A square wave needs sines at 1, 3, 5, ... times its base frequency with strengths 1, 1/3, 1/5, ... The first sine alone already captures 81% of the square wave's power; the first three capture 93%. But the sharp corners live in the high harmonics: you never fully get them, and a small overshoot of about 9% of the jump sits next to each corner however many sines you add. That is the Gibbs phenomenon, and it is why a sharply filtered signal "rings". The interactive goes one step past the one in The Fourier Transform (12/12): it also lets you scramble the phases. The strengths (the spectrum you see in a plot) stay identical, but the shape changes. The ear mostly cannot tell, because it hears the strengths; a communications engineer can, because the shape is what a receiver has to detect. Interactive · Build a wave from sines SquareSawtoothTriangleScramble phases Sines added 3 sines. Dashed: the ideal wave. Solid: the sum of the first sines, each computed from its Fourier coefficient. Bars: the strength of each sine. Scrambling changes only the phases: the bars do not move. Frequency response and filters Once you think in frequencies, a filter is just a gain that depends on frequency. Treble and bass knobs are filters. So is the way a thick wall lets a bass note through and stops speech. The simplest electronic filter, a resistor followed by a capacitor to ground, passes slow signals and blocks fast ones (a low-pass). Swap the two parts and you get a high-pass. Its gain has an exact form: |H(f)| = 1 / √(1 + (f / fc)²)low-pass: gain at frequency f for a cutoff of fc = 1 / (2π R C) At the cutoff frequency fc the gain is 1/√2 = 0.707, which is 3 dB lower (a halving of power). Ten times above it, the gain is 0.0995, 20 dB down: every factor of 10 in frequency costs a further factor of 10 in amplitude. Engineers say "20 dB per decade" for this one-part filter and "40 dB per decade" for two parts in a row, and they plot it with logarithmic axes because their ears and their problems both work in ratios. A worked case: a 160 Ω resistor and 1 µF capacitor give fc = 1/(2π · 160 · 10⁻⁶) = 995 Hz. The same transform of the impulse response gives the frequency response, so the echo from earlier and this filter are one object seen two ways. An echo of delay d has a response that is a series of peaks and notches (a "comb"), which is why a hall can make a note boom and its neighbour vanish. Interactive · Which sounds get through? Low-passHigh-pass Cutoff frequency One-pole RC filter, exact gain. Three test sounds: 60 Hz hum, a 1 kHz voice tone, 8 kHz hiss. Move the cutoff to 300 Hz as a low-pass and the hiss is down by 28 dB. Sampling and Nyquist: why the wagon wheel runs backwards A computer stores a sound as a list of measurements taken at a fixed rate. The sampling theorem says this loses nothing, provided you sample at more than twice the highest frequency present. The idea has several fathers: Nyquist's 1928 work on telegraph rates, Kotelnikov in 1933, Shannon's proof in 1949. Hence CD audio at 44,100 samples per second: human hearing tops out near 20 kHz, and twice that is 40 kHz. Break the rule and the damage is not a blur but a lie. A frequency above half the sample rate shows up as a lower one, an alias: falias = f − fs · round(f / fs)the sampled signal cannot tell f from falias; a negative value means "running backwards" Take the film: fs = 24 frames per second. A one-spoked wheel making 23 turns a second looks like 23 − 24 = −1 turn a second: one slow turn backwards. The same arithmetic makes a helicopter's rotor appear to stand still on video, a ripple pattern appear on a finely striped shirt, and a high-pitched whine alias into the audio band of a badly designed recorder. The cure is to filter out everything above fs/2 before sampling, an "anti-alias filter", because afterwards the damage cannot be undone. Interactive · The wagon wheel 5 turns/s23 turns/s24 turns/s26 turns/s Wheel speed (frequency) Camera / sample rate Blue: the true signal. Dots: the samples. Orange: the slower sine that passes through the same dots. The dial shows where the one-spoked wheel is in frames 1 to 6. The FFT and the spectrogram Finding the sines in a list of N samples directly means N sums of N terms, so about N² multiplications. In 1965 James Cooley and John Tukey published the fast Fourier transform, which does it in about N log₂N steps (Gauss had used the same trick around 1805 on asteroid orbits, but never published it). For a million samples, direct evaluation needs about 1.1 × 10¹² steps and the FFT about 2.1 × 10⁷: roughly 50,000 times fewer. That is why a phone can run a spectrum analyser in real time, and why Wi-Fi and 4G, which use the FFT inside every transmission, are possible at all. One spectrum describes the whole recording at once, which hides when things happen. A spectrogram chops the signal into short windows and takes the spectrum of each, drawing time across, frequency up, loudness as colour. It is how speech scientists see vowels, how bird-song apps name a bird, and how Shazam-style apps match a tune. The catch is a trade-off built into the maths: a short window tells you when but blurs which frequency; a long window sharpens the frequency but smears the time. Frequency resolution is fs/N, window length N/fs, and you can never improve both. Interactive · Spectrogram of a rising whistle and a click 32 samples64128256 A whistle sliding from 50 Hz to 400 Hz over one second, plus a click at 0.7 s, sampled at 1,024 Hz and analysed with an FFT and a Hann window. Time runs left to right, frequency upward (0 to 512 Hz). Short windows: a crisp click, a thick line. Long windows: a thin line, a smeared click. Laplace and z, in words The Fourier view uses only steady sines. Real systems also grow and decay: a plucked string dies away, a microphone near a speaker squeals louder and louder. The Laplace transform (for continuous systems) and the z-transform (for sampled ones) widen the sines to decaying or growing ones, and in return every LTI system collapses into a handful of numbers called poles. A pole in the left half of the Laplace plane is a decay: the system forgets a disturbance. Farther left means faster. A pole on the vertical axis rings forever, and a pole in the right half is growth: instability. A pole that has an imaginary part also oscillates. A mass on a spring with ωn = 10 rad/s and damping ratio ζ = 0.2 (see Vibrations, 34/17) has poles at −2 ± 9.8 j: decaying at rate 2 per second, ringing at 9.8 rad/s. For sampled systems the same story is told on a circle: poles inside the unit circle are stable. A digital echo y[n] = x[n] + g·y[n−d] has all its poles of size g1/d, so for g = 0.9 it fades, for g = 1 it never dies, and for g above 1 it grows without limit: the squeal of a guitar amp. The whole of Control Systems (34/20) is this idea turned into design: place the poles where you want them, and the machine behaves. What a graduate just sees A system's whole personality is one signal, its reply to a click; every output is a pile of shifted, scaled copies of it. Convolution in time is multiplication in frequency, so filters are just gain versus frequency. Sines are the natural shapes of linear systems: they come out as sines, only scaled and shifted. A sharp edge needs high frequencies; a signal that is brief in time is wide in frequency, and you can't have both narrow. Anything above half the sample rate is not lost but disguised as something lower, so filter first, then sample. An FFT is not a trick for a few cases: it is the reason frequency thinking is cheap, which is why it is inside audio, radio, MRI and image codecs. Left-half-plane poles fade, right-half-plane poles blow up; feedback with too much gain turns a decay into a squeal. Where it shows up next Communication Systems (34/40): a radio station is a message shifted in frequency, a receiver is a filter, and every bandwidth limit is a statement about a spectrum. Control Systems (34/20): the Laplace half of this subject, with feedback added to move the poles. Embedded Systems (34/41): the anti-alias filter, the sample rate and the digital filter in firmware are all this subject on a chip with a tight budget. Machine Learning (34/32): a convolutional layer is literally a convolution with learned h, and speech models read spectrograms. The real world: noise-cancelling headphones, MP3 and JPEG, MRI scanners, seismology, radar, the equaliser in your music app, the vibration monitor on a wind turbine. If you want to go deeper 📘 Textbook: Alan V. Oppenheim and Alan S. Willsky (with S. Hamid Nawab), Signals and Systems, 2nd ed. (Pearson, 1997): the standard, used for decades; clear on convolution, Fourier, Laplace and z. 📘 Textbook: B. P. Lathi and Roger Green, Linear Systems and Signals, 3rd ed. (Oxford University Press, 2017): more worked examples and a friendlier pace; Oppenheim and Schafer's Discrete-Time Signal Processing is the next step for sampled signals. 📘 Gentler: Steven W. Smith, The Scientist and Engineer's Guide to Digital Signal Processing, free to read in full at dspguide.com, written for people who want to use the ideas without heavy maths. ▶️ Gentler: 3Blue1Brown, "But what is the Fourier Transform? A visual introduction" (YouTube, 2018). ▶️ Free course: MIT OpenCourseWare 6.003 Signals and Systems (Fall 2011), Prof. Dennis Freeman, with lecture videos and problem sets. 🌐 Easier first reads in this library: The Fourier Transform (12/12), Vibrations (34/17) and Waves and Sound (13/13). The takeaway A linear, time-invariant system is completely described by its reply to one click; the output is that reply, copied and added up (convolution). Sines go through unchanged in shape, so breaking a signal into sines turns the whole problem into multiplication by a frequency response, which is all a filter is. Sampling is safe above twice the highest frequency and a lie below it. The FFT makes frequency thinking cheap, and poles tell you whether a system fades or blows up. Try this: record a clap in the bathroom and in a carpeted bedroom with your phone's voice recorder, and listen to the tail. That tail is h. Then open any free spectrogram app, whistle a rising note, and watch the line climb. Sources & Further Reading Sources. LTI theory, convolution, filters and the sampling theorem are standard textbook material (Oppenheim and Willsky; Lathi). Fourier's 1807 memoir on heat: standard history (later published as Théorie analytique de la chaleur, 1822). Sampling theorem history: Nyquist (1928, Trans. AIEE 47), Kotelnikov (1933), Shannon (Proc. IRE 37, 1949); see the Wikipedia article on the Nyquist–Shannon sampling theorem for the fuller account, which also credits Whittaker (1915) and Ogura (1920). FFT: J. W. Cooley and J. W. Tukey, "An algorithm for the machine calculation of complex Fourier series", Mathematics of Computation 19 (1965) 297–301; Gauss's unpublished use around 1805 (Heideman, Johnson and Burrus, 1984). Gibbs overshoot (about 8.95% of the jump), the CD sample rate of 44.1 kHz and hearing near 20 kHz are standard. Computed in node: every number in the interactives, the 81% and 93% power shares, the 995 Hz cutoff, the 50,000-fold FFT saving, and the pole positions. 🌐 MIT OpenCourseWare 6.003 🌐 The Scientist and Engineer's Guide to Digital Signal Processing (free) 📘 Oppenheim and Willsky, Signals and Systems · Lathi and Green, Linear Systems and Signals Step 14. Electromagnetics & Waves: Why Fast Signals Stop Acting Like Wires -------------------------------------------------------------------------- Why: Above a certain speed, wires stop being wires and become waveguides and antennas. Why the cable to your aerial has to be a particular kind of cable, why a pulse can bounce back from the end of a wire, why antennas are the size they are, and why your microwave door has a mesh: four equations, and a speed. In an introductory circuit, a wire is a perfect line: whatever voltage you put in at one end appears at the other end at the same instant. That is a lie, and a useful one. Electrical signals travel at a large fraction of the speed of light, which sounds fast until you notice that light covers 30 cm in a billionth of a second. A modern processor's signals change in less than that. On the circuit board, a wire a few centimetres long is already a long journey. Then things go odd. A signal can bounce back from the far end of a cable as an echo. A TV aerial needs a cable of a particular "impedance". A radio aerial is a rod of a particular length. One subject explains all of it: a wire is not a wire, it is a guide for a wave. The course in one box Who takes it: electrical and computer engineers, usually in year 3; physicists take a theory-heavy cousin of it. What it's for: predicting how electric and magnetic fields move through space and along wires: waves, reflections, antennas and shielding. Comes after: Physics II: Electricity & Magnetism (34/07) and Multivariable Calculus (34/02), because fields are functions of position and the equations use divergence and curl. Leads to: Communication Systems (34/40), since every radio link begins with an antenna and a wave. It also feeds Power Systems & Electric Machines (34/39), where long transmission lines are the same maths at 50 or 60 Hz, and Embedded Systems (34/41), where board layout decides whether the chip works. Honest line: you won't be able to design an antenna or a high-speed board after this. You will see why a cable has an impedance, why an antenna has a length, and why fast edges, not fast clocks, are what make a circuit board hard. For a gentler first read, see Physics II (34/07). Maxwell's four equations, one picture each In the 1860s James Clerk Maxwell gathered what was known about electricity and magnetism into a small set of equations and added one missing piece. Oliver Heaviside later rewrote them into the four compact vector equations in textbooks today. In words, each one is a statement about how fields behave around a point in space. 1. Charge starts E lines 2. B lines never end S N 3. Changing B curls E B grows 4. Current curls B current Gauss's law: electric field lines start on positive charge and end on negative charge. ∇·E = ρ/ε₀ No magnetic charges: magnetic field lines have no beginning and no end; they form closed loops. ∇·B = 0 Faraday's law: a magnetic field that changes in time makes an electric field that circles around it. This is how a generator, a transformer and a wireless charger work. ∇×E = −∂B/∂t Ampère's law, with Maxwell's addition: a current makes a magnetic field that circles it, and so does an electric field that changes in time. ∇×B = μ₀J + μ₀ε₀ ∂E/∂t The last term is Maxwell's own, and it is the one that matters for this subject. Faraday says a changing B makes an E. Maxwell's term says a changing E makes a B. Each makes the other, so once you shake the fields they can keep each other going without any wire or charge, and leave the source behind as a travelling wave. Hertz showed in experiments in 1887 and 1888 that such waves exist, can be reflected and refracted, and travel at the speed of light. Waves, the speed of light, and the spectrum Put numbers into Maxwell's equations and a wave solution drops out with a speed you can calculate from two constants that you can measure with capacitors and coils on a bench. The speed is c = 1 / √(μ₀ε₀) = 299,792,458 m/s. Maxwell noticed that this was the speed of light, and concluded that light is an electromagnetic wave. Since 1983 the number is exact, because the metre is now defined from it: a metre is the distance light travels in 1/299,792,458 of a second. c = f · λspeed = frequency × wavelength, for any wave; for these, c ≈ 3 × 10⁸ m/s in vacuum One relation, and the entire spectrum follows. Slow waves are long and fast ones are short: at 50 Hz the wavelength is 6,000 km, at 100 MHz (FM radio) it is 3 m, at 2.45 GHz (Wi-Fi and microwave ovens) 12 cm, and for green light about 555 nanometres. Radio, microwaves, infrared, light, ultraviolet and X-rays are the same thing at different frequencies. What changes is how they interact with matter, and that decides what each band is used for. Inside a material the wave is slower: a coax cable with a plastic insulator carries it at about two-thirds of c, a glass fibre about 0.68 c, and a signal in a buried layer of a typical circuit board about half c (these depend on the insulator's permittivity, as 1/√εr). Interactive · The spectrum and its wavelengths Mains 50 HzFM 100 MHzWi-Fi 2.45 GHz5G mmWave 28 GHzGreen light Vacuum / airCoaxGlass fibreCircuit board Frequency λ = (speed in the medium) ÷ f. Band edges follow the usual ITU naming; the uses are examples, and allocations differ between countries. When a wire stops being a wire Now take a two-wire cable and connect a battery with a switch to one end. When you close the switch, the far end does not know yet. The wave has to travel down the cable, and at each instant the part of the line the wave has reached carries a voltage and a current in a fixed ratio: its characteristic impedance Z₀, set only by the geometry and the insulator, not by the length. Common ones: 50 Ω (lab, radio), 75 Ω (TV, video), about 100 Ω (Ethernet twisted pair). The wave then reaches the far end and meets whatever is connected there. If the load resistance is exactly Z₀, the wave sees what looks like more line, and all of it is absorbed. If it is anything else, the load cannot take that voltage and current ratio, and the difference goes back as a reflected wave: Γ = (RL − Z₀) / (RL + Z₀)reflection coefficient: +1 for an open end, −1 for a short, 0 when matched The reflection runs back to the source, and if the source impedance does not match the line either, it reflects again. The result is a staircase of voltage with ringing that settles only after several round trips. Worked example, in the interactive below: a driver of 20 Ω pushing a 1 V step down a 50 Ω line into an open end (a chip input draws almost no current). The first wave is 1 × 50/(20+50) = 0.714 V. At the open end the voltage doubles to 1.43 V, an overshoot of 43% above the 1 V it will finally settle at. It then rings down: 0.82, 1.08, 0.97, 1.01 V. When does this matter? A common rule of thumb is that a connection needs treating as a transmission line once it is longer than about a tenth of the wavelength of the highest frequency present. A signal edge that rises in 1 ns has significant content up to about 0.35/(1 ns) = 350 MHz. On a circuit board, where waves travel at about 0.48 c, that wavelength is about 41 cm, and a tenth of it is about 4 cm. So a signal trace longer than that, carrying a 1 ns edge, can ring. The clock speed does not matter. The edge speed does, which is why a slow clock on a fast chip is still a signal-integrity problem. Interactive · A pulse bouncing on a mismatched line Open endMatched far endShorted endLoad too smallMatched source Load resistance (far end) Source resistance (near end) A 1 V step on a lossless 50 Ω line, computed with the standard bounce (lattice) method. T is the one-way travel time: 5 ns for a 1 m coax cable, about 7 ps per millimetre in a typical circuit board. "Matched source" is 50 Ω at the source and an open far end: one reflection, then silence. Matching: absorbing the wave instead of bouncing it Every reflection is power that was not delivered. Engineers measure a mismatch in a few equivalent ways: the reflection coefficient Γ, the fraction of power reflected |Γ|², the standing-wave ratio (VSWR, which is what the little meter on an amateur radio shows), or the return loss in decibels. A 75 Ω TV cable plugged into a 50 Ω instrument has Γ = 25/125 = 0.2: that is 4% of the power reflected, a VSWR of 1.5, and 14 dB of return loss. For an antenna that is a small price; for a pulse train in a computer it can be fatal, because the echo arrives at the receiver just as the next bit does. The cures are all ways to give the wave something that looks like more line: a termination resistor equal to Z₀ at the end, a matching network, a transformer, or lines cut to exact fractions of a wavelength. Interactive · Power lost to a mismatch Matched 50 ΩTV cable 75 ΩTwisted pair 100 ΩPoor 10 Ω Load A real line with a real (resistive) load. The dashed green line marks the matched value, where nothing is reflected. Skin effect: current hugs the outside At steady DC, current spreads through the whole cross-section of a wire. As the frequency rises, the changing magnetic field inside the wire induces eddy currents that push the current out toward the surface. The current crowds into a thin layer, the skin depth: δ = √( ρ / (π f μ₀) )skin depth: grows with resistivity ρ, falls with frequency f (for non-magnetic metals) For copper at 60 Hz this is 8.4 mm, so the middle of a thick cable carries little: this is why high-voltage lines use steel-cored aluminium with the current in the outside strands, and why large conductors for 50 or 60 Hz are made of insulated strands or hollow tubes. At 1 MHz the skin is 65 µm, as thick as a hair. At 1 GHz it is 2 µm, so the inside of the conductor is irrelevant and only its surface finish matters, which is why microwave hardware is silver- or gold-plated. A 2 mm radius copper wire at 1 MHz has an AC resistance about 16 times its DC value in the thin-skin approximation. Interactive · How deep does the current go? CopperAluminiumSilver 60 Hz mains1 MHz (AM radio)1 GHz Frequency Wire size Resistivities at about 20 °C: copper 1.68×10⁻⁸, aluminium 2.65×10⁻⁸, silver 1.59×10⁻⁸ Ω·m. Skin depths ignore temperature and magnetism. The resistance ratio is the thin-skin approximation; it is accurate when the skin is much thinner than the wire. Antennas: why a quarter or a half wavelength A wire carrying a steady current does not radiate. One carrying a current that changes fast enough does: the changing magnetic field makes an electric field, which makes a magnetic field, and the disturbance leaves for good. The trouble is efficiency. A rod much shorter than a wavelength has the charge slosh back and forth across so small a distance that it radiates little and mostly heats up the wire. The special lengths come from the fact that current has to go to zero at the open end of a rod: there is nowhere for it to go. A rod that is half a wavelength long (a half-wave dipole) carries a standing wave with its current largest in the middle and zero at both ends, like a guitar string vibrating at its lowest note. At that length the rod resonates, its input looks like a pure resistance of about 73 Ω (so you can drive it without a matching coil) and both halves radiate in step. A quarter-wave rod above a metal plane does the same job, because the plane makes a mirror image of the missing half: a car's radio whip for 100 MHz is about 75 cm, a quarter of 3 m. Practical rods are cut a few per cent shorter than the free-space number because of end effects. Worked example: for Wi-Fi at 2.45 GHz, a wavelength is 12.2 cm, so a half-wave rod is 6.1 cm and a quarter-wave stub is 3.1 cm. That is why a router antenna is a stubby rod and an FM aerial is a long one. A dipole radiates nothing along its own axis and most at right angles to it, which is why you tilt a TV aerial. Interactive · Cut an antenna to length FM 98 MHz433 MHz900 MHzWi-Fi 2.45 GHz5.8 GHz Rod length Ideal thin rod, centre-fed, sine-shaped current (the textbook model). The pattern is from the standard formula [cos(kL/2·cosθ) − cos(kL/2)] / sinθ; directivity and feed resistance are integrated numerically (half-wave: 1.64 and 73.1 Ω). Shielding and EMC Every circuit is also an antenna, and every antenna is also a receiver. Electromagnetic compatibility (EMC) is the discipline of making sure a device neither radiates enough to disturb others nor is easily upset by them. The law follows the physics: the EU's EMC Directive (2014/30/EU) requires equipment to neither generate nor be disturbed by electromagnetic disturbances beyond set levels, and in the US the FCC's Part 15 limits what unintentional radiators may emit. The main tool is a conducting shield. Charges in a closed metal box move until the fields inside cancel, so a closed box shields its contents, and a mesh does nearly as well as long as the holes are much smaller than the wavelength. The microwave oven is the classic case: its door window has a metal mesh with holes a millimetre or two wide, far below the 12 cm wavelength at 2.45 GHz, so the microwaves reflect off it while visible light, whose wavelength is a few hundred thousand times smaller, passes through. Loops are the enemy: by Faraday's law a loop picks up noise in proportion to its area, and a twisted pair (Ethernet) has loops that alternate in direction and cancel. What a graduate just sees Any wave has c = f·λ; a "long" wire is long compared with the wavelength, not with a ruler. A signal's trouble is in its edges: a fast edge contains high frequencies even if the clock is slow. A cable has an impedance that doesn't depend on its length; anything that isn't that impedance reflects part of the wave. An open end doubles the voltage, a short flips it: reflections explain ringing, overshoot and glitches. Current flows on the surface at high frequency, so the inside of a conductor is wasted metal. An antenna is a resonator: its length is about half or a quarter of the wavelength, and a small one is always a compromise. A shield works if its holes are small against the wavelength; a loop collects noise in proportion to its area. Where it shows up next Communication Systems (34/40): an antenna and a channel are the first and last parts of every radio link, and the band you pick decides the antenna size and how far the signal goes. Power Systems & Electric Machines (34/39): long high-voltage lines are transmission lines too, and skin effect decides conductor design. Embedded Systems (34/41) and Computer Architecture (34/25): memory buses and chip-to-chip links are carefully matched lines; a layout rule like "keep the clock trace short" is a wavelength rule. Signals & Systems (34/37): the spectrum of an edge is why edges are the problem. The real world: Wi-Fi routers, GPS, radar, MRI, fibre-optic cables, noise in audio gear, and a phone passing its radiated-emissions test. If you want to go deeper 📘 Textbook: Fawwaz T. Ulaby and Umberto Ravaioli, Fundamentals of Applied Electromagnetics (Pearson): the usual engineering text, strong on transmission lines, waves and antennas. 📘 Textbook: David J. Griffiths, Introduction to Electrodynamics (Cambridge University Press): the physicist's favourite, brilliant on the equations; for the microwave and board side, David M. Pozar's Microwave Engineering (Wiley). 📘 Gentler: The Feynman Lectures on Physics, Vol. II, free online, for the physical picture of the fields and of light as a wave. ▶️ Free course: MIT OpenCourseWare 6.013 Electromagnetics and Applications (Spring 2009). 🌐 Easier first reads in this library: Physics II (34/07), Waves and Sound (13/13) and The Fourier Transform (12/12). The takeaway Maxwell's equations say that changing electric and magnetic fields make each other, so they can travel as a wave at c = fλ. A wire is a wave guide with an impedance: any mismatch reflects part of the wave, and fast edges, not fast clocks, make a board behave like a transmission line. At high frequency current stays near the surface, antennas are sized by the wavelength (half or a quarter), and shields work when their holes are small against it. Try this: look at the back of your Wi-Fi router, then at the car aerial or the FM radio's wire. The router's rod is a few centimetres long and the FM wire is about a metre: one is a half or quarter of 12 cm, the other of 3 m. Then check your microwave: shine a torch at the door and see the mesh. Sources & Further Reading Sources. Maxwell's equations, transmission-line theory, skin effect and dipole antenna theory are standard textbook material (Ulaby; Griffiths; Pozar; Balanis, Antenna Theory). Maxwell's field theory dates from 1861 to 1865 (see Wikipedia, "Maxwell's equations", which credits Heaviside with the modern vector form); Hertz's experiments 1886 to 1889 (Wikipedia, "Heinrich Hertz"). The speed of light exactly 299,792,458 m/s since the 1983 redefinition of the metre (17th CGPM, Resolution 1; BIPM and NIST). Copper skin depth of 8.4 mm at 60 Hz, 65 µm at 1 MHz, 2 µm at 1 GHz: Wikipedia, "Skin effect"; resistivities from standard tables (CRC Handbook). FM 87.5 to 108 MHz (88 to 108 in the Americas) and AM medium wave 531 to 1602 kHz (530 to 1700 in the Americas): ITU and Wikipedia, "FM broadcast band" and "Medium wave". EU EMC Directive 2014/30/EU and FCC 47 CFR Part 15: the regulations themselves (eCFR; EUR-Lex). The "tenth of a wavelength" and 0.35/rise-time rules are rules of thumb. Computed in node: every number in the interactives, including the bounce voltages, the 73.1 Ω and 1.64 dipole values, skin depths and the 16-fold AC resistance. 🌐 MIT OpenCourseWare 6.013 🌐 The Feynman Lectures on Physics, Vol. II (free) 📘 Ulaby and Ravaioli, Fundamentals of Applied Electromagnetics · Griffiths, Introduction to Electrodynamics · Pozar, Microwave Engineering Step 15. Computer Architecture: Why Your Processor Spends Its Life Waiting -------------------------------------------------------------------------- Why: What the processor you are designing around actually does. A core can start several instructions every nanosecond, then waits hundreds of ticks for one number from memory. Caches, pipelines and extra cores are all tricks for hiding that wait, and the reason clock speeds stalled is a power bill. A modern processor core can start several instructions on every tick of a clock that ticks around four billion times a second. Then it needs one number that isn't in its cache, asks main memory for it, and waits about 60 nanoseconds. On one measured chip that is roughly 250 ticks, long enough to have started on the order of a thousand instructions. Nearly everything a computer architect does is a trick for not noticing that wait. There is a second puzzle in the same machine. The fastest Pentium 4 chips ran at 3.8 GHz, and in 2004 Intel cancelled its plans for a 4 GHz one and turned to dual-core designs. Two decades later, typical clock speeds are still only a few GHz, yet the chip in your pocket is many times faster than that desktop. Speed came from somewhere other than a faster clock. This piece is about both puzzles: why a processor spends its life waiting, and why it got faster anyway. Along the way you will build an adder out of gates, encode a real instruction by hand, and break a cache on purpose. The course in one box Who takes it: computer science and computer engineering students, usually in the second year. What it's for: showing what a processor physically does with your program, so that you can predict why some code is fast and some is slow. Comes after: Programming Fundamentals (34/09) and Discrete Mathematics (34/08). Leads to: Operating Systems (34/26), then Computer Networks (34/27) and Computer Security (34/33); it also quietly underlies Algorithms (34/24) and Programming Languages & Compilers (34/30). Honest line: you won't be able to design a chip after this piece. You will see why a loop over a table is fast one way and slow the other, and why your phone has eight cores and not one 20 GHz core. For a gentler first read on the history, see Processors: How a CPU Thinks (29/11); this piece goes into the mechanisms. GPUs (29/12) tells the graphics-chip story that appears here only as a single idea. Everything is a pattern of bits A bit is a wire that is either low or high. Eight of them in a row make a byte, and the meaning of the pattern lives in how you read it, not in the wire. The same eight bits can be a small number, a negative number, a letter or the colour of a pixel. Read as a plain number, each position counts double the one to its right: 1, 2, 4, 8 and so on. Negative numbers use two's complement: the leftmost bit counts as minus its weight instead of plus. In eight bits that makes the top bit worth −128, so 1111 1111 is −128 + 127 = −1. value = Σ bitk × 2k    (top bit counts as −27 in two's complement) Interactive · Flip the bits Add 1Flip all (NOT)Clear Tap to make 0111 1111 (127), then press Add 1. The answer is −128. The hardware did nothing wrong: the counter ran out of room and wrapped, the way a car odometer rolls over. Programs that forget this have real bugs, and a 64-bit register simply moves the cliff a long way off: it holds 264, about 1.8 × 1019, different patterns. Gates add numbers Adding is the first thing a processor learns to do with bits, and it needs no new idea. Add two bits and you get a sum bit and a carry bit. The sum is 1 when exactly one input is 1 (an XOR gate); the carry is 1 when both are (an AND gate). Chain one such unit per column, passing each carry into the next column, and you have a ripple-carry adder. sum = A ⊕ B ⊕ carry-in   ·   carry-out = (A · B) + (carry-in · (A ⊕ B)) Interactive · A clickable adder Tap the A and B bits. Watch the carry travel left, one gate stage at a time. Each column is a full adder: five gates (two XOR, two AND, one OR). Sixty-four columns need 64 × 5 = 320 gates. Try 0111 + 0001: the carry ripples through three columns before the answer settles at 1000. The carry's path through each column is two gates long, so a 64-bit ripple adder has a path of 128 gate delays, which is slow. Real processors use carry-lookahead and cleverer variants that compute many carries at once, trading more gates for less waiting. That trade (spend hardware, save time) comes back in every idea below. Subtraction costs almost nothing extra. To compute A − B, flip every bit of B and add 1 (that is two's complement negation), and use the same adder. One circuit does both. Multiplication, comparison and the rest of the arithmetic unit are built in the same way, from the same few gates. A program is numbers in memory: the instruction set The processor's loop is short enough to say in four steps. Fetch the instruction whose address is in the program counter. Decode it: which operation, which registers. Execute it in the arithmetic unit, memory or branch logic. Write back the result and move the counter on. Then repeat, billions of times a second. If you want to watch that loop run, the toy CPU in Processors (29/11) steps through it. Here is the part that is easy to miss: the instruction itself is just a number. The list of instructions a chip understands, and how each is written as bits, is its instruction set architecture (ISA). It is the contract between hardware and software: x86-64 (most PCs and servers), ARM (most phones) and RISC-V (open and growing) are different contracts. A register is one of a small number of named slots inside the core, typically 16 to 32 general ones, that instructions read and write directly. Registers are the fastest storage there is, and the compiler's main job is to keep the hot values in them. Interactive · Hand-assemble a RISC-V instruction Result into x  from x  and x With the defaults, add x5, x6, x7 ("put x6 + x7 into x5") becomes the 32-bit pattern whose hex is 0x007302B3. The fields are fixed-width slices: a 7-bit opcode, three 5-bit register numbers, and 10 bits that pick which arithmetic. Because the fields are always in the same place, decoding is mostly wiring, not thinking. That regularity is the RISC idea: keep instructions simple and uniform so the hardware can be fast. x86 instead has instructions from 1 to 15 bytes long and decodes them into simpler internal steps first. Both work; they are different bets. Memory is a hierarchy, and distance is time Fast memory is small and expensive; big memory is slow and cheap. So computers stack them. A few registers, then a small L1 cache, bigger and slower L2 and L3 caches, then main memory (RAM), then an SSD or hard disk. Each level holds copies of the most recently used pieces of the level below. Numbers make the gap real. Here are measured latencies on one desktop chip (an Intel Core i7-6700 at 4.0 GHz), with the disk figures from the widely quoted "latency numbers" list. The last column scales one clock tick to one second so the gaps are human-sized. LevelTimeTicks at 4 GHzIf a tick were 1 s L1 cache1 ns44 seconds L2 cache3 ns1212 seconds L3 cache10.5 ns4242 seconds Main memory~62 ns~250about 4 minutes SSD random read (2012-era figure)150 µs600,000about 7 days Hard-disk seek10 ms40,000,000about 1.3 years Caches work because programs are lazy in a predictable way: after touching a byte they usually touch its neighbours (spatial locality) and touch the same byte again soon (temporal locality). A cache therefore never loads one byte; it loads a whole line of 64 bytes. The rule a graduate remembers is the average-time formula, with hits cheap and misses expensive: average access time = hit time + miss rate × miss penalty Worked: with a 1 ns hit and a 62 ns miss, a 1% miss rate averages 1 + 0.01 × 62 ≈ 1.6 ns; a 10% miss rate averages about 7.2 ns, more than four times worse. The following simulator makes the point with real code. A 32 × 32 table of 4-byte numbers sits in memory row after row. Sum it in two orders. Interactive · Break the cache Cache size (64-byte lines) Red = miss, green = hit. Simplified: a direct-mapped cache (each line has exactly one possible slot), no prefetcher, hit 1 ns, miss 62 ns. Real caches are set-associative and prefetch, which softens these cliffs but does not remove them. Row by row, only 64 of the 1,024 accesses miss, because one load brings in 16 numbers. Column by column with 64 lines it is the same 64, but with 32 or fewer lines it collapses to 1,024 misses out of 1,024 (the two halves of the table fight for the same slots) and the run is roughly thirteen times slower. Same arithmetic, same answer. Choose "In 8×8 tiles" and even a 16-line cache gets back to 64 misses, because each small block of the table is finished while it is still cached. This is why programmers who know architecture tile their loops, and why linked lists scattered around memory lose to arrays. Pipelining: an assembly line for instructions Doing one instruction fully before starting the next wastes the hardware, as if a laundromat ran one load through wash, dry and fold before touching the next. A pipeline overlaps them. The textbook design has five stages: fetch, decode, execute, memory, write-back. When the line is full, one instruction finishes per tick even though each takes five. cycles ≈ instructions + (stages − 1) + stall cycles For 100 instructions that is about 104 ticks instead of 500: nearly a 5× gain from the same parts. The catch is hazards. If an instruction needs a result the one ahead of it hasn't finished, the line must wait, and the empty slot is called a bubble. A branch is worse: until the processor knows which way it goes, it guesses (branch prediction) and starts fetching down that path. On a wrong guess the half-done work is thrown away, which on a modern deep pipeline costs roughly 15 to 20 ticks. Interactive · Bubbles in the pipeline A hazard hits every Each hazard costs Blue = the instruction is in a stage (F D X M W), red = a bubble. First 8 instructions drawn; the numbers use 100. Slide the cost to 2 ticks every 4th instruction: the 100 instructions take 152 ticks, about 1.5 ticks each, and the gain over no pipeline falls from nearly 5× to about 3.3×. Push the cost to 16 ticks, like a mispredicted branch, and a hazard every 12th instruction cuts the gain to about 2.2×. That is why predictors are among the most elaborate parts of a modern core. Parallelism helps, up to the part you can't split When one core can't go faster, use more of them. Four cores can run four programs at once, or one program split in four. The limit was written down by Gene Amdahl in 1967: whatever part of the job must run in sequence caps the gain, however many workers you add. In its usual form, with p the fraction of the work that can be parallelised and N the number of cores: speedup = 1 ÷ ( (1 − p) + p ÷ N ) Interactive · Amdahl's wall Fraction that can run in parallel With 95% parallel work, 16 cores give 9.1×, not 16×, and no number of cores can ever beat 20×, because the 5% serial part is still there. Even at 99.9% parallel, 1,024 cores give only about 506×. A GPU is the extreme version of the bet: thousands of simple lanes running the same instruction on different data. It shines on graphics and matrix maths, where p is nearly 1, and is poor at code full of branches and dependencies. See GPUs (29/12) for how that bet turned into the AI hardware industry. Amdahl's pessimism has a famous rebuttal (Gustafson, 1988): bigger machines are usually given bigger problems, which keeps p high. Both are true, in different jobs. The power wall: why the clock stopped Every time a transistor switches it charges and discharges a tiny capacitor, which burns energy. Add up billions of them, thousands of millions of times a second, and you get heat that has to leave a chip the size of a postage stamp. The dynamic power of a chip is: P ≈ C × V2 × f Capacitance, voltage squared, frequency. For decades, shrinking transistors cut C and V so fast that you could raise f for free in power-per-area terms: this is Dennard scaling, from a 1974 paper by Robert Dennard and colleagues. Around 2005 to 2007 it broke, because leakage current and the threshold voltage stopped shrinking with the transistor, so voltage could no longer fall. Intel's Prescott Pentium 4 already drew up to 103 watts, and on 7 May 2004 Intel cancelled its successors, Tejas and Jayhawk, and pivoted to dual-core. Herb Sutter's 2005 article "The Free Lunch Is Over" told software developers what followed: programs would stop getting faster for free and would have to use several cores. Interactive · Spend your power budget Clock speed per core Number of cores Model: voltage rises in proportion to clock speed, so one core's power grows as the cube of the clock; throughput assumes perfectly parallel work (see Amdahl above). Power 1.0 = one core at 3 GHz. Real chips can't lower voltage below a floor, so the cube law flattens at low clocks. Under that model, doubling the clock from 3 to 6 GHz needs 23 = 8 times the power. Two cores at 3 GHz give the same ideal throughput for 2 times the power, and four cores at 1.5 GHz for half a unit. The modern chip is the answer: many modest cores, some specialised units, and parts switched off when idle, because a chip can now hold more transistors than it can power at once. What a graduate just sees Bits have no meaning until you choose how to read them; overflow is the odometer rolling over. Arithmetic is a few gates, and the slow part of an adder is the carry travelling across it. An instruction is a number, and a program is a list of them in memory. Distance is time: a trip to RAM costs hundreds of ticks, so access patterns matter more than operation counts. Walking memory in order is nearly free; jumping around it is not. A pipeline is full only if the code lets it be, and every branch is a bet. More cores help only as far as the work can be split (Amdahl). Power rises with the square of voltage and the cube of speed, so wide and slow beats narrow and fast. Where it shows up next Operating Systems (34/26) rests on hardware features introduced here: interrupts, the user/kernel privilege split, and the address-translation hardware behind virtual memory. Computer Security (34/33): the Spectre and Meltdown flaws, disclosed in January 2018, came from speculative execution (the guessing described under pipelining) leaking data through the cache. Algorithms (34/24) and Data Structures (34/23): two algorithms with the same operation count can differ tenfold in speed because of locality. Compilers (34/30): a compiler's whole output is instructions in one ISA, and its hardest job is choosing registers. Machine Learning (34/32): training is matrix multiplication on GPUs and similar chips, where Amdahl's p is nearly 1. If you want to go deeper 📘 Textbook: David Patterson and John Hennessy, Computer Organization and Design: RISC-V Edition (2nd ed., 2020). The standard first course; builds a processor step by step. 📘 Textbook: Hennessy and Patterson, Computer Architecture: A Quantitative Approach (7th ed., late 2025, with Christos Kozyrakis). The graduate-level reference for caches, parallelism and accelerators. 📘 Programmer's view: Randal Bryant and David O'Hallaron, Computer Systems: A Programmer's Perspective (3rd ed.). Best for locality and for how code becomes machine instructions. 📘 Gentler: Charles Petzold, Code: The Hidden Language of Computer Hardware and Software (2nd ed., 2022). Builds from a flashlight signal to a CPU with no maths beyond arithmetic. 🌐 Free, hands-on: Nand2Tetris (Nisan and Schocken, The Elements of Computing Systems): build a working computer from a NAND gate up, free in a non-profit setting. 🌐 Free course: MIT OpenCourseWare 6.004 Computation Structures (Spring 2017): lectures and problem sets from gates to a pipelined processor. The takeaway A processor is gates arranged to follow a list of numbers, and almost all of its cleverness goes into hiding two limits: memory is hundreds of ticks away, and power grows with the cube of speed. Caches, pipelines and extra cores are three ways of spending transistors to hide them, each with a cliff (misses, hazards, the serial fraction). Try this: write a loop that sums a big two-dimensional array (a spreadsheet-sized grid in Python with NumPy, or any language), once row by row and once column by column, and time both. The gap you find is the memory hierarchy. Sources & Further Reading Sources. Cache and memory latencies for the i7-6700 (4.0 GHz): 7-cpu.com, "Intel Core i7-6700 Skylake"; disk and SSD figures: the "Latency Numbers Every Programmer Should Know" list (Jeff Dean's figures as circulated, c. 2012). Dennard scaling and the power wall: Dennard et al., IEEE JSSC (1974); Wikipedia, "Dennard scaling". Tejas and Jayhawk cancellation (7 May 2004) and Prescott power: The Register, EE Times and Wikipedia, "Tejas and Jayhawk"; Pentium 4 top speed 3.8 GHz and cancelled 4 GHz part: Wikipedia, "Pentium 4". Branch mispredict penalty of 15 to 20 cycles (Skylake): Agner Fog's measurements as cited by Chips and Cheese. Sutter, "The Free Lunch Is Over", Dr. Dobb's Journal 30(3), March 2005. Amdahl, AFIPS Spring Joint Computer Conference (1967); Gustafson, Communications of the ACM (1988). RISC-V R-type encoding: The RISC-V Instruction Set Manual, Volume I. All interactive numbers were recomputed in code. 🌐 nand2tetris.org 🌐 MIT OpenCourseWare: 6.004 Computation Structures 🌐 Herb Sutter: The Free Lunch Is Over 📘 Code, Charles Petzold (Microsoft Press, 2nd ed., 2022) Step 16. Control Systems: How Machines Hold a Setpoint ------------------------------------------------------ Why: Feedback holds a voltage, a motor speed or a temperature steady. A thermostat, a cruise control and a hovering drone do the same thing: measure, compare, correct. Why that works, and why too much of it makes things oscillate. You set the cruise control to 100 km/h and drive up a long hill. The engine works harder and the speed barely moves. Nobody told the car about the hill. A thermostat holds a room at 21 °C through a heatwave and a cold snap. A drone hovers in a gusty wind as if hung from a hook. A rocket booster lands upright. All of them do the same thing: they measure what is happening, compare it with what should be happening, and push on the difference. That loop is feedback, and control systems is the course on how to do it well, and what happens when you do it badly (it oscillates, sometimes violently). The course in one box Who takes it: mechanical, aerospace, electrical and chemical engineers, usually in year 4 (sometimes late year 3). What it's for: making a machine hold a speed, a heading, a temperature or a position on its own, even when the world pushes back. Comes after: Differential Equations (34/03), Vibrations (34/17) and Linear Algebra (34/04). Leads to: Systems Engineering & the Capstone (34/22), where the controller has to agree with every other subsystem. In Computer Science it meets Software Engineering (34/31) in embedded code and Machine Learning (34/32) in reinforcement learning, which is control with a learned rule. Honest line: you won't be able to design a flight controller after this. You will see why a loop oscillates when its gain is too high, and what the three letters PID are doing in the car you drive. Open loop versus closed loop A toaster is open loop: you pick a time, it heats for that long and stops, whether the bread is pale or black. It never looks at the result. A thermostat is closed loop: it measures the room and decides, again and again, whether to heat. Open loop works when you know the system exactly and nothing disturbs it. The real world is not like that, and closed loop is how machines cope. Here is the cruise control, as a toy car with a first-order lag (a time constant of 25 s). In open-loop mode the throttle is set once to hold 25 m/s on flat ground. Add a hill and watch. Interactive · Open loop, then closed Open loopClosed loop (P) Hill steepness Feedback gain Open loop on a 2% hill: speed drops about 4.7 m/s (17 km/h) and stays down. A toy car: 1,200 kg, drag 48 N per m/s, engine force 0 to 3,000 N, starts at 20 m/s. Closed loop adds force in proportion to the speed error. Gain 600 cuts the loss on a 2% hill to about 0.36 m/s. Feedback and the block diagram Control engineers draw every loop as a block diagram: boxes for things that transform a signal, arrows for the signals. The set speed goes in; a comparator subtracts the measured speed to give the error; the controller turns the error into a command; the actuator (here the engine) does the pushing; the plant (the car) responds; the sensor (a speed sensor) reports back. A disturbance, the hill, enters from the side. ControllerActuatorPlantSensor set speederrorPIDenginecarhillspeed−+ Why is this so powerful? Suppose the plant's gain G is uncertain by ±30% (a heavy load, a worn part). Open loop passes that uncertainty straight to the output. With feedback and a high controller gain K, the output is KG/(1 + KG), which is close to 1 for any G, so the loop all but ignores the uncertainty. That is also why feedback rejects disturbances: the error shows up and the controller pushes back. Interactive · Feedback shrinks uncertainty Controller gain K With K = 10, plant gains of 0.7, 1.0 and 1.3 give 0.875, 0.909 and 0.929 instead of 0.7, 1.0 and 1.3. Command = 1. Open loop output = G; closed loop output = KG ÷ (1 + KG). The leftover shortfall is what integral control, below, removes. First- and second-order response: time constant, overshoot, settling To judge a controller you look at its step response: how the output follows when the command suddenly jumps. Two shapes cover most of what you meet. First order is a smooth exponential approach, like the car's speed or a cooling cup of tea. It has one number, the time constant τ: the output covers 63% of the gap in one τ, 95% in 3τ and 98% in 4τ, which engineers call "settled". It can never overshoot. Second order has inertia as well as lag, like the mass-spring of Vibrations (34/17), and it is described by the very same two numbers: the natural frequency ωn and the damping ratio ζ. Low ζ gives fast but overshooting ringing; ζ near 0.7 gives about 5% overshoot and is a common design target. The overshoot depends only on ζ: 53% at ζ = 0.2, 16% at 0.5, 4.6% at 0.7. Settling time is roughly 4 ÷ (ζωn). Interactive · Step response First orderSecond order Time constant τ Natural frequency ωₙ Damping ratio ζ First order, τ = 2 s: 63% at 2 s, 98% at 8 s. Never overshoots. Exact formulas, checked against a step-by-step numerical solution. In second order: ζ = 0.2 gives 52.7% overshoot, 0.5 gives 16.3%, 0.7 gives 4.6%. Transfer functions and Laplace, in plain words Here is the trick that makes control theory work. Differential equations are awkward to combine: put two systems in a chain and you must solve their equations together. The Laplace transform turns calculus into algebra. It replaces "take the derivative" with "multiply by s", so that a differential equation becomes a fraction, the transfer function: output divided by input, written in terms of s. Chains of systems become products, and feedback loops become the formula you already met, G ÷ (1 + GC). The part a graduate carries for life is what the fraction's roots, the poles, mean. Each pole is an exponent the system can "ring" with: a pole at s = −σ gives e−σt, a decay with time constant 1/σ. A pair at s = −σ ± jω gives a decaying oscillation at frequency ω. The first-order car has one pole at −1/τ. The mass-spring has the pair −ζωn ± jωn√(1 − ζ²). If every pole sits on the left side of the s-plane, every response dies out: the system is stable. One pole on the right and something grows without limit. Interactive · Place a pole Real part (decay or growth) Imaginary part (ringing) Pole at s = −0.8 ± 3.0j: stable, ringing about 0.5 times a second. The response plotted is eσt·cos(ωt). Left of the vertical axis everything decays; right of it, everything grows. PID control, explained with a cruise control The workhorse controller turns the error into a command with three terms: P (proportional): the present. Push in proportion to the error now. Simple, but it leaves a permanent leftover error when something (the hill) needs a steady push, because a push needs an error to exist. I (integral): the past. Add up the error over time and push on the total. A leftover error keeps growing the sum until the leftover is gone. This is what removes the steady-state error, but too much of it overshoots. D (derivative): the future. Push against how fast the error is changing, a guess at where it is heading. It adds damping, but it amplifies sensor noise. u = Kp·e + Ki·∫e dt + Kd·de/dtcommand = P × error + I × accumulated error + D × rate of change of error This three-term rule runs most of industry. Åström and Hägglund, authors of the standard text on PID, report that more than 95% of the control loops in process industries are of the PID type, and most of those are really PI. Ziegler and Nichols published the classic tuning rules in 1942 and they are still taught. Try tuning the cruise control yourself. Start with P only: see the leftover. Switch to PI: the leftover vanishes. Then switch on sensor noise and add D. Interactive · Tune the cruise control P onlyPIToo much IPID Clean sensorNoisy sensor Hill steepness P gain I gain D gain P only on a 2% hill: about 0.36 m/s below the set speed, for good. With PI: back to the set speed. The same toy car, with a feed-forward trim of 1,200 N plus PID on the measured speed (anti-windup included). Noise: 0.15 m/s of random error on the speed sensor, fixed seed so runs repeat. Stability: why too much gain oscillates If a little feedback is good, is more better? Not forever. Imagine steering a car by looking only at your distance from the centre line. If you correct with too much force, you overshoot, correct back too hard, and weave. A real loop has delays: sensors take time, computers take time, motors take time. A correction that arrives late can push at the wrong moment, adding energy instead of removing it. Past a certain gain a loop goes from settling to ringing to growing: it has gone unstable. It is the same negative-damping story as the bridges in Vibrations. The drone below holds an altitude. Its plant is almost pure inertia (thrust makes acceleration), so P alone swings it up and down forever: there is no damping in the plant. The D term is the missing damper (ζ = Kd ÷ 2√Kp). Now add what real hardware does: delay the commands, or update them only every so often. Interactive · Hold the altitude P onlyWell damped250 ms delayUpdates every 0.3 s P gain D gain Delay in the loop Controller update interval Well damped: settles on the target with no overshoot. A 1 kg drone, thrust limited to ±1 g about hover, 0.3 m step. P only never settles (it swings at √Kp rad/s). With the D term, a 250 ms delay is enough to make it grow, and a 0.3 s update interval makes the higher-gain setting ring. Sensors, actuators and digital control A loop is only as good as what it can see and what it can do. Sensors (encoders, gyros, thermocouples, pressure and GPS) set the noise floor, and you can't control what you can't measure. Actuators (motors, valves, heaters, control surfaces) have limits: the engine in the sim can't produce negative force or more than 3,000 N. When an actuator saturates while the integral keeps adding up, the controller "winds up" and overshoots badly, so real controllers freeze the integral at the limits. The sim does exactly that. Today nearly every controller is digital: a microcontroller samples the sensor, computes, writes an output, and holds it until the next sample. The delay and the holding are real and are what the "update interval" slider is imitating. A textbook rule (Franklin, Powell and Emami-Naeini) is to sample about 20 times faster than the closed-loop bandwidth you want, and other texts quote anywhere from 10 to 30. So a flight controller usually runs its loop tens to hundreds of times a second. This is where this course meets Computer Science: a control loop is a real-time program with a deadline, and missing the deadline is a stability problem, not a slowdown. What a graduate just sees Every automatic machine is a loop: measure, compare, correct. Ask what the sensor sees and what the actuator can do. Open loop is a bet that nothing changes; feedback is how you stay right when it does. A first-order system has a time constant (98% settled in four of them); a second-order one has ζ and ωn, and ζ sets the overshoot. Poles tell the story: left of the axis it decays, right of it it grows, and the imaginary part is the ringing. P alone leaves an error, I removes it, D adds damping, and each has a cost (overshoot, windup, noise). More gain is faster until delay turns correction into a push in the wrong direction. Real actuators saturate, real sensors are noisy, and real computers sample. A controller is a damper you can reprogram. Where it shows up next Systems Engineering & the Capstone (34/22): a controller is useless without sensors, actuators, power and software that agree with each other; the capstone is where they are tested together. Vibrations (34/17): feedback is active damping; a flight controller does for a drone what a viscous damper does for a bridge. Propulsion (34/21): engines run on control loops for fuel flow and turbine speed. The real world: thermostats, cruise control, 3-D printers, robot arms, drones, aircraft autopilots, rockets, chemical plants, insulin pumps. Computer Science and Electrical Engineering: embedded and real-time software (Embedded Systems, 34/41; Operating Systems, 34/26) run the loops; reinforcement learning (34/32) learns a control rule from experience instead of deriving it. If you want to go deeper 📘 Textbook: Franklin, Powell and Emami-Naeini, Feedback Control of Dynamic Systems (Pearson; 8th edition): the standard undergraduate text, with a full digital control chapter. 📘 Textbook: Katsuhiko Ogata, Modern Control Engineering (Pearson): a long-time classroom favourite, strong on transfer functions and root locus. 📘 Gentler (free): Karl Johan Åström and Richard Murray, Feedback Systems: An Introduction for Scientists and Engineers (Princeton University Press; 2nd edition, 2021), which its authors keep available free online. Also Brian Douglas's Control Systems Lectures on YouTube and Steve Brunton's Control Bootcamp, both free video series. ▶️ Free course: MIT OpenCourseWare 2.04A Systems and Controls (Spring 2013): from differential equations to Laplace, transfer functions and root locus, with a lab on a rotating flywheel. 🌐 Easier first read in this library: Fourier Transform (12/12). The takeaway Feedback means measuring, comparing and correcting, and it lets a machine hold a setpoint despite hills, loads and uncertainty. Systems respond with a time constant (first order) or with overshoot set by the damping ratio (second order), and their poles show whether they will decay, ring or blow up. PID combines the present, the past and the future of the error, and every term costs something. Too much gain, or too much delay, turns correction into oscillation, which is why stability comes before performance. Try this: take a hot shower and adjust the tap. Notice how long the water takes to answer. If you turn it again before the last change arrives, you swing between scalding and freezing. That is a loop with delay and too much gain, and you are the controller. Sources & Further Reading Sources. Block diagrams, step response, Laplace and stability are standard textbook material (Franklin et al.; Ogata; Åström and Murray). PID prevalence (over 95% of process-control loops, mostly PI): K. J. Åström and T. Hägglund, PID Controllers: Theory, Design and Tuning and Advanced PID Control. Tuning rules: J. G. Ziegler and N. B. Nichols, "Optimum settings for automatic controllers", Transactions of the ASME 64 (1942) 759–768. Sampling rule of thumb: Franklin, Powell and Workman / Emami-Naeini, discussion of digital control (about 20 times the bandwidth; other texts say 10 to 30). Computed in node: the exact step responses (checked against numerical integration), overshoot and settling figures, the cruise-control and drone simulations and the stability thresholds quoted in the notes. 🌐 MIT OpenCourseWare 2.04A Systems and Controls 🌐 Åström and Murray, Feedback Systems (2nd edition site) ▶️ Steve Brunton, Control Bootcamp 📘 Franklin, Powell and Emami-Naeini, Feedback Control of Dynamic Systems Step 17. Power Systems & Electric Machines: Supply Must Match Demand, Every Second ---------------------------------------------------------------------------------- Why: How a power station, a transformer and a motor keep a continent switched on. The biggest machine ever built is the electric grid: a continent of spinning rotors held in step. How it moves power, turns it into motion, and what happens when the balance breaks. When the penalty shoot-out in England's 1990 World Cup semi-final ended, millions of people in Britain got up and put the kettle on. National Grid's records are reported to show demand jumping by about 2,800 megawatts within minutes. A power station had to supply that, on cue, with no warning. Electricity cannot be stored in the wires. Whatever is drawn at this second has to be generated at this second. That fact organises the whole subject. Generators are huge spinning machines, and they are tied together so tightly that the whole network turns in step. Demand is what slows that rotation down, and generation is what speeds it up. Keep them equal and the mains hum at 50 or 60 cycles a second. Let them drift apart and the hum changes pitch, and then things start tripping. This is power systems, and the machines at both ends of the wire, the generators and the motors, are electric machines. The course in one box Who takes it: electrical engineers, usually in year 4, often as two courses (machines, then power systems). Mechanical and aerospace students meet motors in a shorter form. What it's for: generating, moving and using electrical energy at the scale of cities, and keeping the whole network stable. Comes after: Circuit Analysis (34/34) and Physics II: Electricity & Magnetism (34/07). Leads to: the capstone, Systems Engineering & the Capstone (34/22), and to jobs in utilities, wind and solar, electric vehicles and industrial drives. It also meets Control Systems (34/20), since the grid is the biggest feedback loop there is. Honest line: you won't be able to plan a grid after this. You will see why power travels at hundreds of thousands of volts, why a motor spins slightly slower than the field that drives it, and why engineers worry about a few hundredths of a hertz. High voltage and the transformer You met the rule in Circuit Analysis (34/34): heat in a wire is I² × R. To send a given power you can have a high voltage and a small current, or the reverse, because power is voltage times current. Take the small current. Ten times the voltage means a tenth of the current and a hundredth of the heat. For three-phase lines the current is P ÷ (√3 × V), and three conductors lose 3 I² R: I = P ÷ (√3 × V), loss = 3 I² Rdouble the line voltage and the loss falls to a quarter The catch is that generators make electricity at about ten to twenty kilovolts, and a socket must deliver a few hundred volts. Direct current of the 1880s could not be changed from one voltage to another. Alternating current could, with a device that has no moving parts: the transformer. Two coils share an iron core; the changing current in one makes a changing magnetic field, which induces a voltage in the other, in proportion to the number of turns. Step the voltage up 20 times and the current goes down 20 times, and power in equals power out less a percent or two. That was the whole reason the world's grids are alternating current, and why transformers hum. In the slider below, 100 MW is carried for 100 km through a line with 5 Ω per phase (an illustrative figure, 0.05 Ω per km). At 132 kV that is 437 A and 2.9 MW lost, about 3%. At 33 kV the same line would lose almost half the power. At 11 kV it could not carry it at all. Real transmission lines run at 110 to 765 kV or higher. Voltage differs around the world for a related reason of economics, not physics. Most countries distribute at about 220 to 240 V; North America and several others use about 120 V (with 240 V for big loads). At 120 V a given appliance draws almost twice the current, so it needs thicker wire and loses more. Both choices were made a century ago and are now baked into a billion appliances. Interactive · Why the grid uses high voltage 11 kV33 kV132 kV400 kV Power delivered Line length Unity power factor, 0.05 Ω per km per phase (illustrative; real values depend on the conductor). The highlighted bar is the voltage chosen above. Three-phase power A single alternating voltage has an awkward habit: the power it delivers to a resistor pulses, falling to zero twice every cycle. Mechanically, that is a shaft that gets pushed hard, then not at all. Now use three voltages of the same size, each delayed by a third of a cycle (120°) from the last. When one phase is passing through zero, the others are pushing. Add the three powers and, remarkably, the total is constant. va + vb + vc = 0, ptotal = constantfor equal loads on all three phases The first part is a trigonometry fact: three equally spaced sines cancel. It means that a balanced load needs no return wire at all, so a three-phase line needs three conductors rather than the six that three separate circuits would. The second part means a smooth, steady push on the shaft of a motor and a steady pull on a generator. It also creates a rotating magnetic field from three fixed coils, which is the next section. The voltage between two phases is √3 times the phase voltage: 230 V per phase is 400 V between phases in Europe, which is why a house can have both. Three-phase transmission arrived in the 1890s; a line in 1891 carried power 175 km from Lauffen to Frankfurt in Germany. Interactive · Three phases against one Single-phaseThree-phase Moment in the cycle Size of the third phase (imbalance) Equal resistive loads, 50 Hz. Pull the third phase below 100% and a neutral current appears, the price of an unbalanced load. Generators and motors: one machine, two directions Both rest on two laws from Physics II (34/07). Move a conductor through a magnetic field and a voltage appears (Faraday); put current through a conductor in a field and a force appears (Lorentz). A generator uses the first: a turbine spins a magnet, called the rotor, inside coils, and three-phase voltage comes out. A motor uses the second. It is the same device run the other way, and engineers treat it that way. In a power station the rotor is a magnet driven by steam, gas, water or wind, spinning at a speed that fixes the frequency: f = (poles × rpm) ÷ 120. A two-pole machine at 3,000 rpm gives 50 Hz; at 3,600 rpm, 60 Hz. Every large generator on one interconnected grid turns in lock-step. The induction motor is the workhorse of industry. Its stator, the fixed part, has three coils fed by three-phase current, and because of the 120° delays they make a magnetic field that rotates at the synchronous speed ns = 120 × f ÷ poles. A 4-pole motor on 50 Hz has a field turning at 1,500 rpm. The rotor is just a cage of conductors with no connection to anything. The rotating field cuts the rotor bars and induces currents in them, and those currents feel a force. But if the rotor ever caught up with the field there would be nothing cutting its bars and no force. So it runs a few percent behind, and the difference is the slip. A more heavily loaded motor slips more, up to a peak torque beyond which it stalls. Permanent-magnet motors, in electric cars and drones, replace the cage with magnets and need an electronic controller to switch the stator currents at exactly the right moment. They are smaller and more efficient. Their key physics is back-EMF: a spinning rotor is also a generator, producing a voltage E = k × ω that opposes the supply. Motor current is the difference between the supply and this back-voltage, divided by the winding resistance. So a motor draws a huge current when stalled (no back-voltage), a small one at speed, and, if driven faster than its no-load speed, a negative one: it becomes a generator and feeds energy back. That is regenerative braking. Interactive · The induction motor that lags its field 50 Hz grid60 Hz grid Number of poles (2 per step) Load torque (percent of the motor's peak) Torque curve from the simple approximate (Kloss) form with the peak at 20% slip. Real motors differ in detail; their full-load slip is typically a few percent. Interactive · A permanent-magnet motor, and what braking does Supply voltage Load torque (negative = braking) Winding resistance 0.2 Ω, motor constant 0.1 V·s per radian (0.1 N·m per amp). Friction, iron losses and the controller are ignored, so real efficiency is lower. Power factor: current that does no work With a resistive load, voltage and current rise and fall together. With a motor or a transformer, which are mostly coils, the current lags the voltage (the phasor picture of Circuit Analysis, 34/34). The product V × I is then bigger than the useful power by a factor called the power factor. P = V I × cos φ, S = V I, S² = P² + Q²real power P in watts, reactive power Q in vars, apparent power S in volt-amperes. Power factor = P ÷ S = cos φ Picture a horse pulling a barge from the bank at an angle. Only the component along the canal moves the boat; the sideways component just stretches the rope. P is the useful part, Q is the sideways part, and the wires must carry S. A 10 kW load at power factor 0.7 draws 62 A at 230 V, where the same 10 kW at power factor 1 would draw 43.5 A. The extra current heats the supply wires by (62.1 ÷ 43.5)² = 2.04 times as much, and it uses up the capacity of cables and transformers. The remedy is elegant. A capacitor's current leads the voltage, the opposite of a coil's, so a bank of capacitors placed beside an inductive load supplies the reactive current locally and lets the grid carry only the real part. Factories do this, and utilities in many places charge large customers a penalty for a poor power factor. Interactive · Cancel the sideways current Real power of the load Power factor of the load (lagging) Capacitors added (share of reactive power cancelled) Single-phase 230 V for simplicity. The grey dashed triangle is the original; the red line is what the wires carry after correction. Frequency: the grid's heartbeat Because every large generator is locked to the same frequency, that frequency is a live gauge of the balance between supply and demand. If demand exceeds supply even slightly, the extra energy is taken from the spinning machines and they slow down. Everywhere, together. If supply exceeds demand they speed up. How fast the frequency moves depends on the inertia stored in the spinning rotors, measured by an inertia constant H, in seconds: the time the rotating energy could cover the whole load by itself. A system with H = 5 s that suddenly loses 5% of its generation starts falling at about 0.25 Hz per second. df/dt = −ΔP × f0 ÷ (2 H)the initial rate of change of frequency: the bigger the missing power, the faster; the more spinning mass, the slower The grid answers in layers. Within seconds, generators with governors open their valves in proportion to the frequency error (the P term of Control Systems, 34/20) and reserve plant takes over minutes later. If the frequency keeps falling, automatic load shedding disconnects whole districts on purpose to save the rest. In continental Europe the conventional scheme starts at 49.0 Hz; in normal operation the frequency is held in a very narrow band around 50 Hz. Modern grids are changing the physics at this point. A solar panel is a DC source and a wind turbine's output is variable-frequency AC. Both connect through an inverter, a bank of transistor switches (see Electronics, 34/35) that chops the DC into pulses and shapes them into a sine wave locked to the grid. Standard inverters follow the grid; they have no rotor and so contribute no inertia. With more of them and fewer spinning machines, the frequency moves faster after a fault, which is why system operators have started to call for fast-acting batteries and for grid-forming inverters that behave like a stiff voltage source. The point is not that renewables are a problem: it is that the old inertia came free with the old generators, and the new system has to supply it on purpose. Worldwide, grids settled on two frequencies. By the 1890s Westinghouse in the United States had chosen 60 Hz, and the German company AEG, after a 40 Hz demonstration line that made lamps flicker, chose 50 Hz; Europe followed AEG and the Americas followed Westinghouse. Japan is split by an accident of history: Tokyo bought German 50 Hz generators and Osaka American 60 Hz ones, so the two halves of the country are joined only through frequency-converting stations. Utility frequency is 50 Hz across much of Europe, Africa and Asia, and 60 Hz in North America, parts of South America and a few Asian countries (South Korea and Saudi Arabia among them). Interactive · Lose a power station Governors respondNo response Generation lost (percent of demand) System inertia constant H Share of generation that is inverter-based A one-machine model: 2H dΔ/dt = ΔPm − ΔPloss − DΔ (D = 1), governor droop 5% with a 3 s lag and 8% reserve. Inverter-based generation is assumed to add no inertia, so Heff = H × (1 − share). Illustrative, not a forecast. Interactive · Your wall socket, anywhere Choose a country Nominal single-phase socket voltage and grid frequency, simplified (several countries use more than one; Brazil also uses 220 V; Colombia and others mix). Always check the label on a device before plugging it in abroad. Blackouts: when the balance breaks A blackout is rarely one failure. It is a loop that feeds itself: a fault removes a line or a generator, the rest take more load or more voltage stress, a protective relay on the next one trips because it must, and so on. Engineers call this a cascade. The art of the grid is to keep a margin, so that the loss of any one element, the N−1 rule, is survivable, and to see the state of the whole system as it changes. Two events show the pattern. Both are summarised below from the official investigations, step by step. Interactive · Two blackouts, step by step 2003 Northeast2025 Iberia Next stepStart again 2003 summary: the U.S.-Canada Power System Outage Task Force final report (April 2004), with the 508 units and 265 plants from its official analysis. 2025 summary: ENTSO-E's expert panel (final report 20 March 2026) and its factual report of 3 October 2025; times are CEST and some sources differ by a second or a minute. Notice what the two share. In 2003 the operators lacked a picture of the system; in 2025 the system had too little voltage control for the conditions of the day, and no one saw the risk approach. Neither was caused by weather alone or by one wrong decision. Both ended in the same state: an island of generators that had lost their grip on each other, which protection then shut down to avoid damage. Restarting a dead grid, a black start, is its own craft: you need a few plants that can start without help from the grid, and you rebuild outward carefully, matching load to supply at every step. What a graduate just sees Power is voltage times current and heat is current squared, so long-distance power goes at high voltage and low current, with transformers at both ends. Three phases cancel their return currents and make constant power: three wires, a smooth shaft, a rotating field. An induction motor lags its field by a few percent, and the lag is the force; a permanent-magnet motor's back-voltage is what lets it brake and recharge a battery. Power factor is the share of the current that does work; capacitors cancel the sideways part and cut wire losses. Frequency is the grid's heartbeat: supply must match demand every second, and the rotating mass buys a few seconds of grace. Inverters connect renewables but carry no inertia, so a grid with a lot of them has to supply stability on purpose. Blackouts are cascades; the defence is margin (N−1), visibility, and the discipline of automatic protection. 50 vs 60 Hz and 230 vs 120 V are historical accidents that stuck, and they decide which appliance works where. Where it shows up next Systems Engineering & the Capstone (34/22): an electric vehicle or a wind farm combines motor, inverter, battery and controller, which only work when all agree. Control Systems (34/20): governors, voltage regulators and motor drives are feedback loops; the grid's frequency response is a control problem with a lag, a gain and a stability limit. Electronics (34/35): every inverter and charger is a transistor bridge; efficiency and heat decide their size. Embedded Systems (34/41): the controller that fires an EV motor's switches is a microcontroller with a hard deadline. The real world: Batteries (29/08) and what they feed; the heat-engine limits of Thermodynamics (34/14) set how much of a power station's fuel becomes electricity; and every country's choice between nuclear, gas, hydro, wind and solar is, at bottom, a choice about this balance. If you want to go deeper 📘 Textbook (systems): J. Duncan Glover, Mulukutla Sarma and Thomas Overbye, Power System Analysis and Design (Cengage): the usual university text, with simulation software. 📘 Textbook (machines): Stephen Chapman, Electric Machinery Fundamentals (McGraw-Hill): clear on transformers, induction and synchronous machines. 📘 Gentler: David MacKay, Sustainable Energy: Without the Hot Air (free online at withouthotair.com), on how much energy everything takes; and Gretchen Bakke, The Grid (Bloomsbury, 2016), a readable history of the American grid and its troubles. ▶️ Free course: MIT OpenCourseWare 6.061 Introduction to Electric Power Systems (Spring 2011). 🌐 Easier first reads in this library: Batteries (29/08) and Physics II: Electricity & Magnetism (34/07). The takeaway Electric power cannot be stored in the wires, so a grid is a balance held every second: generation equals demand, and the frequency is the gauge. Transformers let power travel at high voltage where losses are small; three phases make constant power and a rotating field; induction and permanent-magnet motors are generators run backwards; power factor measures the current that does no work. As inverter-based generation grows, inertia and voltage control must be provided deliberately, because blackouts in 2003 and 2025 were cascades in which visibility and margin ran out. Try this: find the label on any plug-in appliance (kettle, charger, hairdryer). It says the voltage and frequency it was designed for, and often the power. Work out the current (power divided by voltage) and ask which of your wires carries it. If you travel, notice which country has the 230 V sockets and which the 120 V. Sources & Further Reading Sources. Transformers, three-phase power, induction and permanent-magnet motors, power factor and the swing equation are standard textbook material (Glover, Sarma and Overbye; Chapman; MIT 6.061); the Kloss torque-slip form and the typical slip of a few percent are textbook approximations (a vendor page gives 2 to 5% for medium motors). British TV pickup: about 2,800 MW after the 4 July 1990 England v West Germany penalty shoot-out, as listed on Wikipedia's "TV pickup" and by Drax (secondary sources; National Grid's own data not found). History of 50 and 60 Hz, the 1891 Lauffen to Frankfurt line (175 km, 40 Hz) and Japan's split: Wikipedia, "Utility frequency" (the article itself notes the early details are uncertain). Mains voltage and frequency by country: Wikipedia, "Mains electricity by country", checked against the IEC list in general; Brazil, Mexico and others use more than one voltage. ENTSO-E: nominal 50 Hz, conventional underfrequency load shedding starting at 49.0 Hz (ENTSO-E underfrequency report 2024), a normal-operation dead band of 10 mHz. 2003 blackout: final report of the U.S.-Canada Power System Outage Task Force (April 2004, via the U.S. Department of Energy and NERC), about 50 million people (Canadian government release; Wikipedia gives about 55 million, a figure that counts differently), 508 units at 265 plants (official analysis), cascade from about 4:10 to 4:13 pm EDT, the control-room software fault, overgrown trees. 2025 blackout: ENTSO-E expert panel final report of 20 March 2026 (reported by Euronews, Renewable Energy World and others), preliminary factual report 3 October 2025; timestamps from Wikipedia, which cites the ENTSO-E data and warns that sources conflict on restoration times. The ENTSO-E panel's count of recommendations differs between press reports (17 or 21), so none is given. Computed in node: every reading in the interactives, including the 437 A and 2.9 MW at 132 kV, the 1,500 rpm field, the 1,440 rpm loaded 4-pole motor, 62.1 A against 43.5 A, and the initial −0.25 Hz per second. 🌐 MIT OpenCourseWare 6.061 Introduction to Electric Power Systems 🌐 David MacKay, Sustainable Energy: Without the Hot Air 📘 Glover, Sarma and Overbye, Power System Analysis and Design 📘 Chapman, Electric Machinery Fundamentals Step 18. Communication Systems: How Much Can a Noisy Channel Carry? ------------------------------------------------------------------- Why: How information rides on a wave, and how much a channel can carry. Why a phone can send a movie through the air at the edge of what physics allows: a message riding on a wave, noise that turns bits into guesses, a 1948 theorem that names the limit, and codes that quietly repair the damage. Stand at the edge of your Wi-Fi's range and watch what happens. The video does not slowly go to pieces, it first drops in resolution, then stutters, then stops. Your phone's signal bars fall and the "5G" label flips to "4G". Behind them is a continuous negotiation: the radio measures how noisy the channel is right now and picks the fastest way of sending bits that will still arrive intact. There is a limit to that negotiation, and it is not a limit of technology. In 1948 Claude Shannon proved that every noisy channel has a maximum rate, set by just two things: how wide it is and how loud the signal is compared with the noise. Below that rate, errors can be made as rare as you like. Above it, no cleverness helps. This subject is the story of getting close to that line. The course in one box Who takes it: electrical and computer engineers, usually in year 4 (sometimes year 3 as an elective). What it's for: designing the layer that turns bits into waves and back, across a channel that adds noise, echoes and other people's signals. Comes after: Signals & Systems (34/37) for spectra and filters, and Probability & Statistics (34/05) for noise. Electromagnetics & Waves (34/38) supplies the antenna and the wave. Leads to: nothing in this list requires it. It is where signals, noise and waves meet, and it is the physical layer under Computer Networks (34/27), which tells the story of the layers above. The radios in Embedded Systems (34/41) use it. Honest line: you won't be able to design a modem after this. You will see why more bits per second costs more power, why noise turns "wrong" into a probability, what a parity bit really does, and what 4G, 5G and Wi-Fi do under the hood. A message rides on a carrier A voice is made of frequencies from a few hundred hertz to a few kilohertz. You could, in principle, send it as is: a voltage on a wire. To send it through the air you meet two problems. An antenna must be a decent fraction of a wavelength long to radiate (a quarter of the wavelength of a 3 kHz tone is 25 km), and everyone sending at the same low frequencies would all be on top of each other. The fix is the oldest trick in radio: shift the message up. Generate a steady high-frequency wave, the carrier, and modulate it, which means changing one of its properties in step with the message. AM (amplitude modulation) changes the size of the carrier. The message shows up as the wave's outline, the envelope. It is simple to receive (a diode and a capacitor will do), but any noise that changes the size of the wave is heard. AM takes a bandwidth of twice the highest message frequency. FM (frequency modulation) changes how fast the carrier oscillates and leaves its size constant. Noise mostly changes size, so FM shrugs it off, at the price of more bandwidth: Carson's rule gives about 2 × (swing + highest message frequency). Edwin Armstrong's wide-band FM was granted five US patents on 26 December 1933. Worked example: broadcast FM swings its carrier ± 75 kHz for 15 kHz audio, so the bandwidth is about 2 × (75 + 15) = 180 kHz, compared with 2 × 5 = 10 kHz for an AM station. That is FM's trade: bandwidth spent to buy noise immunity, a small version of what Shannon proved in general. In the Americas AM stations are spaced 10 kHz apart (9 kHz elsewhere), and FM stations in the US 200 kHz apart. Interactive · AM and FM AMFM Modulation strength Message frequency Drawn with 20 carrier cycles per message cycle so you can see the shape; a real carrier has thousands. For AM the strength slider sets the depth (0 to 1.5); for FM it sets the index β (0 to 8). Digital modulation: constellations with noise To send bits instead of a voice, you modulate the same properties with a short list of allowed values. Switching the carrier on and off is ASK; flipping its phase by 180° is BPSK; using four phases gives QPSK, two bits per symbol. Putting in both phase and size gives QAM: 16-QAM carries 4 bits per symbol, 64-QAM 6, 256-QAM 8. A convenient picture is the constellation: each allowed symbol is a point on a plane, whose horizontal coordinate is the in-phase part of the carrier and the vertical coordinate the quarter-cycle-shifted part. The transmitter sends a point. The receiver sees that point plus noise, and decides which allowed point is nearest. The noise is the thermal jiggle of electrons in every receiver, spread evenly over all frequencies and bell-curve shaped (Gaussian). The noise power in a channel of bandwidth B is kTB: at room temperature this is about −174 dBm per hertz, so a 20 MHz Wi-Fi channel has a floor of about −101 dBm before anything else is added. The chance of a wrong decision is a tail probability from Probability & Statistics (34/05): the noise must push the point more than half the gap to the neighbour. P(error) ≈ Q( half the gap ÷ noise spread )Q(x) = the area under the bell curve beyond x; it falls extremely fast as x grows That fall is why digital wins. Below a threshold of signal to noise almost every symbol is wrong; a few dB above it, almost none. And because the receiver snaps to the nearest allowed point, noise does not accumulate along a chain of relays the way it does in an analogue copy of a copy. The price is spacing: more points in the same power shrink the gaps. For the same chance of error, each four-fold increase in points (2 more bits per symbol) needs about 6 dB more signal: going from QPSK to 16-QAM takes 7.0 dB, 16 to 64 takes 6.2 dB, 64 to 256 takes 6.1 dB. Interactive · Squeeze more bits, hear more noise ASKBPSKQPSK16-QAM64-QAM256-QAM Signal-to-noise ratio Send 600 new symbols Rings: the ideal points (average power 1). Dots: 600 received symbols, blue if decoded correctly, red if the nearest ideal point is the wrong one. Theory: the standard error formulas, checked against a long simulation. SNR here means energy per symbol over noise density. Bandwidth, noise and Shannon's limit Before 1948 engineers believed that noise forced a trade-off, and that zero errors needed a rate of zero. Shannon's paper "A Mathematical Theory of Communication" (Bell System Technical Journal, July and October 1948) said that this was wrong. For any channel there is a number, the capacity, such that below it you can make the error probability as small as you want (with clever enough coding), and above it you cannot. For the standard channel with Gaussian noise the number has a famous form, sometimes called the Shannon–Hartley theorem (after Ralph Hartley's 1928 work on counting signal levels): C = B · log₂(1 + S/N)capacity in bits per second = bandwidth in hertz × log₂(1 + signal power ÷ noise power) Read it as two dials. Bandwidth multiplies the rate: double B, double C. Signal-to-noise sits inside a logarithm: at high S/N, doubling the signal power adds only one bit per second per hertz, about 3 dB. So more bandwidth is a straight-line gain and more power a very slow one, which is why every wireless generation hunts for new spectrum. Worked numbers: a telephone line (3.1 kHz, 35 dB) is limited to about 36 kbit/s, and that is where dial-up modems topped out. A 20 MHz Wi-Fi channel at 25 dB allows 166 Mbit/s; a 100 MHz 5G carrier at 20 dB, 666 Mbit/s (per stream; extra antennas multiply it). Shannon proved the limit exists but not how to reach it. It took until the 1990s to get close. Plain QAM with no error correction sits some 8 dB above the limit if you demand one symbol error in 100,000, and 6 dB at one in a thousand; the orange squares in the chart show it. Coding (the next section) closes most of that gap. There is also a lower bound that no amount of bandwidth removes: the energy per bit must exceed about −1.59 dB of the noise density (ln 2 as a ratio), or nothing can be sent reliably. Interactive · The speed limit of a channel Phone lineWi-Fi 20 MHz5G 100 MHzWeak signal Bandwidth Signal-to-noise Blue curve: log₂(1 + S/N), the capacity per hertz. Orange squares, left to right: BPSK (1 bit), QPSK (2), 16-QAM (4), 64-QAM (6), 256-QAM (8), each placed at the SNR where theory gives 1 symbol error per 1,000, without any error-correcting code. Bit errors and error-correcting codes At the edge of a channel a few bits come out wrong, and a flipped bit in a file is a corrupt file. The fix is to add redundancy cleverly. The crudest scheme is repetition: send every bit three times and vote. It works, but it triples the cost. The elegant idea is to add a few check bits, each of which is a parity (even or odd count of ones) of a chosen subset of the data bits. Richard Hamming, frustrated at Bell Labs in the late 1940s by a relay computer that stopped whenever it hit one bad bit, published a code in 1950 that does this for four data bits. The Hamming (7,4) code adds three check bits so that the three parities, read as a binary number (the syndrome), spell out the position of a single flipped bit: zero means all is well, five means the fifth bit is wrong. The receiver flips it back. Two errors defeat it. That is the trade-off of all codes: more redundancy corrects more errors, at the cost of rate. Later codes work on this idea at scale. Reed–Solomon codes (1960) correct bursts of errors and sit in CDs (together with interleaving, which can repair a scratch of a few millimetres) and in QR codes. Turbo codes (Berrou, Glavieux and Thitimajshima, 1993) came within about 0.7 dB of Shannon's limit. LDPC codes, which Robert Gallager had invented in the early 1960s and which were forgotten for three decades, did the same, and 5G uses them for user data and polar codes (Erdal Arıkan, 2009) for control signalling. Interactive · Hamming (7,4): the one-bit repair 1. Pick four data bits (tap to flip). 2. Tap any received bit to add noise. Watch the checks point at the damage. Remove all noise Chance that any one bit flips Check bits p1, p2 and p4 sit at positions 1, 2 and 4; each covers the positions whose binary number contains that bit. The table uses exact binomial probabilities for independent flips. Sharing the air: FDMA, TDMA, CDMA and OFDM Many users must share the same air, and there are only a few ways to keep their signals apart. Each one slices a different dimension. FDMA (frequency): each user gets its own slice of frequency. Radio and TV stations; each GSM phone carrier is a 200 kHz slice. TDMA (time): users take turns on the same frequency, in short slots. GSM puts eight users on each 200 kHz carrier. CDMA (code): everyone transmits at once on the same frequency, each multiplying its bits by its own pattern of +1 and −1 "chips". A receiver multiplies by the same pattern, and the others, whose patterns are orthogonal to it, average away. 3G phones and GPS satellites (all on one frequency) work this way. OFDM (many slow carriers): split the channel into hundreds or thousands of narrow subcarriers, each carrying a slow stream, spaced so their spectra don't interfere. It is computed with an FFT (Signals & Systems, 34/37). Slow symbols mean echoes, whose delays Electromagnetics & Waves (34/38) explains, cause little harm, particularly when a short copy of each symbol's end is added as a guard (the cyclic prefix). In LTE the subcarrier spacing is 15 kHz, so a symbol lasts 66.7 µs and the normal guard is 4.7 µs. OFDM is a way of sending; OFDMA is a way of sharing: different users get different groups of subcarriers at the same moment, assigned by the base station every millisecond or so. Interactive · Two senders, one channel (CDMA) User A sends User B sends Noise New noise Each bit is spread over 8 chips with a Walsh code (A uses +−+−+−+−, B uses ++−−++−−), and the two are simply added. Each receiver correlates with its own code. Real systems also need synchronisation and power control, because codes only cancel perfectly when they are aligned. What 4G, 5G and Wi-Fi actually do Now every part has a place. A modern radio does four things, in a loop: it measures the signal-to-noise ratio of the channel; it picks the densest modulation and the lightest code that will still give an acceptable error rate (this is adaptive modulation and coding, and the reason your link speed changes as you walk away); it spreads users over time and frequency; and it adds an error-correcting code with retransmissions as the last resort. The generations differ mainly in how wide a channel they use, how dense a constellation they dare, which codes they choose, and how finely they share. SystemSharingDensest modulationCodingChannel width GSM (2G)FDMA + TDMAGMSK (low rate)convolutional200 kHz 3G (UMTS)CDMAQPSK, later 16- and 64-QAMconvolutional, turbo5 MHz 4G LTEOFDMA down, SC-FDMA upto 256-QAM in later releasesturbo1.4 to 20 MHz 5G NROFDMA, flexible spacing256-QAMLDPC (data), polar (control)up to 100 MHz (below 7 GHz), 400 MHz (mmWave) Wi-Fi 6OFDMA, listen-before-talk1024-QAMLDPC20 to 160 MHz Wi-Fi 7OFDMA4096-QAMLDPCup to 320 MHz (6 GHz) Each step adds bandwidth (the straight-line gain), adds modulation density (a slow logarithmic gain, available only close to the tower or router, where S/N is high), and swaps in a code nearer to Shannon's limit. 5G adds two things outside the table: very high bands (24 to 71 GHz) where there is plenty of spectrum, and arrays of many small antennas that steer a narrow beam to each user, which raises S/N exactly where it is needed. Those millimetre waves die quickly in walls and rain, which is why they come with small cells and short range: the wave rules in Electromagnetics & Waves (34/38) set the design. What a graduate just sees A radio is a message shifted up to a carrier; the bandwidth you need is set by the message and the modulation, not by the carrier. Noise is not an obstacle but a probability: error rates fall off a cliff as SNR rises, so there is always a threshold. Every extra bit per symbol costs about 3 dB, each four-fold increase in points about 6 dB. Capacity is B·log₂(1 + S/N): bandwidth is worth more than power, and neither can be bought past the limit. Redundancy, arranged cleverly, can repair errors without asking for a resend; a check-bit pattern can name the guilty bit. Sharing is slicing: by frequency, time, code or many slow subcarriers. Where it shows up next Computer Networks (34/27): the physical layer is this subject; the layers above treat the link as a pipe with some loss and delay, and that "loss" is the leftover error after coding. Embedded Systems (34/41): every IoT device chooses a radio and a duty cycle; battery life is a trade between rate and range, which is Shannon's dials. Computer Security (34/33): jamming, eavesdropping and spoofing a radio link are attacks on this layer. Machine Learning (34/32): entropy, cross-entropy and compression come from the same 1948 paper. The real world: phones, satellites, GPS, cable and fibre, QR codes, hard drives and SSDs. If you want to go deeper 📘 Textbook: Simon Haykin and Michael Moher, Communication Systems (Wiley): the long-standing standard, with analogue modulation as well as digital. 📘 Textbook: John G. Proakis and Masoud Salehi, Fundamentals of Communication Systems (Pearson), and Andrea Goldsmith, Wireless Communications (Cambridge University Press) for the wireless channel, OFDM and capacity. 📘 Gentler: Jimmy Soni and Rob Goodman, A Mind at Play (2017), a biography of Shannon; James Gleick, The Information (2011). ▶️ Gentler: Khan Academy, "Journey into information theory" (free). ▶️ Free course: MIT OpenCourseWare 6.02 Introduction to EECS II: Digital Communication Systems (Fall 2012). 🌐 Easier first reads in this library: Computer Networks (34/27), The Internet (29/04) and The Fourier Transform (12/12). The takeaway A radio link moves a message by modulating a carrier, and digital modulation puts symbols on a constellation that noise smears. Errors are tail probabilities, and denser constellations need more signal. Shannon's 1948 limit C = B·log₂(1 + S/N) says how much a channel can carry, and error-correcting codes (Hamming to LDPC) are how real links approach it. 4G, 5G and Wi-Fi are the same recipe in different proportions: wider channels, denser modulation, better codes, finer sharing. Try this: in your phone's Wi-Fi or mobile settings, look at the signal strength and the link speed while walking away from the router. Watch the speed drop in steps. Each step is the radio dropping to a sparser constellation as the SNR falls. Sources & Further Reading Sources. Modulation, error formulas and OFDM are standard textbook material (Haykin; Proakis; Goldsmith). C. E. Shannon, "A Mathematical Theory of Communication", Bell System Technical Journal 27 (July and October 1948); Hartley, BSTJ 7 (1928) 535–563; Nyquist, Trans. AIEE 47 (1928) 617–644 (see Wikipedia, "Shannon–Hartley theorem"). Armstrong's five patents of 26 December 1933 (Wikipedia, "Edwin Howard Armstrong"). Hamming (7,4): R. W. Hamming, 1950 (Wikipedia, "Hamming code"). Reed and Solomon, 1960, CD and QR use (Wikipedia, "Reed–Solomon error correction"). Turbo codes: Berrou, Glavieux and Thitimajshima, 1993, within about 0.7 dB of the limit at a bit error rate of 10⁻⁵ for rate 1/2 (later confirmed by Divsalar and Pollara, JPL, 1995). LDPC from Gallager (early 1960s) and polar codes (Arıkan, 2009); 5G uses LDPC for data and polar for control (3GPP TS 38.212). 5G NR bands: FR1 410 to 7125 MHz, FR2 24.25 to 71 GHz (3GPP; Wikipedia, "5G NR frequency bands"); carriers up to 100 and 400 MHz. LTE: OFDMA downlink and SC-FDMA uplink, 1.4 to 20 MHz channels (Wikipedia, "LTE"); 15 kHz spacing; 256-QAM in later releases. Wi-Fi 6 (802.11ax): OFDMA, 1024-QAM; Wi-Fi 7 (802.11be): 4096-QAM, 320 MHz in 6 GHz. AM and FM band plans: ITU, Wikipedia, "Medium wave" and "FM broadcast band". Thermal noise floor −174 dBm/Hz at 290 K (kT). Computed in node: all interactive numbers, the 7.0, 6.2 and 6.1 dB steps, the 6 and 8 dB gaps, and the Hamming and repetition error rates; the simulation matched the exact formulas on 200,000 symbols. 🌐 MIT OpenCourseWare 6.02 📘 Haykin and Moher, Communication Systems · Goldsmith, Wireless Communications · Soni and Goodman, A Mind at Play Step 19. Embedded Systems: Tiny Computers With Deadlines -------------------------------------------------------- Why: Small computers inside every device, running on milliwatts and deadlines. The computers that run the world are mostly invisible: a few kilobytes of memory, a few milliamps of power, one job, and a deadline. Embedded engineering is the craft of making them read the real world, act on it in time, and last for years on a coin. Count the computers in a car. Not the screen: all of them. One reads the throttle pedal, one meters fuel into the engine, one decides when an airbag fires, one runs a door lock. Counts depend on what you call a computer. One industry whitepaper (TE Connectivity) puts a modern car at 70 to 100 electronic control units, each with its own processor, and one analyst quoted on Wikipedia says up to 150 in a luxury model. None of these is a general-purpose computer. Each does one job, forever, with a deadline: an airbag controller that thinks for a moment too long has failed. A washing machine, a thermostat or a remote hides the same thing: a small chip reading a button or sensor and driving a motor, a light or a radio. The course in one box Who takes it: electrical and computer engineering students, usually in the third or fourth year, often as a lab course with a development board. What it's for: building a small computer that senses, decides and acts within a fixed time and power budget. Comes after: Digital Logic Design (34/36), Computer Architecture (34/25) and Programming Fundamentals (34/09). Leads to: Systems Engineering & the Capstone (34/22), where it is one subsystem among many, and it sits next to Control Systems (34/20), whose feedback loops usually run on exactly these chips. Honest line: you won't be able to bring up a new board after this piece. You will see why a button must be filtered, what a sensor reading really resolves, how a motor is dimmed, and how long a coin cell lasts. Easier first reads: Digital Logic Design (34/36) for the counters and state machines inside every peripheral, Processors: How a CPU Thinks (29/11), and Batteries (29/08). A computer on one chip, with pins that touch the world A microcontroller (MCU) puts the processor, the program memory (flash), the working memory (RAM) and the peripherals (timers, converters, serial ports) on one chip. Power it and it runs; no operating system, no external memory. The chip on the classic Arduino Uno, the ATmega328P, has a 16 MHz clock, 32 KB of flash, 2 KB of RAM and 1 KB of EEPROM. A phone's processor runs billions of ticks a second with gigabytes of memory beside it. The gap buys a chip that is cheap, sips power, and has nothing between your code and the pin. MicrocontrollerApplication processor ExampleATmega328P (Arduino Uno)Phone or laptop chip Clock16 MHzBillions of ticks per second Memory2 KB RAM, 32 KB flash on the chipGigabytes, off the chip SoftwareYour program, alone ("bare metal")An operating system and apps Built inTimers, ADC, UART, I2C, SPIMostly outside the chip PowerMilliwatts, microwatts asleepWatts The door to the world is the GPIO pin (general-purpose input/output): a pin tied to one bit in a register. Write a 1 and the pin goes high, perhaps lighting an LED; read it and you learn whether it is high or low. To read a button, wire it between the pin and ground and turn on a small pull-up resistor inside the chip: the pin idles at 1, and a press pulls it to 0 ("active low"). Then the real world bites. The metal contacts of a button spring against each other, so one press makes the line flicker between 0 and 1 several times before it settles. Jack Ganssle's well-known test of many switches, as summarised by EE Journal, found an average bounce of about 1.6 ms and a longest of about 6.2 ms among the ordinary ones, with one switch bouncing for 157 ms. To a chip that reads a pin every microsecond, a single press looks like a burst of presses. The fix is debouncing: accept a change only after the pin has held its new value for longer than the worst bounce. Interactive · One press, many edges Worst-case bounce of this switch Debounce window: line must hold this long Press the button again Red = the raw pin, blue = the debounced signal (checked every 1 ms). First 40 ms drawn; the button is then held. Each press uses a new random bounce pattern. With a 6 ms bounce and a 10 ms window, every press counts exactly once, while the raw pin shows three to five falling edges. Now drag the window down to 1 or 2 ms: the filter is shorter than the gaps between bounces, so it accepts some of them as real presses, and the count is often wrong again. The rule: the window must be longer than the longest bounce gap and shorter than anything a human can do (tens of milliseconds is common). ADC and DAC: turning voltage into numbers and back The world is analog and the chip is digital, so an analog-to-digital converter (ADC) measures a voltage and reports a whole number. An N-bit ADC slices the range from 0 V up to its reference voltage into 2N equal steps, and reports which step the input falls in. The rule graduates remember is the size of one step, the "least significant bit": 1 LSB = Vref ÷ 2N   ·   error ≤ ½ LSB   ·   ideal SNR ≈ 6.02N + 1.76 dB Worked: a 10-bit ADC on a 3.3 V reference has 1,024 steps of 3.22 mV; a 12-bit one has 4,096 steps of 0.806 mV. A sensor that outputs 10 mV per degree is then resolved to about 0.32 °C by the 10-bit converter, and no software can recover detail below that. Each extra bit halves the step and adds about 6 dB in the ideal case; real converters add noise, so a "12-bit" chip often delivers fewer effective bits. The converter also samples in time: it takes a reading, holds it, and moves on. Sample slower than twice the fastest wiggle and the shape is lost (the sampling theorem, which Signals & Systems, 34/37, develops). A DAC runs the same ladder backwards, turning a number into a voltage. Interactive · Quantise a sine wave Resolution Samples per cycle Dotted = true 0 to 3.3 V sine; orange = what the ADC reports, each sample held until the next. Two cycles shown. At 1 bit the output is a square wave, because the converter only knows "above or below half". At 4 bits you see 16 stairs. At 10 bits the steps vanish into the line. The second slider shows the other cost: with 3 samples per cycle the wave is barely a triangle. Timers and PWM: counting ticks to dim an LED A timer is the counter from Digital Logic Design, fed by the chip's clock divided by a prescaler, which can raise a flag, trigger an interrupt, or flip a pin when it reaches a chosen value. Every wait and every precise pulse comes from a timer; the CPU does not count. Most pins can only be fully on or fully off, so to dim an LED or slow a motor you switch very fast and vary the fraction of each period the pin is on. This is pulse-width modulation (PWM). A motor, or an eye, is too slow to follow the flicker and responds to the average: Vavg = duty × Vcc   ·   fPWM = fclock ÷ (prescaler × 2bits) Worked: a 16 MHz clock, prescaler 64, and an 8-bit timer (256 counts) gives 16,000,000 ÷ (64 × 256) = 976.6 Hz, the "about 980 Hz" Arduino documents for pins 5 and 6, which run on the same timer that keeps millis(). That timer overflows every 1.024 ms. The "about 490 Hz" pins count up and then down, so one cycle takes twice as long. The cost of more resolution is lower frequency: a 16-bit timer at the same settings gives only 3.8 Hz, which would visibly flicker. Interactive · A timer-driven PWM pin Clock Prescaler Timer width Duty (set by the compare value) With the defaults (16 MHz, 64, 8-bit), 25% duty is compare value 64 of 255 and averages 1.25 V from a 5 V pin: an LED at that setting looks dim, though the eye's response is not linear, so it looks brighter than a quarter. Switch to prescaler 1 and 8 bits and the same code runs at 62.5 kHz, too fast for some motor drivers to follow. Interrupts versus polling: ask, or be told A chip must also notice events. Polling means the main loop keeps asking "has the button been pressed?". It is simple, but the answer can only be as fresh as one trip around the loop, a short pulse between two looks is never seen, and the CPU can never sleep. An interrupt turns it round: the hardware stops whatever the code is doing, saves where it was, jumps to a short handler (the ISR), and returns afterwards. On an ARM Cortex-M3 this takes 12 clock cycles with zero-wait-state memory (ARM's documentation); at 48 MHz that is 0.25 µs. Interactive · Polling loop against interrupt One trip around the main loop takes Each event lasts Ten events in one second (green = polling saw it, red = missed). The loop looks at the pin once per trip. The interrupt line assumes a 48 MHz Cortex-M3 and ignores handler length. At 60 ms per loop and 20 ms events, polling misses seven of the ten, and the three it catches arrive 6 to 18 ms late. The interrupt catches all ten in a quarter of a microsecond. Slide the loop to 1 ms and polling looks perfect, but only because it now does nothing else. Serial buses: UART, I2C and SPI Chips talk to sensors and to each other over a few wires, one bit at a time. Three buses cover most of it. UART (the "serial port") uses two wires and no shared clock: each side agrees on a speed (the baud rate) beforehand. The line idles high; a byte is sent as a start bit (0), eight data bits least-significant first, and a stop bit (1): ten bit-times for eight bits of data. I2C uses two shared wires, a clock (SCL) and data (SDA), for many devices at once. Every device has an address; the controller (older texts say "master") sends the address, then data bytes, and the target acknowledges each byte with one extra bit, so a byte costs nine clocks. NXP's specification sets 100 kbit/s for standard mode and 400 kbit/s for fast mode. SPI uses four wires (clock, data out, data in, and one chip-select per device) and has no addresses or acknowledgements, which makes it simple and fast, often megabits per second. Interactive · A UART frame, and how long a transfer takes Character to send Bytes to move (e.g. reading a sensor) I2C speed SPI clock Model: UART 10 bits per byte; I2C 9 clocks per byte including an address byte, ignoring start and stop; SPI 8 clocks per byte, no overhead. Real transfers add gaps and command bytes. The letter "A" is 0x41, which is 01000001 in binary; sent least significant bit first it is 1 0 0 0 0 0 1 0, framed by a start 0 and a stop 1. At 9,600 baud one bit lasts 104 µs and one byte 1.04 ms. For six bytes from an accelerometer, a typical amount, that is 6.25 ms at 9,600 baud against 48 µs for SPI at 1 MHz. Real time: deadlines, and the watchdog behind them "Real-time" does not mean fast; it means guaranteed on time. In a hard real-time system (an airbag, a pacemaker, an engine controller), a late answer is a wrong answer. In a soft one (video playback), a late answer is merely worse. With several jobs sharing one core, a scheduler must decide who runs. A classic rule, proved by Liu and Layland in 1973, is rate-monotonic scheduling: give the highest priority to the task with the shortest period. Their test adds up each task's utilisation, its run time divided by its period: U = Σ Ci ÷ Ti   ·   guaranteed if U ≤ n(21/n − 1) For three tasks the bound is 0.78, and it falls toward 0.69 as tasks are added. The test is sufficient, not necessary: above the line the deadlines may still be met, and exact analysis is needed to know. The simulator below runs three tasks with periods of 10, 20 and 50 ms. Interactive · Three tasks, one core Task 1 (every 10 ms) needs Task 2 (every 20 ms) needs Task 3 (every 50 ms) needs Fixed priorities, shortest period first; a higher task preempts a lower one. Red mark = a job still unfinished when its next one arrives. 100 ms is one full cycle for these periods. Try "Above the bound, still fine": U is 0.79, over the 0.78 line, yet the simulation meets every deadline. Then "Under 100%, yet it fails": U is 0.94, and the slowest task still misses at 50 ms, because the fast tasks keep interrupting it. Utilisation under 1 is necessary, not sufficient. When a deadline is missed or code hangs, the last defence is the watchdog: a hardware timer that resets the chip unless the software "kicks" it before it runs out. A healthy program kicks it as a matter of routine; a hung one cannot, and after one timeout the chip restarts. Mars Pathfinder (landed July 1997) showed both sides. Its computer began resetting itself on Mars. The cause, found by reproducing it in a lab, was priority inversion: a low-priority weather task held a lock the high-priority bus task needed, a medium-priority task kept the weather task from finishing, and the bus task missed its deadline. Glenn Reeves of JPL reports that the high-priority scheduling task noticed the missed deadline and forced the reset; other accounts say a watchdog did. Either way, the fix was a patch sent from Earth that turned on priority inheritance for the lock. The coin-cell budget: living on microamps Many embedded devices must run for years on a battery they will never see again. The discipline is a budget. A chip is usually in one of two states: active, drawing milliamps while it works, and asleep, drawing microamps while a timer waits. The average current is the weighted mix, and the battery's capacity divided by that average is its life: Iavg = (Iactive · tactive + Isleep · tsleep) ÷ T   ·   life = capacity ÷ Iavg Energizer's CR2032 sheet gives a typical 235 mAh, measured down to 2.0 V under a 15 kΩ load; capacity is lower under heavy pulses, in the cold, or after years on a shelf. Worked: a chip that draws 5 mA and never sleeps drains 235 mAh in 47 hours. The same chip waking for 20 ms every 10 s, and sleeping at 2 µA in between, averages about 12 µA: 235 ÷ 0.012 is roughly 19,600 hours, or 2.2 years. A coin cell is also a weak source: some makers' sheets recommend pulses of only 15 to 20 mA, so radio bursts often need a capacitor beside the cell. Interactive · How long does the battery last? Current while active Current while asleep Wakes per hour Awake for, each time The derated cell is an assumed two-thirds of the datasheet figure, for pulses, cold and age. Ignores the cell's own slow self-discharge and voltage droop. With the defaults (5 mA, 2 µA, 360 wakes an hour, 20 ms) the life is the 2.2 years above. Now raise the awake time: at 200 ms the life drops to about three months (96 days), because the active part dominates the average. Lower the sleep current from 2 to 0.1 µA and life only improves from 2.2 to 2.7 years: sleep current matters once the chip sleeps nearly all the time, and not before. The art is to wake rarely, work fast, and go back to sleep. What a graduate just sees A microcontroller is a whole computer on a chip, chosen for cost, power and predictability, not speed. A button is a bouncing wire: any signal from the physical world needs filtering before it is a fact. An N-bit converter gives 2N steps of Vref ÷ 2N; resolution is a budget, and so is sample rate. Switching fast and varying the on-time is how a pin that can only be on or off produces a middle value. Polling asks and interrupts tell; short pulses and low power both favour interrupts. Buses trade wires against speed: UART needs no clock, I2C shares two wires among many, SPI is fast and needs a select line per device. Real-time means a guaranteed deadline, and total utilisation under 100% does not guarantee it. Battery life is set by the average current, which is set by how much of the time the chip is asleep. Where it shows up next Control Systems (34/20): a digital controller is a timer interrupt that reads an ADC, computes, and writes a PWM output; the sample period you choose here sets how stable the loop can be. Systems Engineering & the Capstone (34/22): timing and power budgets are allocated across subsystems exactly as above, and a missed budget is a system-level failure. Operating Systems (34/26): a real-time operating system is a small scheduler plus interrupts, with the priority rules from the real-time section. Signals & Systems (34/37) and Power Systems & Electric Machines (34/39): sampling theory limits the ADC, and PWM drives the inverters that run motors from the grid to the electric car. If you want to go deeper 📘 Textbook: Jonathan Valvano, Embedded Systems: Introduction to Arm Cortex-M Microcontrollers (Valvano, 5th ed.). The University of Texas course text: ports, timers, ADC, interrupts and serial buses on one real chip. 📘 Reference: Joseph Yiu, The Definitive Guide to ARM Cortex-M3 and Cortex-M4 Processors (3rd ed., Newnes, 2013). How the interrupt system and core of the most common microcontroller family work. 📘 Practical and gentler: Elecia White, Making Embedded Systems (2nd ed., O'Reilly, 2024). Design patterns for firmware, written for people who have programmed before. 🌐 Free course and book: Valvano's free online e-book, "Embedded Systems - Shape The World", which accompanies the UT Austin course on edX (check whether the current edX run is open). 🌐 Debouncing: Jack Ganssle, "A Guide to Debouncing", with oscilloscope traces of real switches. The takeaway An embedded system is a small computer whose problems are physical: noisy inputs, a deadline, and an energy budget. Filter what you read, choose your resolution and your timer frequency deliberately, let hardware tell you about events instead of asking, prove your deadlines instead of hoping, and spend your life asleep. Try this: tap a button on a microwave or a remote twice as fast as you can. Notice how fast you can go before the second tap is ignored. That limit is a debounce window or a polling loop at work. Sources & Further Reading Sources. ECU counts: TE Connectivity, Electromagnetic Compatibility in Connected and Electrified Vehicles (whitepaper, 70 to 100); Wikipedia, "Electronic control unit" (up to 150, IHS Markit analyst); figures differ widely by definition. ATmega328P: Microchip/Atmel datasheet (32 KB flash, 2 KB SRAM, 1 KB EEPROM, 10-bit ADC); Arduino Uno clock and EEPROM: Arduino documentation. Switch bounce: Ganssle, "A Guide to Debouncing", as summarised by EE Journal ("Ultimate Guide to Switch Debounce", part 4). Ideal ADC SNR of 6.02N + 1.76 dB: standard result (Analog Devices tutorial MT-001). Arduino PWM frequencies (about 490 Hz and 980 Hz): Arduino analogWrite() reference. Cortex-M3 interrupt latency of 12 cycles with zero-wait-state memory: Arm Community, "Beginner guide on interrupt latency". I2C 100 and 400 kbit/s: NXP UM10204. Rate-monotonic bound: Liu and Layland, Journal of the ACM 20(1), 1973. Mars Pathfinder: Glenn Reeves' account (JPL) and Mike Jones' earlier message; the two differ on whether a watchdog or the scheduling task triggered the reset. CR2032 capacity and test load: Energizer CR2032 datasheet (235 mAh typical, 15 kΩ, to 2.0 V); pulse limits of 15 to 20 mA from other makers' sheets. All interactive numbers were computed in code. 🌐 Valvano: Embedded Systems - Shape The World 🌐 Ganssle: A Guide to Debouncing 📘 Making Embedded Systems, Elecia White (O'Reilly, 2nd ed., 2024) Step 20. Systems Engineering & the Capstone: Many Parts, One Product -------------------------------------------------------------------- Why: A product is many subsystems agreeing with each other; that's the capstone. A product is many subsystems agreeing with each other. Most famous failures happened on the lines between them. A phone in your pocket does several things at once. The radio talks to a tower, the screen glows, the processor thinks, the battery pushes current through all of it, and the whole slab has to stay cool enough to hold. Every one of those parts was designed by a different team. Yet when you press a button, they agree. Now imagine one of them quietly disagrees. One team counts in one unit, another in a different one. Nothing is wrong with any single part. Everything is wrong with the product. This subject is about that gap: not how to design a part, but how to make a hundred parts mean the same thing. Engineers who have taken it carry a habit for life. When someone proudly shows a clever component, they ask three questions before admiring it: what does it plug into, what does it assume about its neighbours, and what happens when it is wrong? The course in one box Who takes it: final-year mechanical and aerospace engineers, usually as a capstone (a year-long team project), and in many universities every engineering major has a version of it. What it is for: turning a pile of well-designed parts into one product that meets what someone actually needed, on a budget and a schedule. Comes after: Machine Design (34/19), Control Systems (34/20) and Software Engineering (34/31), which is also a Computer Science subject. Leads to: nothing on the map; it is the last stop of the Mechanical & Aerospace path. What it leads to is the job, where most of the work is this. What this piece gives you, and what it doesn't: you won't be able to run a real programme or write a requirements document an auditor would accept. You will see why a spacecraft is weighed more often than it is admired, and why the famous failures were almost all about neighbours disagreeing. A product is many subsystems agreeing A system is a set of parts whose combined behaviour is something none of them can do alone. A car is not an engine plus wheels. It is an engine that must fit in the space the body left, make heat the radiator can reject, and shake at frequencies the mounts can absorb. Each sentence in that list is an agreement between two subsystems. Systems engineers write those agreements down in two forms. A requirement says what the whole thing, or one part, must do, in a sentence a test can settle: "The device shall have a mass of no more than 12 kg." A good requirement can be checked by measuring; "lightweight" cannot. An interface says exactly what passes between two parts: the bolt pattern, the voltage and connector, the message format, the units. A document that holds these agreements is called an interface control document, and it is the most boring and most valuable thing in the project. Most of a system's trouble is not inside the parts. It is on the lines between them. The craft is a loop: break the whole into subsystems, give each a budget and an interface, let teams work, then bring the results back and see whether the whole still works. The V: build down, prove up The classic picture is a letter V. Down the left side you go from need to detail: what the customer wants, then system requirements, then the design of each subsystem, then the parts. At the bottom you build. Up the right side you test, and the key idea is that each level of testing answers to the level on the left at the same height. You test the part against the part's design, the assembled subsystem against its requirements, the whole product against the system requirements, and finally the product in real use against the original need. That gives the two words every systems engineer uses precisely. Verification asks whether you built it right: does it meet the requirements as written? Validation asks whether you built the right thing: does it actually solve the problem the person had? A product can pass every verification and still fail validation, if the requirements were wrong. The V-model has several roots: it appeared in the late 1980s in Germany (for defence procurement) and in the United States, and it is a teaching picture, not a law. Real projects loop back and forth across it, and agile software teams (see Software Engineering, 34/31) spiral through small versions of it. Interactive · Walk the V Now classify. Is each of these verification or validation? Worked example: a requirement says the part "shall have a mass of no more than 12 kg". You put it on a scale and read 11.6 kg. That is verification, and the requirement is met with 0.4 kg (about 3%) to spare. Whether anyone can lift a 12 kg part into place is a validation question, and the scale cannot answer it. Trade studies: choosing without a perfect answer Early in a project there are always several ways to do a job, and none is best at everything. A trade study scores each option against the criteria that matter, weights the criteria by how much they matter, and adds it up. The sum is not the point. The point is that the weights make the argument visible: two people who disagree about the answer are usually disagreeing about the weights, and now they can say so. score = (w1s1 + w2s2 + …) ÷ (w1 + w2 + …) Interactive · Wheels, tracks or legs? A small delivery robot. The scores (1 = poor, 10 = excellent) are invented for the exercise; the point is what the weights do. Drag a weight and watch the winner change. Try it: with the starting weights, plain wheels win comfortably. Put nearly all the weight on performance over rough ground and legs edge ahead. Nobody's arithmetic changed, only what they cared about. A mature team also checks sensitivity: how far can a weight move before the winner flips? If a tiny change flips it, the study has not decided anything yet. Budgets and margins: mass, power and heat Hardware obeys sums. A spacecraft has a mass budget (everything added up against what the rocket can lift), a power budget (everything it draws against what the panels and battery supply) and a thermal budget (everything it dissipates against what it can radiate away). Each subsystem gets an allowance. The system engineer's job is to add them up, often, and to be suspicious of the total. The suspicion is earned. Designs get heavier. NASA's systems engineering handbook, in an example, says spacecraft dry mass can grow by as much as 25 to 30 percent between detailed design and launch. So teams hold a margin: they add a deliberate percentage to the current best estimate, larger when the design is young and shrinking as parts are actually built and weighed. A margin is not padding. It is an honest admission of what you don't know yet. Interactive · A mass budget with margin An invented small satellite with a 500 kg launch limit. Its best estimate is the sum of the parts below. Choose how much margin you hold, then how much the design really grows. Margin you hold Growth the design really shows The best estimate here is 415 kg. With 20% margin the budget reads 498 kg, comfortably under 500. Then the design grows by 25%: 415 × 1.25 = 518.75 kg, and the satellite no longer fits. The margin was not too generous; the growth was bigger than the margin. That is how real programmes learn their margins were thin, usually late. The budgets are coupled, which is the deeper lesson. Take mass in a rocket. The rocket equation says that the ratio of liftoff mass to final mass is e raised to (Δv divided by exhaust speed). One extra kilogram of dead weight at the end therefore costs several kilograms of propellant at the start. mstart ÷ mend = eΔv ÷ (Isp × g0) Interactive · The snowball A stage with 1,000 kg of structure and payload that must change its speed by 4,000 m/s. The engine's specific impulse is 320 s (a good chemical engine). Add dead weight and see what it costs at the start. Extra dead weight Velocity change needed At 4,000 m/s the ratio is 3.58, so each extra kilogram at the end costs 2.58 kg more propellant, and 3.58 kg of extra liftoff mass in all. Ten kilograms of "just a bracket" becomes about 36 kg on the pad. This simple model ignores that the bigger tanks weigh more too, so reality is worse. This is why aerospace engineers fight for grams. Heat has a vacuum-sized version of the same coupling. In space a spacecraft can only get rid of heat by radiating it. The hotter and larger its radiating surface, the more it sheds, following the fourth power of temperature: P = ε σ A T4  →  A = P ÷ (ε σ T4) Interactive · Where does the heat go? Every watt the electronics use comes out as heat. With a radiator at 300 K (27 °C), emissivity 0.85, and ignoring sunlight, how much area must it have? Heat to reject Radiator temperature At 300 K, 100 W needs about 0.26 m² of radiator. So a power decision, a few more watts for a faster computer, becomes a thermal decision (more area or a hotter radiator) and a mass decision (more structure to carry it). One number changed and three subsystems moved. That chain is what "system" means. Interfaces are where the failures live Two famous losses show this better than any lecture, and both are now textbook cases. On 23 September 1999, NASA's Mars Climate Orbiter lost contact during its orbit-insertion burn. The investigation found the root cause was a failure to use metric units in a piece of ground software. A ground file that described the effect of the spacecraft's small thruster firings reported it in pound-force seconds. The navigation software that used them expected newton-seconds, as the specification said. One pound-force is 4.448 newtons, so every number was read about 4.45 times too small. The board noted that the project's own checks did not catch it: it cited a lack of complete end-to-end verification of the navigation software and models. It was meant to pass Mars at about 226 km. Calculations afterwards put it as low as about 57 km, below the roughly 80 km thought survivable. On 4 June 1996, the first Ariane 5 rocket, flight 501, veered and broke up about 37 seconds after the launch sequence began, roughly 30 seconds after liftoff. The inquiry, chaired by Jacques-Louis Lions, traced it to the inertial reference system, which had reused software from Ariane 4. A number describing horizontal motion, held as a 64-bit floating-point value, was converted to a 16-bit signed integer. Ariane 5 flew a faster early trajectory than Ariane 4, the value no longer fitted, and the conversion raised an error that was not handled. The two redundant units had identical hardware and software, so both shut down together, and the rocket's computer then read diagnostic data as if it were real guidance. The board's lesson was not "a programmer blundered". It was that an assumption valid for the old rocket was never tested against the new one: the board reported that the new rocket's trajectory data was left out of the unit's specification, and the units were not part of the full closed-loop simulation. Three of the seven vulnerable conversions had been left unguarded on the belief that the values were physically limited, and that belief proved wrong for this one. Interactive · The two interface lab 1. Units. Slide the real impulse. The ground file gives a number in pound-force seconds; navigation reads it as newton-seconds. Real impulse of a burst 2. Range. A 16-bit signed integer holds -32,768 to 32,767. Slide the value being converted, and choose whether the conversion is guarded. Value to convert Both stories share a shape. Every part worked for the conditions it was designed for. The failure sat in an assumption between parts. Systems engineers answer this with the same few habits: write the interface down, put units and ranges in it, test the parts together as early as possible, and test with real data rather than the data you expect. FMEA: asking how it breaks before it does Failure mode and effects analysis, or FMEA, is a structured way of asking "how could this break, and what happens then?" It was first described in a US military procedure in 1949, and by the early 1960s NASA contractors were using versions of it on programmes such as Apollo. For each part, a team lists the ways it can fail, the effect of each, and how it could be noticed. Then they rate three things from 1 to 10: how severe the effect is, how often it will happen (occurrence), and how hard it is to detect before it hurts someone. Multiply the three and you get a risk priority number (RPN), used to rank what to fix first. Interactive · Rank the risks An electric scooter. The ratings are invented for the exercise. Pick a failure and move its three ratings (for detection, 10 means hardest to detect). Severity Occurrence Detection (10 = hardest) The starting numbers show the catch. Battery overheating while charging has the worst possible consequence (severity 10), yet its RPN of 60 ranks it third, below a loosening bolt that scores 192. That is not an arithmetic error: multiplying scales hides a very bad but rare event behind a mild but common one. Critics have said exactly this, and the 2019 joint automotive handbook of the AIAG and VDA replaced the RPN with an "action priority" table that looks at severity first. The honest use of FMEA is the conversation it forces, not the number. Integration, test and what the capstone teaches The right-hand side of the V is where the plan meets the world. Integration means putting the parts together a little at a time, not all at once on the last week, so a surprise is found when it is cheap. Test is the evidence: vibration tables for launch, thermal-vacuum chambers for space, wind tunnels, fatigue rigs, and for software, automated suites (see 34/31). The rule teams repeat: test as you fly, and fly as you tested. The senior capstone is the one place most students get to feel all of this at small scale. In the US, accreditation criteria from ABET require engineering programmes to include a culminating major design experience that draws on earlier coursework and works within multiple realistic constraints, which is why almost every programme has one. A typical capstone gives a team of four to six students a real or realistic customer, a budget of a few thousand dollars, and two semesters. The lessons are rarely the equations. They are these: The customer's first sentence is not the requirement; asking the second question is. The part you assumed would be easy is the one that is late. The interface you did not write down is the one that does not fit. Your design is only done when somebody else can build, test and run it from your documents. What a graduate just sees The hard part of a product is on the lines between its parts, so read the interfaces first. A requirement you cannot test is a wish; every requirement should say how it will be verified. Verification is "built it right"; validation is "built the right thing". They fail separately. Everything gets heavier, hotter and later than the first estimate, so hold margin and shrink it only with evidence. In aerospace a kilogram late in the stack costs several at the start, and a watt costs heat, area and mass. Redundancy that runs the same software fails the same way twice. Ask "how could this break?" before it does, and rank by consequence, not only by a product of scores. Disagreements about a design are usually disagreements about weights; make them explicit. Where it shows up next Machine Design (34/19) gives the parts and their loads; this is how they become a product. Mechanics of Materials (34/11) supplies the mass you can afford to remove. Control Systems (34/20): a controller is a subsystem whose interface is a sensor, an actuator and a rate. A wrong sign or a wrong unit there is the Mars Climate Orbiter story with a motor. Heat Transfer (34/16) and Thermodynamics (34/14) are the thermal budget. Vibrations (34/17) is why launch loads decide what survives. Software Engineering (34/31) is the other half of nearly every modern product; the Ariane story sits exactly between them. Computer Security (34/33) is the same discipline applied to an adversary. In the real world: spacecraft, aircraft, cars, medical devices and phones are built this way, and in companies the job title is often "systems engineer" or "integration and test engineer". If you want to go deeper 📘 Systems Engineering: Principles and Practice, Kossiakoff, Seymour, Flanigan and Biemer (Wiley, 3rd ed., 2020): the standard university textbook for the whole subject. 📘 Product Design and Development, Ulrich, Eppinger and Yang (McGraw-Hill, 7th ed., 2020): a common capstone text on turning a need into a product. 🌐 NASA Systems Engineering Handbook: free to download from NASA, and the best-known practical reference. 📘 To Engineer Is Human: The Role of Failure in Successful Design, Henry Petroski (1985): a gentle, readable book on why things fail and what designers learn. 🌐 MIT OpenCourseWare 16.842, Fundamentals of Systems Engineering (Prof. Olivier de Weck): free and complete, though a graduate-level course, so expect it to be dense. The takeaway A product is many subsystems agreeing. Systems engineering is the discipline of writing the agreements down (requirements and interfaces), proving them level by level (the V, verification and validation), spending scarce mass, power and heat honestly (budgets and margins), and hunting the ways it can break (FMEA). The big failures were not exotic physics. They were two neighbours with different assumptions. Try this: pick something in your home with two or more parts that must work together, such as a phone and its charger, or a door and its lock. Write down three things the one part assumes about the other. Which assumption would be most expensive if it were wrong? Sources & Further Reading Sources. Mars Climate Orbiter: NASA press release of 10 November 1999 on the Mishap Investigation Board's Phase I report; Wikipedia, "Mars Climate Orbiter". Ariane 5 flight 501: the Inquiry Board report of 19 July 1996 (chair Jacques-Louis Lions); Wikipedia, "Ariane flight V88". V-model and verification versus validation: Wikipedia, "V-model". FMEA history and the move to action priority: Wikipedia, "Failure mode and effects analysis". Dry-mass growth of 25 to 30 percent: NASA Systems Engineering Handbook SP-6105 (1995), p. 62, as quoted in a project document. Accreditation: ABET Criterion 5. Computed in node: rocket-equation ratio 3.577, radiator area 0.256 m², the mass-budget sums, trade-study scores and RPN products. 🌐 NASA Systems Engineering Handbook 🌐 MIT OpenCourseWare 16.842: Fundamentals of Systems Engineering 🌐 NASA: Mars Climate Orbiter Phase I report press release 🌐 Ariane 501 Inquiry Board report (mirror) 📘 To Engineer Is Human, Henry Petroski