Stand on a hillside with a map. Height depends on two things at once: how far east you are and how far north. Single-variable calculus handles one input at a time. The hillside needs a new question: of all the directions I could step, which one rises fastest?
Almost nothing real depends on one number. Temperature in an engine block depends on position in three dimensions. The error of a language model depends on billions of adjustable weights. Air pressure depends on place and time. This course takes the slope-and-area ideas of Calculus I (34/01) and lets them live in more than one dimension. Surprisingly little new is needed: one new arrow, the gradient, and one new accounting rule, the idea that what flows out of a region is what is made inside it.
Who takes it: second-year Mechanical & Aerospace Engineering students; Computer Science students meet it in year 4 when machine learning needs it.
What it is for: calculating change and totals when many things vary at once.
Comes after: Calculus I (34/01).
Leads to: Physics II: Electricity & Magnetism (34/07), Fluid Mechanics (34/15), Machine Learning (34/32), and through them Heat Transfer (34/16) and Propulsion (34/21).
Honest line: you will not be able to grind through a surface integral after this. You will see why a gradient points uphill, why training an AI is "just" rolling downhill, and what the big theorems say in plain words.
Two inputs make a landscape, and a partial derivative is one slope
A function of two inputs is a surface: height above each point of a map. Engineers draw it the way hikers do, with contour lines: closely spaced lines mean steep ground, widely spaced lines mean gentle ground.
To ask "how steep?" you must say in which direction. Walk due east and note the slope: that is the partial derivative with respect to x, written ∂f/∂x. Hold east still and walk north: that is ∂f/∂y. The recipe is simple: treat every other variable as frozen and differentiate as before.
The landscape below has a tall hill near (1, 1) and a lower one near (−1.2, −0.8). Drag the point on the map (or use the sliders). The cyan arrow shows the slope eastward and the green arrow the slope northward; a leftward or downward arrow means the ground falls in that direction.
At the starting point (−0.2, 0.3) the height is 0.684, the eastward slope +0.621 and the northward slope +0.226: the ground rises both ways, more steeply towards the east.
The gradient points straight uphill
Put the two partial slopes together as one arrow, east component first: that is the gradient, written ∇f. It has a beautiful property: it points in the direction of steepest ascent, and its length is how steep that is. Step in any other direction and the slope is the gradient's length times the cosine of the angle between your step and the arrow. Straight along it, cosine is 1: the most you can get. Sideways to it, cosine is 0: you walk along a contour and stay level.
Two consequences follow. The gradient is always perpendicular to the contour line through that point, which is why water runs across contours, not along them. And the same arrow governs heat: in Fourier's law, heat flows down the temperature gradient, from hot to cold, at a rate proportional to its steepness. Many diffusion laws have this shape: flow follows the negative gradient.
Turn the pink direction until the slope is largest: it lines up with the orange arrow. Turn it 90° from the arrow and the slope is zero. At the very top of a hill the gradient shrinks to nothing, which is the multivariable version of "zero slope means a peak" from Calculus I.
Rolling downhill is how a network learns
Now flip the sign. A machine-learning model has adjustable numbers (weights), and a loss measures how wrong it is on the training data. The loss is a landscape over the weights, with one dimension per weight. To learn, take a small step against the gradient, so downhill, then recompute and repeat. This is gradient descent, which Augustin-Louis Cauchy proposed in 1847 for solving equations, long before computers. The chain rule from Calculus I is what lets the gradient be computed layer by layer, in the method called backpropagation, published by Rumelhart, Hinton and Williams in Nature in 1986. A model like GPT-3 (2020) has 175 billion weights, so its landscape has 175 billion dimensions, yet the rule is the same arrow.
The step size matters, and you can feel why. In the bowl below, one direction is twelve times steeper than the other. Too small a step crawls; just right is fast; too large and each step overshoots the valley floor by more than it started and the loss explodes. For this bowl, steps above 1/6 (about 0.167) diverge.
Real training uses refinements (momentum, adaptive step sizes, random mini-batches), but all of them are ways of coping with exactly this picture: a long narrow valley where the steepest direction is not the direction of the destination.
Double and triple integrals add up a landscape's worth
Integrals extend the same way. A single integral adds thin slices along a line. A double integral adds tiny patches over an area: if each patch has a density (kilograms per square metre), the sum is the plate's mass. A triple integral does the same over a solid, with density in kilograms per cubic metre. The recipe is slice, multiply, add, repeated once per dimension.
The same sums also give the centroid, the balance point: weight each position by its piece's mass and divide by the total. Engineers use it constantly: the centre of mass of a rocket, a car or an aircraft decides whether it is stable. Try a plate 2 m long and 1 m wide whose density rises along the length.
The density here is 1 + ax + by kg/m². With a = 3 and b = 0, the plate's mass is 2 + 2×3 = 8 kg and it balances at x = 1.25 m, shifted from the middle (1 m) towards the heavy end.
A vector field puts an arrow on every point
Wind on a weather map, water in a pipe, the pull of a magnet: each attaches an arrow (a direction and strength) to every point in space. That is a vector field. Two numbers computed from the field's partial derivatives tell you most of what it is doing locally.
Divergence: is the field spreading out from here, or converging? A tap under water is a source; a drain is a sink. A positive divergence means a small blob of fluid is expanding. Curl: is the field spinning? Put a tiny paddle wheel in the flow. If it turns, the curl is not zero, and the sign tells you which way.
Pick a field and let a round blob of fluid (grey dashes show where it started) be carried along. You will see that a source inflates the blob without turning it, a whirlpool turns it without changing its area, and a saddle stretches it one way and squeezes it the other while keeping its area.
Divergence and curl are the two words in Maxwell's equations of electromagnetism (Physics II, 34/07) and in the equations of fluid flow (Fluid Mechanics, 34/15). Spreading and spinning are the two things a field can do locally, besides moving along as a whole.
What flows out of a region is what is made inside
Here is the idea behind the three big theorems, in the language of a bathtub. If water pours out through the boundary of a region, then some taps inside must be producing it (or the level inside must be falling). Add up every tiny source and sink inside and you get the net flow out through the boundary. Nothing more is needed.
That is the divergence theorem, the three-dimensional version of the statement. Green's theorem is the flat version (the name honours George Green, whose Essay of 1828 contained the related surface-to-volume formula), and Stokes's theorem relates the spinning of a field inside a surface to the flow around its edge. All three say the same kind of thing: a sum over a boundary equals a sum of local changes inside. It is why one can measure a rotating machine from outside and why conservation laws can be written either as "this region's total changes by what crosses its walls" (integral form) or as an equation about each point (differential form).
The first number adds up the field's push through the circle itself. The second never looks at the circle's edge: it adds the divergence in every patch inside. They agree for every field, radius and position. For the plain source (x, y), the divergence is 2 everywhere, so a unit circle encloses 2 × π = 6.283 units of production, and exactly 6.283 flows out through its rim.
What a graduate just sees
- A slope needs a direction; a partial derivative is the slope with every other input held still.
- The gradient points straight uphill, its length is the steepness, and it is perpendicular to the contour.
- "Downhill" is the whole idea behind training a model: step against the gradient, and mind the step size.
- Heat, and many other things, flow down a gradient from high to low.
- A double or triple integral is slice, multiply, add, once per dimension; mass and balance point come from it.
- A field can only spread (divergence) or spin (curl), locally.
- What leaves a region through its wall is exactly what its insides produce.
Where it shows up next
- Physics II: Electricity & Magnetism (34/07): Maxwell's equations are statements about the divergence and curl of the electric and magnetic fields.
- Fluid Mechanics (34/15): conservation of mass is a divergence equation, and the divergence theorem turns "what crosses the walls of this control volume" into "what happens inside it".
- Heat Transfer (34/16): conduction is heat flowing down the temperature gradient; the heat equation is the gradient and divergence together.
- Machine Learning (34/32): training is gradient descent on a loss landscape with millions or billions of dimensions.
- Mechanics of Materials (34/11) and Dynamics (34/12): centroids and moments of inertia are integrals of exactly the kind you added up above.
If you want to go deeper
- 📘 Textbook: Stewart, Clegg and Watson, Calculus: Early Transcendentals (9th ed.), chapters 14 to 16: the standard multivariable text, rich in examples.
- 📘 Textbook: Jerrold Marsden and Anthony Tromba, Vector Calculus (6th ed., W. H. Freeman): the classic for the geometry of divergence, curl and the big theorems.
- ▶️ Gentler: Khan Academy's Multivariable Calculus course, which has videos with a strong visual feel, and Grant Sanderson's multivariable lessons.
- 🌐 Free course: MIT OpenCourseWare 18.02SC Multivariable Calculus, with lectures, notes and exercises.
With many inputs, a slope needs a direction. The partial derivatives are the slopes along the axes; the gradient joins them into one arrow that points uphill and measures how steep it is. Walk against it and you are doing gradient descent, which trains models with billions of weights and also describes how heat flows. Integrals extend to areas and volumes, giving mass and balance points, and for flows the whole subject rests on one bookkeeping rule: what leaves a region through its boundary is what its interior produces.
Try this: on any hill, a park slope or a street, find the line along which a ball would roll: that is the negative gradient. Then notice that the ball's path crosses the contours at right angles.
Sources. Gradient descent: A.-L. Cauchy, "Méthode générale pour la résolution des systèmes d'équations simultanées", Comptes rendus 25 (1847), pp. 536–538; Claude Lemaréchal, "Cauchy and the gradient method", Documenta Mathematica, Extra Volume ISMP (2012). Backpropagation: Rumelhart, Hinton and Williams, Nature 323 (1986), pp. 533–536. GPT-3's 175 billion parameters: Brown et al., "Language Models are Few-Shot Learners" (2020, arXiv 2005.14165). Green's Essay of 1828 containing the theorem named after him: MacTutor biography and the essay itself. Computed in node: the landscape f = 2e−((x−1)²+(y−1)²)/1.5 + 1.2e−((x+1.2)²+(y+0.8)²)/1.0 and its analytic gradient (checked against finite differences, agreement to 3 × 10−10); gradient descent on ½(x² + 12y²), stable for step sizes below 2/12; plate mass and centroid by 3,200-cell sums against the exact formulas; blob area against ediv·t; circle flux against the divergence integral for four fields (agreement to 4 decimals).
- 🌐 MIT OpenCourseWare 18.02SC Multivariable Calculus
- 🌐 Khan Academy: Multivariable calculus
- 🌐 Documenta Mathematica, ISMP volume, which contains Lemaréchal's short history of gradient descent
- 📘 Stewart, Clegg and Watson, Calculus: Early Transcendentals, 9th ed. (Cengage)
- 📘 Marsden and Tromba, Vector Calculus, 6th ed.