You pinch two fingers on a photo and it rotates and zooms smoothly under them. A game character turns to face you and every corner of its face swings round together. A search for "best pizza near me" ranks millions of pages in a fraction of a second. A bridge designer presses "solve" and a screen of colour shows how every beam is being squeezed.
Underneath all four is the same object: a grid of numbers, and one habit of thought about what a grid of numbers does. A graduate of a linear algebra course no longer sees a table of numbers. They see a machine that grabs the whole plane, or the whole of space, and moves it.
This piece builds that picture with seven things you can drag. You will not come out able to compute a determinant of a 5×5 by hand (nobody does that on purpose). You will come out seeing why a system of equations sometimes has no answer, why a bridge hums at particular pitches, and why the chips that run AI are built for one operation only.
Who takes it: nearly every engineering and computer science student, usually in the first or second year. It is the single most reused maths course in both majors.
What it's for: handling many unknowns and many dimensions at once, with one set of tools.
Comes after: only school algebra and a little geometry. (Calculus helps but is not required.)
Leads to: Statics (34/10), Control Systems (34/20) and Machine Learning (34/32) list it as a prerequisite; Vibrations (34/17), Computer Architecture (34/25) and Multivariable Calculus (34/02) lean on it constantly.
Honest line: you won't be able to run a finite element program after this piece. You will see why it spends nearly all its time solving one equation, Ax = b. If you want an easier first read, try the library's Vectors, Matrices and Linear Transformation (12/14); this one goes further into solving, eigenvectors, fitting and why hardware cares.
A matrix moves all of space at once
Start with the picture a graduate keeps. A 2×2 matrix is not four numbers. It is two arrows. The first column says where the arrow pointing one step right (call it i) ends up. The second column says where the arrow pointing one step up (j) ends up. Once you know where those two land, you know where everything lands, because every point is some amount of i plus some amount of j, and the machine is straight and even: grid lines stay straight, and evenly spaced lines stay evenly spaced.
Read that in words: the matrix times a point is "that many of the first column, plus that many of the second". Try it. The grey grid is the original plane; the blue grid is where the matrix sends it.
Without JavaScript: the rotate-30° matrix sends the arrow (1, 0) to (0.87, 0.5) and (0, 1) to (−0.5, 0.87). Every grid line turns 30° and the unit square keeps area 1.
Notice what the shaded unit square does. In "Rotate" it keeps its size. In "Stretch" it doubles. In "Shear" it leans over but keeps its area. That area change is the next idea.
The determinant is how much area gets scaled
The determinant is one number that says how much the machine scales area. If the unit square becomes a shape with area 5, the determinant is 5. If it becomes a mirror image, the determinant is negative: area is kept (or scaled) but orientation is flipped. For a 2×2 matrix with columns (a, c) and (b, d) the rule is short:
Worked number: the matrix with columns (2, 1) and (1, 3) has det = 2×3 − 1×1 = 5, so it stretches every area in the plane by a factor of five. In three dimensions the determinant scales volume.
The interesting case is zero. A determinant of zero means the machine flattens space: the plane is squashed onto a line (or a point), and the area becomes nothing. Squashing cannot be undone, because many different starting points land on the same spot. Drag the slider until the determinant reaches zero.
Without JavaScript: the matrix with columns (1, 0.5) and (s, 1) has determinant 1 − 0.5s. At s = 2 the determinant is 0, the whole plane lands on one line, and the points (0, 0) and (2, −1) both land on the origin.
The orange dot is a probe at (2, −1). At s = 2 it lands exactly on the origin, the same place as the point (0, 0). Two different inputs, one output: no machine could tell you which one you started from. That is the whole meaning of "this matrix has no inverse", and it is why a zero determinant is the alarm bell in every solver you will ever meet.
Solving many equations at once: Ax = b
Now use the machine backwards. Given the output b, which input x did the machine start from? That is the equation Ax = b, and it is the most common problem in applied mathematics. Every time you have several unknowns that are tied together by several straight-line relationships, you have one.
A circuit is the cleanest example. Two loops share a resistor. Kirchhoff's voltage law says the voltage gained around each loop equals the voltage lost, which gives one equation per loop. With a 10 V battery, R1 = 2 Ω, R2 = 3 Ω (shared) and R3 = 4 Ω:
−3I1 + 7I2 = 0
The determinant is 5×7 − 3×3 = 26, so there is exactly one answer: I1 = 2.69 A and I2 = 1.15 A. Change the resistors below and watch the matrix and the answer change together.
Without JavaScript: with 10 V, 2 Ω, 3 Ω and 4 Ω the equations are 5I1 − 3I2 = 10 and −3I1 + 7I2 = 0, its determinant is 26, and the loop currents are 2.69 A and 1.15 A.
The reason an engineer cares: nothing about this method knows it is a circuit. A truss is the same. At each joint of a bridge, the horizontal forces must add to zero and so must the vertical ones: two equations per joint. A simple triangle truss has three joints and so six equations, for three member forces and three support reactions, and the answer is one solve of Ax = b. A real structure has thousands of joints. Finite element programs routinely solve systems of many thousands of equations, and research codes have reported solving over a million unknowns at once. The method is the same one you just used; only the size has changed. (Statics, 34/10, is this idea applied to forces.)
A zero determinant in a structure's equations is a warning: the structure may be able to move without any member pushing back.
Eigenvectors: the directions a transformation only stretches
Most arrows are turned by a matrix as well as stretched. But for many matrices there are special directions where the arrow comes out pointing the same way (or exactly backwards), only longer or shorter. Those are the eigenvectors, and the stretch factor along each is its eigenvalue. (Eigen is German for "own": the matrix's own directions.)
In words: feed the matrix the special arrow v and it hands back the same arrow multiplied by a plain number λ. For the matrix with columns (2, 1) and (1, 2) the special directions are 45° (stretched by 3) and 135° (stretched by 1). Every other arrow gets bent towards the stretchy one. Turn the dial and look for the moments the two arrows line up.
Without JavaScript: for the matrix with columns (2, 1) and (1, 2), an arrow at 45° comes out at 45° three times longer, and an arrow at 135° comes out at 135° unchanged. A pure rotation has no such direction at all.
Three things to notice. The shear has only one special direction (along the x-axis, where nothing happens at all). The rotation has none, because every arrow turns. And "Two springs" has two special directions with different stretch factors, 1 and 3; hold that thought.
Eigenvectors at work: bridges that hum, pages that rank
Vibration. Picture two equal masses on springs between two walls. Push them any way you like and the motion looks messy. But there are exactly two clean patterns: both swing together, or they swing in opposite directions. These are the modes, and they are the eigenvectors of the "two springs" matrix above, (1, 1) and (1, −1). The eigenvalues 1 and 3 fix how fast each mode vibrates: the squared frequency is the eigenvalue times k/m, so the opposite-motion mode vibrates √3 ≈ 1.73 times faster. Any messy motion is just a mix of the modes. That is how engineers find which pitches will shake a wing, a bridge deck or a drive shaft (Vibrations, 34/17).
Ranking pages. Google's founders modelled a web surfer who follows a random link from each page, and now and then jumps to a random page instead. The long-run share of time spent on each page is its PageRank, and it is the eigenvector (with eigenvalue 1) of the matrix describing one step of that surfer. In their 1998 paper Brin and Page used a "damping" probability of 0.85 for following links. You can find the eigenvector without solving anything fancy: start with equal scores and apply the machine over and over. Press Step.
Without JavaScript: A links to B and C, B links to C, C links to A, D links to C. After many steps with damping 0.85 the ranks settle to A 0.373, B 0.196, C 0.394, D 0.038. D, which nobody links to, keeps only the base share.
After about twenty steps the numbers stop changing: the vector has become an eigenvector, because applying the machine again returns the same vector. Page C wins because two pages point at it and one of them (A) is itself well-ranked.
Least squares: the best answer when there is no exact one
Real data is never perfect. Six measurements that "should" lie on a straight line will not, so the equations Ax = b have more equations than unknowns and no exact solution. Do you give up? No. You choose the line whose total mistake is smallest. Measure each miss vertically, square it, and add up the squares. The least squares line makes that total as small as possible.
In pictures: each squared miss is literally a square. Drag the line and watch the squares shrink and grow.
Without JavaScript: for the six points (1, 1.2), (2, 2.9), (3, 3.1), (4, 4.8), (5, 5.2), (6, 7.1) the best line is y = 1.089x + 0.24, with total squared miss 0.798. The guess y = x gives 2.75.
The graduate's picture is geometric: the right-hand side b is a point outside the flat sheet of everything the matrix can reach, and the best answer is the point on the sheet closest to it, found by dropping a perpendicular. That perpendicular condition gives the one equation to remember:
In words: multiply both sides by the matrix flipped on its side and you get a square system you can solve. The history is a priority fight: Legendre published the method in 1805, Gauss said he had used it since 1795 and published in 1809, and Gauss's use of it to predict where the asteroid Ceres would reappear in 1801 made it famous. Today it sits under every trend line, every calibration of a sensor, and the first model most machine-learning students ever train.
Why GPUs and AI are matrix multiplication
Multiplying two matrices is just many dot products: every cell of the answer is one row of the first matrix paired with one column of the second, multiplied and added. Click a cell below and see the row and column that make it.
The speeds are round illustrative numbers, not any particular chip.
Without JavaScript: multiplying two n×n matrices takes n×n×n multiplications. For n = 1,000 that is 1,000,000,000, and the 1,000,000 cells of the answer can all be computed independently.
Two facts follow. First, the work is huge: n×n matrices need n³ multiplications, so 1,000×1,000 is a billion. Second, the cells are independent, so a thousand workers (or a million) can each take cells at the same time. That is exactly the shape of work a graphics chip was built for, because drawing a 3D scene means pushing every corner of every triangle through a matrix that rotates, moves and projects it.
Modern neural networks turned out to be the same shape. A layer of a network is a matrix of learned numbers times a vector (or a matrix) of inputs. In their scaling-law paper, Kaplan and colleagues estimate that a transformer language model with N parameters needs roughly 2N arithmetic operations to process one token. GPT-3, with 175 billion parameters, therefore needs about 3.5×1011 operations for each token, nearly all of them inside matrix products. NVIDIA's Volta GPUs (2017) put dedicated "tensor core" circuits on the chip that do nothing but a small matrix multiply-and-add (4×4) per clock tick. When people say AI runs on GPUs, they mean it runs on hardware shaped like linear algebra. (Computer Architecture, 34/25, and Machine Learning, 34/32.)
What a graduate just sees
- A matrix is a machine that moves all of space; its columns say where the basis arrows go.
- The determinant is the area (or volume) scale factor, and zero means the machine flattens space and cannot be undone.
- "Solve Ax = b" is the same question for a circuit, a truss, a fitted line and a game physics step.
- If there are as many independent equations as unknowns there is one answer; fewer, many answers; contradictory, none.
- Eigenvectors are the directions a machine only stretches; they simplify any problem the machine drives.
- A vibrating structure is a mix of modes, one per eigenvector; the eigenvalues are the pitches.
- When there is no exact answer, take the closest one: project onto what you can reach.
- Matrix multiplication is many independent dot products, which is why it parallelises and why hardware is built around it.
Where it shows up next
- Statics (34/10) and Mechanics of Materials (34/11): joint and member forces are a linear system; stiffness matrices tie loads to deflections.
- Vibrations (34/17) and Dynamics (34/12): mode shapes and natural frequencies are eigenvectors and eigenvalues of mass and stiffness matrices.
- Control Systems (34/20): the eigenvalues of a system's matrix say whether a drone's wobble dies away or grows.
- Machine Learning (34/32): data is a matrix, a layer is a matrix, and training adjusts matrices; least squares is the model you meet first.
- Computer Architecture (34/25) and graphics: GPUs and tensor cores exist to multiply matrices fast.
If you want to go deeper
- 📘 Gilbert Strang, Introduction to Linear Algebra (Wellesley-Cambridge Press; 6th edition, 2023). The standard first course for engineers; best for the four fundamental subspaces and for application.
- 📘 David C. Lay, Steven R. Lay and Judi J. McDonald, Linear Algebra and Its Applications (Pearson; 6th edition, 2021). Gentle, example-heavy, widely assigned. For proofs, Sheldon Axler's Linear Algebra Done Right (Springer; 4th edition, free to read online at linear.axler.net) is the elegant theoretical route.
- ▶️ 3Blue1Brown, Essence of Linear Algebra: the best visual series for learning alone; it builds exactly the "matrix as moving space" picture used here.
- 🌐 MIT OpenCourseWare 18.06, Linear Algebra (Gilbert Strang): full video lectures, free.
Linear algebra is the study of machines that move all of space at once. Read a matrix by its columns, read its determinant as the area scale, solve Ax = b when you need to run the machine backwards, look for the eigenvector directions when you want it to be simple, and project onto what you can reach when there is no exact answer.
Try this: next time an app zooms, rotates or recolours a photo, picture the two column arrows moving. And for the fun of it, in the circuit widget raise R2 to 10: the shared resistor starts to decide what both loops do.
Sources. Strang, Introduction to Linear Algebra (6th ed., 2023) and MIT 18.06 for the standard course content. Kirchhoff loop equations and the spring-mass modes are textbook derivations; every number above was recomputed in code. Legendre's 1805 publication and Gauss's priority claim and Ceres prediction: Wikipedia, "Least squares"; MacTutor biography of Legendre. PageRank and the 0.85 damping factor: Brin and Page, "The Anatomy of a Large-Scale Hypertextual Web Search Engine" (1998). Volta tensor cores (4×4 matrix multiply-accumulate): Markidis et al., "NVIDIA Tensor Core Programmability, Performance & Precision" (2018). About 2N operations per token: Kaplan et al., "Scaling Laws for Neural Language Models" (2020); GPT-3 at 175 billion parameters: Brown et al. (2020). Million-unknown structural solves: NASA Technical Reports Server and OSTI conference papers on parallel sparse solvers.
Go deeper. The books and courses above.