GPUs: From Video Games to the Engine of AI

Follows: Processors: How a CPU Thinks, From the Transistor to the Chip in Your Pocket, Vectors and Matrices: The Geometry of Linear Transformations and How Large Language Models Actually Work

In this deep dive
1. Painting a frame: what "rendering" actually means
2. A few smart workers, or thousands of simple ones
3. Before the GPU: arcades, Silicon Graphics and 3dfx
4. "The world's first GPU": the GeForce 256 claim, examined
5. From shaders to CUDA: graphics chips learn to do anything
6. 2012: AlexNet and two gaming cards in a bedroom
7. Why matrix multiplication made GPUs the AI engine
8. Tensor cores, stacked memory and the AI data centre
9. The state of affairs in 2026
10. Open questions and trade-offs

Part 1 — Painting a frame

Look at any modern video game for one second. In that second your screen has been completely redrawn sixty times or more. A standard 1080p screen has 1,920 × 1,080 = 2,073,600 pixels, so at 60 frames a second the computer must decide the colour of about 124 million pixels every second. At 4K it's about 500 million. And each colour isn't simply looked up: it is calculated from the shape, position, material and lighting of everything in the scene.

The previous piece in this series showed how a CPU works: a fetch–decode–execute loop running through instructions one after another, very fast. This piece is about a different kind of processor, the GPU (graphics processing unit), built for a different kind of job. To see why it exists, start with what drawing a 3D scene involves. It's called rendering, and it runs as an assembly line.

Vertices → Rasterisation → Pixel shading → Screen

1. Vertices. Every object in a game, a car, a face, a tree, is built from triangles, often millions of them. Each triangle has three corners, called vertices, each with an x, y and z position. To move the car, turn the camera or squash the 3D world onto your flat screen, every vertex is multiplied by a small grid of numbers, a matrix. (This is exactly the maths of Vectors and Matrices in the Mathematics section: a matrix is a recipe for moving every point in space at once.)
2. Rasterisation. Once the triangles have screen positions, the hardware works out which pixels each triangle covers, like laying a triangle of tracing paper over graph paper and ticking the squares underneath. It also keeps a "depth buffer" recording how far away the nearest thing drawn at each pixel is, so that a wall correctly hides what's behind it.
3. Pixel shading. For every covered pixel, a small program called a shader calculates the colour: which bit of the texture image lands there, how the light falls on it, whether it's in shadow, whether it's shiny.
4. Screen. The finished colours are written into a block of memory, the frame buffer, which is sent to the display. Then it all starts again for the next frame, about 16 milliseconds later.

Rendering one frame means doing the same small calculation millions of times, once for every corner and every pixel, and then doing it all again 16 milliseconds later.

Here is the crucial property. The colour of the pixel in the top-left corner doesn't depend on the colour of the one next to it. Each vertex can be transformed without waiting for the others. Graphics is millions of small, independent jobs. Computer scientists call this "embarrassingly parallel", and it is the whole reason the GPU looks the way it does.

Part 2 — A few smart workers, or thousands of simple ones

Imagine two decorating firms bidding to paint a wall-sized mosaic of two million tiles, each tile a single colour you are told in advance.

Firm C sends eight master craftsmen. Each is brilliant: fast, versatile, able to handle any surprise, and able to plan around a complicated job. Each paints one tile per second.

Firm G sends 2,048 apprentices. Each is four times slower than a master, can only follow simple instructions, and works in squads that must all do the same thing at the same moment. But there are a lot of them.

Here is the race, worked out (a toy model, but the arithmetic is exact):

When the work splits into millions of independent pieces, the apprentices win by a mile. When every step depends on the last, only one worker can be busy at a time, and the master wins. That second row is Amdahl's law from the previous piece, in a different costume.

That is the real difference between a CPU and a GPU:

• A CPU spends most of its transistors on making a few workers (cores) brilliant at sequential work: large caches so they rarely wait for memory, branch predictors that guess which way the program will go, machinery to reorder instructions on the fly. A laptop chip such as Apple's M1 has eight CPU cores.
• A GPU spends its transistors on numbers of workers. NVIDIA's RTX 5090 gaming card of 2025 lists 21,760 "CUDA cores" and draws up to 575 watts. Those cores are much simpler than CPU cores and are grouped into squads, called warps on NVIDIA chips, of 32 threads that execute the same instruction in lock-step, each on its own data.

A caution on numbers: a "CUDA core" is not comparable to a CPU core. It is closer to one lane of a 32-lane road, with one driver giving directions to all the lanes. If threads in a warp need to do different things (some pixels in shadow, some not), the warp has to do each version in turn, and some lanes sit idle. GPUs are fast only when lots of data needs the same treatment.

A CPU is a few experts who can handle anything. A GPU is an army that is unbeatable as long as everyone is doing the same thing.

Part 3 — Before the GPU

Special graphics hardware is nearly as old as computer graphics. Arcade machines of the 1970s already used dedicated circuits to draw moving sprites; Namco's Galaxian of 1979 had multi-coloured sprites on a coloured background. In 1981 NEC's µPD7220 put a graphics display controller on a single chip for personal computers, and in 1986 Texas Instruments' TMS34010 was a fully programmable graphics processor.

The high end belonged to Silicon Graphics (SGI), founded in 1982 by the Stanford professor Jim Clark with a group of students and staff. Its secret weapon was the Geometry Engine, designed by Clark and Marc Hannah at Stanford: the first chip to do the geometry stage of the 3D pipeline, the matrix maths on vertices, in custom hardware. SGI's first graphics terminal shipped in 1984. Its workstations became the standard tools of Hollywood visual effects; one appears on screen in Jurassic Park (1993). In 1992 SGI opened its graphics programming interface to everyone as OpenGL, which became an industry standard. Then cheaper PC hardware ate its market, and SGI filed for bankruptcy in April 2009.

The PC revolution came from three SGI veterans, Ross Smith, Gary Tarolli and Scott Sellers, who founded 3dfx in August 1994. Their Voodoo Graphics chip reached shops in October 1996 on cards such as Orchid's Righteous 3D. It was an odd beast by modern standards: an add-in card that did only 3D. You kept your ordinary 2D video card and ran a short cable from it into the Voodoo, which took over the screen when a 3D game started. It did one job, rasterising and texturing triangles, and did it so well that PC games suddenly looked smooth and solid. 3dfx also wrote its own programming interface, Glide, for games to talk to the chip directly.

For a couple of years 3dfx was the name in PC gaming. Its Voodoo2 of 1998 could even split the work between two cards, each drawing alternate lines of the screen (an early form of the multi-chip tricks that matter so much in AI today). But it made a run of late and uncompetitive products, game developers moved to Microsoft's Direct3D interface rather than Glide, and in December 2000 its rival NVIDIA bought most of its assets. 3dfx filed for bankruptcy in 2002.

Part 4 — "The world's first GPU"

NVIDIA was founded on 5 April 1993 by Jensen Huang, Chris Malachowsky and Curtis Priem, to bring 3D graphics to games and multimedia. On 31 August 1999 it announced the GeForce 256, and it chose its words carefully. NVIDIA called it "the world's first GPU", and defined a GPU as "a single-chip processor with integrated transform, lighting, triangle setup/clipping, and rendering engines", able to process at least 10 million polygons a second.

The new part was hardware transform and lighting (T&L). Until then, cards like the Voodoo drew triangles, but the CPU had to do the vertex maths: moving, rotating and lighting every corner. The GeForce 256, made by TSMC with about 17 million transistors, did that on the graphics chip too. The whole assembly line from Part 1 was now on one consumer chip. NVIDIA's own corporate timeline today still reads, for 1999: NVIDIA "invents the GPU".

Is the claim true? It depends, as so often with "firsts", on the definition, and in this case the definition was written by the company making the claim.

• The word was older. Sony used "GPU" in 1994 for the graphics chip, designed by Toshiba, in the original PlayStation.
• The idea was older. SGI's Geometry Engine had done transform work in hardware since the early 1980s, on expensive workstations and across multiple chips.
• The benefit took time. At launch, hardware T&L gave big gains only in some games, mostly on PCs with slower CPUs, and the card was expensive.

The fair verdict: the GeForce 256 was a real milestone, the first time a cheap consumer chip did the entire 3D pipeline. "World's first GPU" was also very effective marketing, a definition drawn neatly around one's own product. Both are true, and the term stuck so thoroughly that we now use it for chips that rarely draw a picture at all.

"First" claims are usually definitions in disguise. The trick is to ask who wrote the definition.

Part 5 — From shaders to CUDA

In 1999 the GPU's pipeline was fixed: it could transform and light vertices and texture pixels in the ways its designers had built in, and no others. Game developers wanted more control, and the next step changed everything.

The GeForce 3, released in February 2001, added programmable shaders: the vertex and pixel stages could now run small programs written by the game's developers, within limits. (A version of the chip powered Microsoft's first Xbox.) ATI's Radeon 9700 of 2002 pushed programmability further. A shader is just a short program that takes some numbers in (a position, a texture value, a light direction) and puts numbers out. It doesn't have to be about light at all.

Researchers noticed. If you disguise a maths problem as a graphics problem, storing your data in "textures" and writing your calculation as a "pixel shader", the GPU will happily run your calculation on millions of data points in parallel. By 2001 people were doing matrix multiplication on graphics hardware, and in 2003 two research groups independently showed GPUs beating CPUs at general linear algebra. This was called GPGPU, general-purpose computing on GPUs, and it was painful: you had to be an expert in graphics programming to do maths.

At Stanford, a PhD student called Ian Buck built a language, Brook, that hid the graphics disguise. In 2004 NVIDIA hired him and paired him with John Nickolls, its director of architecture for GPU computing, to build it properly. The result was CUDA. It arrived with NVIDIA's GeForce 8800 generation, launched in November 2006, which was NVIDIA's first "unified" design: instead of separate vertex and pixel units, 128 identical general processors that could be assigned to any task. The first public CUDA software kit followed in February 2007.

CUDA let a programmer write ordinary-looking code, in an extended version of the C language, that runs as thousands of threads at once. To add two lists of a million numbers you write the instruction for one pair, "add element i of list A to element i of list B", and launch a million threads, each with its own i. The hardware schedules them in warps across its cores.

For years CUDA looked like an expensive hobby. Most GPUs were bought by gamers, and scientific users were a niche. But NVIDIA kept investing in it, putting CUDA on every card, building libraries for common maths and supporting universities. When the moment came, an enormous body of software, and of programmers' habits, already ran on NVIDIA chips. Competitors have had better and worse hardware at various times since, but none had that head start. That head start in software is arguably NVIDIA's most durable advantage, more than any one chip.

Part 6 — 2012: AlexNet and two gaming cards in a bedroom

Neural networks, the subject of Artificial Intelligence: Where It Came From and Where It Stands at the start of this series, were an old idea: layers of simple artificial "neurons", each adding up weighted inputs from the layer before, with the weights adjusted by training on examples. By the 2000s they were out of fashion with many researchers, partly because they seemed to need impossible amounts of data and computing power.

Both arrived at once. The data: from 2006 the Stanford professor Fei-Fei Li led ImageNet, which grew to more than 15 million photographs labelled by hand in some 22,000 categories, much of the labelling crowdsourced through Amazon's Mechanical Turk. It was completed in 2009, and from 2010 an annual competition challenged programs to recognise objects in the photos. After two years, progress had become marginal.

The computing: at the University of Toronto, the graduate student Alex Krizhevsky had already written fast CUDA code for "convolutional" networks, a design suited to images. With his fellow student Ilya Sutskever and their adviser Geoffrey Hinton, he built a much bigger network than anyone had trained: about 60 million adjustable weights and 650,000 neurons, in eight layers. According to the Computer History Museum, it was trained on a computer with two NVIDIA graphics cards in Krizhevsky's bedroom at his parents' house.

The 2012 paper gives the details. The network made about 90 passes through 1.2 million training images, which took five to six days on two NVIDIA GTX 580 gaming cards with 3 GB of memory each. The model was too big for one card's memory, so they split it across two, the cards reading from each other's memory directly. In the ImageNet competition of 2012 it got the right answer among its top five guesses for all but 15.3% of test images. The second-best entry, using careful conventional methods, managed 26.2%.

The authors were frank about what had made it possible. They wrote that current GPUs, paired with a highly optimised implementation, were powerful enough to train such large networks, and that their results could be improved "simply by waiting for faster GPUs and bigger datasets to become available." That sentence turned out to be the business plan of the next decade.

Credit, as always, is shared. In 2011 Dan Cireşan and colleagues at the Swiss lab IDSIA had already published a fast GPU implementation of convolutional networks, with record results on several image benchmarks; the AlexNet paper itself cites their work. What AlexNet did was win the field's most visible benchmark by a margin nobody could ignore, and within a few years almost every leading computer-vision system was a neural network trained on GPUs. Hinton later shared the 2018 Turing Award with Yoshua Bengio and Yann LeCun for deep learning, and the 2024 Nobel Prize in Physics with John Hopfield. His own summary of AlexNet, quoted by the Computer History Museum when it released the original code with Google in 2025: "Ilya thought we should do it, Alex made it work, and I got the Nobel Prize."

The breakthrough of modern AI was trained on two gaming cards, and its authors wrote that it would get better simply by waiting for faster ones.

Part 7 — Why matrix multiplication made GPUs the AI engine

Why did hardware built for video games suit neural networks so well? Because underneath, they are the same calculation.

A layer of a neural network takes a list of numbers in (a vector: the pixels of an image, or the encoded meaning of a word) and produces a list of numbers out. Each output neuron multiplies every input by its own weight and adds them up. Written down, that is a matrix multiplication: the input vector times a grid of weights. Then a simple non-linear step, such as replacing negatives with zero, and on to the next layer.

The numbers are large. A layer with 4,096 inputs and 4,096 outputs needs 4,096 × 4,096 = 16.8 million multiply-and-add operations for each input it processes. Process 4,096 inputs at once, as training does in batches, and it becomes a multiplication of two 4,096 × 4,096 matrices: 4,096³ = 68.7 billion multiply-adds. Training repeats that across dozens of layers, millions of examples and many passes.

Now notice the shape of the work:

• Each output number is independent. Entry (i, j) of the answer needs row i of one matrix and column j of the other, and nothing else. All 16.8 million entries of a 4,096 × 4,096 result can be computed at the same time. That is the mosaic from Part 2, the apprentices' perfect job.
• It's the same operation everywhere. Multiply, add, multiply, add, with no surprising branches. That suits warps of 32 lanes in lock-step.
• It reuses data. Each number in the input matrices is used 4,096 times. So once the data is loaded into the chip, there is a great deal of arithmetic to do per byte fetched, which keeps the arithmetic units busy instead of waiting for memory.
• Graphics was already this. Moving a vertex is multiplying it by a 4 × 4 matrix. GPU makers had spent twenty years building chips to do matrix arithmetic as fast as possible for games.

A video game and a neural network look nothing alike. To the chip they are the same thing: enormous numbers of small matrix multiplications, all independent.

Since 2017, when the paper "Attention Is All You Need" introduced it, the dominant AI design has been the transformer (explained in How Large Language Models Actually Work, Technology 2). It makes the fit even tighter: its core operation, "attention", is itself a set of matrix multiplications, and large language models are, computationally, mostly matrix multiplication at colossal scale.

Part 8 — Tensor cores, stacked memory and the AI data centre

Once AI became the biggest customer, GPUs started to change shape around it.

Tensor cores. In May 2017 NVIDIA announced the Tesla V100, built on its Volta design: 21.1 billion transistors on a chip of 815 mm², nearly the size of a postage stamp. Besides 5,120 ordinary arithmetic cores it had 640 tensor cores, special units that each do one thing: take two small 4 × 4 matrices, multiply them, and add a third, D = A × B + C, all in one step. NVIDIA's white paper rated the V100 at 15.7 trillion ordinary 32-bit operations per second, and 125 trillion on the tensor cores. The 68.7-billion-multiply-add example from Part 7 (137 billion operations, counting multiplies and adds separately) would take about 8.8 milliseconds at the first rate and about 1.1 at the second.

The trick includes using lower precision. The tensor cores multiplied 16-bit numbers (accumulating the results in 32-bit), where scientific computing traditionally insists on 64-bit. Neural networks turn out to tolerate rounding surprisingly well, and smaller numbers mean more of them per second and per byte of memory. The industry has kept going down: 8-bit formats, and with NVIDIA's Blackwell generation, 4-bit numbers, which NVIDIA says can double the size of model that fits in memory while keeping accuracy high.

That has a side effect worth knowing when you read headlines. In April 2025 Google said a full pod of 9,216 of its Ironwood AI chips delivers 42.5 exaflops and compared that with the El Capitan supercomputer's 1.7 exaflops. But the Ironwood figure is a peak rate for 8-bit numbers, while El Capitan's is measured running a demanding benchmark in 64-bit arithmetic. Both are "flops", but they are not the same unit of work.

The memory wall. A chip that can do a hundred trillion multiplications a second is useless if it can't be fed numbers fast enough. The answer is high-bandwidth memory (HBM): memory chips stacked on top of each other like a multistorey car park, connected by thousands of vertical wires through the silicon itself, and placed right next to the processor on a shared base layer. Each stack talks to the processor over 1,024 wires at once (2,048 in the newest standard, HBM4). HBM was developed by AMD and SK hynix, standardised in 2013 and first used in AMD's Radeon R9 Fury X in June 2015. The V100 had four stacks giving 900 gigabytes per second. An NVIDIA DGX B200 server, with eight Blackwell GPUs, has 1,440 GB of HBM and 64 terabytes per second. Only three companies make HBM: SK hynix, Samsung and Micron.

Bigger than a chip. The largest AI models don't fit on one GPU, or even one server, so the unit of computing has grown. NVIDIA's Blackwell GPU is two chips joined by a 10-terabyte-per-second link acting as one, 208 billion transistors in all. Its GB200 NVL72 rack wires 72 of those GPUs and 36 CPUs into one liquid-cooled system with 13.4 TB of HBM. Data centres then link thousands of such racks.

Power. All of this runs hot. The eight-GPU DGX B200 server is rated at up to about 14.3 kilowatts, roughly 1.8 kW per GPU including its share of the server: the whole box draws about as much as half a dozen electric kettles running non-stop. The densest racks, such as the GB200 NVL72, are liquid-cooled.

At national scale: a December 2024 report by Lawrence Berkeley National Laboratory for the US Department of Energy found that US data centres used 176 terawatt-hours of electricity in 2023, 4.4% of the country's total, three times their use in 2014. It projected 6.7% to 12% by 2028. Globally, the International Energy Agency estimated in 2025 that data centres used about 415 TWh in 2024, around 1.5% of the world's electricity, and could reach about 945 TWh by 2030. (These are estimates with wide ranges; the honest answer to "how much will AI use?" is that it depends on choices not yet made.)

Part 9 — The state of affairs in 2026

Company figures below come from their own results releases; dates are given because these numbers move fast.

NVIDIA. In its 2026 financial year (to 25 January 2026), NVIDIA's revenue was $215.9 billion, of which $193.7 billion came from data centres. In the three months to 26 July 2026 it reported revenue of $96.2 billion, more than double a year earlier, with data centres $89.0 billion, about 92% of the total. The company that sold graphics cards to gamers is now overwhelmingly an AI-infrastructure company. It said its next platform, Vera Rubin, was "now in full production." Estimates of its share of the market for AI accelerators commonly exceed 80%. Its chief executive's summary in August 2026: "AI has reached its inflection point. It's doing useful work. Its tokens are productive and profitable." Its next results are due on 17 November 2026.

AMD. NVIDIA's long-time graphics rival, which bought the graphics-chip maker ATI in 2006, reported revenue of $11.5 billion for the quarter to 27 June 2026, with its data-centre business more than doubling to $6.7 billion. It launched its Instinct MI400 series and Helios rack systems, which it says partners including Anthropic, Meta, Microsoft, OpenAI and Oracle are deploying. Its gaming revenue, by contrast, fell 31% to $779 million. AMD chips also power two of the world's three fastest supercomputers, El Capitan and Frontier.

Custom chips. The biggest buyers increasingly design their own. Google has used its TPUs (tensor processing units) since 2015; its seventh generation, Ironwood, was unveiled in April 2025, and on 27 October 2026 Google described an eighth generation rated at up to 12.6 petaflops in 4-bit arithmetic per chip and announced that Oracle's cloud would offer TPUs from 2027, the first time another major cloud company has done so. Amazon offers its own Trainium chips; the current Trainium3 has 144 GB of HBM per chip. Others take radically different approaches: Cerebras builds a single chip the size of a whole wafer, 46,225 mm² with 4 trillion transistors. And NVIDIA itself agreed in December 2025, according to CNBC, to pay about $20 billion for assets and a technology licence from the inference-chip start-up Groq.

Supply. NVIDIA's Blackwell is made by TSMC in Taiwan, and AI chips in general depend on the handful of leading-edge factories described in the previous piece and on HBM from three suppliers. The demand has rippled through the whole chip industry: the industry statisticians WSTS forecast in spring 2026 that memory sales would more than triple this year.

Geopolitics. The US restricted exports of NVIDIA's top data-centre chips to China in October 2022, and NVIDIA responded with cut-down versions for the Chinese market. The rules have changed repeatedly since. NVIDIA recorded $4.5 billion of charges related to its China-specific H20 chip in the quarter to April 2025. In August 2025, it was reported that NVIDIA and AMD had agreed to pay the US government 15% of revenue from certain China sales in exchange for licences; in September 2025, China's internet regulator reportedly told its tech companies not to buy one of NVIDIA's China-market chips. After a presidential announcement in December 2025, the US Commerce Department said on 13 January 2026 that it would consider licences for NVIDIA's H200 and AMD's MI325X case by case, with conditions. Even so, NVIDIA's outlooks in February and August 2026 assumed no data-centre compute revenue from China at all. In June 2026 the world's fastest supercomputer, China's LineShine, reached the top using only home-grown CPUs and no GPUs.

The two cases. The boosters' case: the demand is real and paid for, in record revenues; the systems are doing useful work in code, science and services; and each generation does more per watt (Google says Ironwood roughly doubled performance per watt over its predecessor). The critics' case: spending on AI data centres is running far ahead of proven profits from AI services, chips lose value quickly as new generations arrive, and the electricity, water and grid demands land on communities that didn't choose them. Some see a bubble; others see the early years of an infrastructure build-out like railways or electricity. That argument is unresolved, and the evidence that would settle it, years of returns on today's investment, doesn't exist yet.

Part 10 — Open questions and trade-offs

Can anyone loosen NVIDIA's grip? Its hardware lead is real but not permanent; its software lead, two decades of CUDA, is harder to erode. Custom chips from the cloud companies, AMD's open-source ROCm and AI frameworks that can target many kinds of chip are all bets that the lock-in will weaken. If it does, prices could fall; if it doesn't, one company remains the bottleneck for much of the world's AI.

Is the GPU even the right shape? GPUs became the AI engine by accident: they were the parallel hardware that happened to exist. Purpose-built chips (TPUs, Trainium, wafer-scale engines) strip out the graphics heritage. Yet GPUs keep absorbing the specialist features, from tensor cores to 4-bit arithmetic. Whether general flexibility or narrow efficiency wins may depend on how quickly AI methods keep changing.

Memory, not maths. For many AI tasks, especially answering questions with a trained model, the limit is how fast numbers can be moved, not how fast they can be multiplied. HBM is expensive and made by three companies. Advances in memory and packaging may matter as much as advances in the processor itself.

Energy. More efficient chips don't automatically mean less electricity. When a resource gets cheaper to use, people often use far more of it (economists call this the Jevons paradox). Whether AI's power demand flattens or keeps climbing depends on efficiency, on prices and on what society decides the computing is worth.

Chips and nations. Export controls, subsidies and supply chains now shape where AI can be built. Whether that slows rivals or accelerates their independence is debated, as the previous piece describes.

And the gamers? The people who paid for twenty years of GPU development are now a small part of the market. AMD's gaming sales fell while its data-centre sales doubled; NVIDIA no longer reports gaming as a separate line in its headline results. The chip designed to paint a game's pixels now spends most of its life multiplying matrices for AI, and it is still, at heart, the same army of simple workers.

The takeaway

Rendering a frame means doing small, independent calculations, matrix maths on millions of vertices and a colour for each of millions of pixels, sixty times a second. GPUs grew up to do exactly that with thousands of simple workers instead of a few smart ones. SGI pioneered 3D hardware, 3dfx brought it to PCs in 1996, and NVIDIA's GeForce 256 put the whole pipeline on one consumer chip in 1999 (its "first GPU" claim rests on its own definition). Programmable shaders (2001) and CUDA (2006–07) let GPUs run any parallel maths. In 2012 AlexNet, trained on two gaming cards, showed that neural networks plus GPUs beat everything else, because neural networks are mostly matrix multiplication, which is what GPUs do best. Since then tensor cores, low-precision numbers and stacked HBM memory have reshaped GPUs around AI. In 2026 data centres provide over 90% of NVIDIA's revenue, cloud companies build their own chips, and power use and export controls are live political questions.

Try this: the next time you play a game or watch an animated film, pause on one frame and estimate the work. Count the pixels (an iPhone 16 screen, at 2,556 × 1,179, has about 3 million), multiply by the frames per second, and remember that every one of those colours was calculated, not stored. Then remember that the same chip, doing the same kind of sum, is what trains the AI models you use.

Sources & Further Reading

Sources
- Company results: NVIDIA, fourth-quarter and fiscal 2026 results (25 February 2026) and second-quarter fiscal 2027 results (26 August 2026), including the H20 charge and China outlook; AMD, second-quarter 2026 results (4 August 2026); TSMC as cited in the previous piece.
- NVIDIA product pages (accessed October 2026): RTX 5090, Blackwell architecture, DGX B200, GB200 NVL72, Vera Rubin; NVIDIA Volta Architecture white paper (V100 transistors, tensor cores, 125 TFLOPS, 900 GB/s HBM2); NVIDIA corporate timeline (founding, "invents the GPU").
- Google: "Ironwood: The first Google TPU for the age of inference" (April 2025) and Cloud TPU documentation (TPU7x specifications); "Bringing Google TPUs to Oracle Cloud Infrastructure" (27 October 2026). Amazon Web Services, Trainium page. Cerebras, chip page.
- Krizhevsky, Sutskever and Hinton, "ImageNet Classification with Deep Convolutional Neural Networks", NeurIPS 2012 (architecture, two GTX 580 GPUs, five to six days, 15.3% vs 26.2%, "waiting for faster GPUs" quotation). Computer History Museum, "CHM Releases AlexNet Source Code" (20 March 2025): the bedroom, ImageNet history, CUDA's 2007 release, Hinton's quotation.
- Computer History Museum, Revolution exhibit, "Silicon Graphics" (founding, Geometry Engine, 1984 terminal).
- US Department of Energy, "DOE Releases New Report Evaluating Increase in Electricity Demand from Data Centers" (20 December 2024), on the Lawrence Berkeley National Laboratory report. IEA Energy and AI (10 April 2025), figures as reported by S&P Global Commodity Insights, "Global data center power demand to double by 2030 on AI surge: IEA" (10 April 2025) (the IEA site could not be reached to check directly).
- US Bureau of Industry and Security, press release of 13 January 2026 (H200 and MI325X licence review). TOP500, June 2026 list. WSTS spring 2026 forecast.
- Wikipedia, used for leads and secondary details: "Graphics processing unit", "3dfx Interactive", "Silicon Graphics", "GeForce 256", "GeForce 3 series", "GeForce 8 series", "CUDA", "General-purpose computing on graphics processing units", "Single instruction, multiple threads", "High Bandwidth Memory", "Tensor Processing Unit", "Volta (microarchitecture)", "Geoffrey Hinton", "Turing Award", and "Nvidia" (market-share estimate, the August 2025 revenue-share arrangement, China's September 2025 instruction, and the Groq deal as reported by CNBC).
- Computed in Python: 1,920 × 1,080 × 60 = 124.4 million pixels per second; the race model (2,073,600 ÷ 8 × 1 = 259,200 versus ⌈2,073,600 ÷ 2,048⌉ × 4 = 4,052); 4,096² = 16.8 million and 4,096³ = 68.7 billion multiply-adds; 137.4 GFLOP ÷ 125 TFLOPS = 1.1 ms and ÷ 15.7 TFLOPS = 8.8 ms; 14.3 kW ÷ 8 = 1.8 kW; 89.0 ÷ 96.2 = 92.5%; 9,216 × 4,614 TFLOPS = 42.5 exaflops.

Go deeper
- 📘 The Thinking Machine — Stephen Witt (2025). A history of Jensen Huang and NVIDIA from 3D graphics to AI, including the CUDA bet and AlexNet; FT Business Book of the Year 2025.
- 📘 Chip War — Chris Miller (Scribner, 2022). The manufacturing and geopolitical background: why nearly every AI chip passes through Taiwan.
- 🌐 CHM: "CHM Releases AlexNet Source Code" — the story of the 2012 breakthrough, with the original code.
- 🌐 The AlexNet paper (NeurIPS 2012) — nine pages, surprisingly readable; section 3.2 explains the two-GPU trick.
- 🌐 TOP500 — the twice-yearly ranking of the world's fastest supercomputers, a good way to watch GPUs take over scientific computing.
- 🌐 Companion pieces in this library: Vectors and Matrices (Mathematics) for the maths GPUs do, Processors: How a CPU Thinks (Technology) for the other kind of chip, Artificial Intelligence: Where It Came From and Where It Stands and How Large Language Models Actually Work (Technology) for what the chips are used for, Alan Turing: The Machine That Defined Computing (History-Makers), and What It's Like to Be a Data Scientist (Careers) for the people who use these machines every day.