Agile, Scrum & Kanban: Working in Small Loops, and Knowing When Not To

Stand behind the pass in a busy restaurant kitchen. Order slips hang on a rail. A cook takes the next slip, works it, and sends the plate out. Nobody holds a Gantt chart of the evening. If the grill backs up, you can see it at once, because slips pile up in front of the grill and not elsewhere. If a table changes its mind, the slip changes. The kitchen is not chaos: it runs on a few firm rules (one slip at a time per cook, plates go out hot, the head chef says what is on the menu), and inside those rules it adapts every minute.

That picture is close to what the software world called "agile" and what Toyota's factories called kanban, a Japanese word for a signboard or card. A slip on a rail is a signal: there is room, start the next thing. This piece is about that family of ideas: what the Agile Manifesto said, how Scrum and Kanban turn it into routines, the queue arithmetic that explains why they work, and the honest limits. If you manage projects, it is the other half of the picture in Project Management Foundations (36/90). If you build products, it is how the team you work with plans its weeks.

The Manifesto: four values and twelve principles

In February 2001, seventeen software practitioners met at the Snowbird ski resort in Utah (11 to 13 February). They had been using different lightweight methods, among them Extreme Programming, Scrum, Crystal and Dynamic Systems Development Method, and they were tired of being told that software needed heavy, document-first processes. They did not agree on a method. They agreed on a short statement of what they valued, and they chose the word "agile". The signatories included Kent Beck, Martin Fowler, Ken Schwaber, Jeff Sutherland, Alistair Cockburn, Ron Jeffries and Dave Thomas.

The four values are written as "this over that", and the manifesto adds that the items on the right still have value; the ones on the left are valued more:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

Twelve principles follow. Read them as a set of consequences: deliver working software frequently, "from a couple of weeks to a couple of months, with a preference to the shorter timescale"; welcome changing requirements, even late; business people and developers work together daily; working software is the primary measure of progress; keep a constant, sustainable pace; and "simplicity, the art of maximizing the amount of work not done, is essential". The last one is the one product people tend to love: the best feature is often the one you did not build.

Notice what the manifesto is not. It is not anti-planning (the fourth value says "over", not "instead of"), it contains no word about stand-up meetings or story points, and it was written about software. Taking it to a bridge or a hospital ward needs the second half of this piece. Where project managers and agilists have clashed, the clash has often been over exactly that: whether a plan is a tool for learning or a promise to be kept.

Scrum: a heartbeat of sprints

Scrum is the most widely used agile framework. Its roots are older than the manifesto: in 1986 Hirotaka Takeuchi and Ikujiro Nonaka published "The New New Product Development Game" in the Harvard Business Review, describing Japanese and American firms whose product teams worked like a rugby side, passing the ball back and forth, instead of handing work down a relay. Ken Schwaber and Jeff Sutherland, working separately in the early 1990s, built the idea into a software method and presented Scrum together at the OOPSLA conference in 1995. They still maintain The Scrum Guide; the current version is from November 2020, and it is short (a few pages).

The 2020 Guide describes a Scrum Team, "typically 10 or fewer people", with three accountabilities and no hierarchy inside it. The Developers are the people doing the work (the word covers anyone building the product, not only programmers). The Product Owner is accountable for maximising the value of the product and for ordering the Product Backlog. The Scrum Master is accountable for the team following Scrum and for removing obstacles to its effectiveness. For product readers: a Product Owner is a Scrum role, and a product manager is a job, and the two overlap but are not the same (36/95 untangles them).

Everything happens inside the Sprint, a fixed length of one month or less, during which no change is allowed that would endanger the Sprint Goal. Inside it sit four more events: Sprint Planning (what can be done and how), the Daily Scrum (15 minutes, for the Developers to plan the next day's work), the Sprint Review (inspect the result with stakeholders) and the Sprint Retrospective (inspect the way of working and change something). Three artefacts each carry a commitment: the Product Backlog (commitment: the Product Goal), the Sprint Backlog (the Sprint Goal) and the Increment, which is only an Increment if it meets the Definition of Done, "a formal description of the state of the Increment". That last piece is the quiet one: a shared, written meaning of "finished" is what stops "done except for testing" from piling up.

Teams often count work in story points, a relative size, and track the sum finished per sprint as velocity. The Guide does not mention points or velocity; they are common practice, and people argue about them (36/91 covers the estimating debate). What matters here is the chart most teams draw next to the board. Below is an invented ten-day sprint with 40 points of work in the first plan, and a request that arrives partway through.

Without JavaScript: with 8 points added on day 5 the scope becomes 48. After day 10, 36 points are done, so 12 remain. Without the addition, 4 would remain. A burndown shows the extra 8 only as a flatter slope; a burnup shows the scope line stepping up.

Switch between the two views and the difference is plain. A burndown plots remaining work and slopes toward zero; scope added mid-sprint hides inside the slope, so the team seems to slow down. A burnup plots work done and total scope as two lines, so scope creep is a visible step in one of them. Burndown is simpler to read at a glance; burnup tells the truth about change. Neither tells you whether what you built is useful. That is why the Sprint Review matters more than the chart.

Kanban and flow: Little's law

Kanban starts from a different move. Instead of fixed-length iterations, it makes the work visible on a board and puts a limit on work in progress (WIP). When a column is full, nobody starts new work there; they help finish something. Toyota's pull system, signalled by kanban cards, grew from the late 1940s as a way to make factories produce only what downstream stations had used. David J. Anderson adapted the idea to software work from 2004 (a project at Microsoft), described it in his 2010 book Kanban, and the practice is now written down in The Kanban Guide (first by Coleman and Vacanti in 2020, updated 2025): define the workflow, actively manage the items in it, and improve the workflow. Many teams combine it with Scrum; the older nickname is "Scrumban".

Why would a limit speed anything up? Because of a law about queues proved by John Little in 1961. In any stable system, the average number of items inside equals the rate at which they arrive (and, over time, leave) times the average time each spends inside:

It holds whatever the arrival pattern or the order of work, as long as the system is stable. So if a team's throughput is fixed by its slowest step, then adding more work in progress cannot raise throughput; it can only raise lead time. Forty things "in flight" at 0.45 finished a day means each takes about ninety days to emerge, however busy everyone feels. Try it. The simulator has a team of two developers feeding one tester, with randomly sized (but seeded, so repeatable) pieces of work, 80 items, and a limit on how many may be in progress at once.

Without JavaScript (computed from this model): WIP limit 1 gives 0.28 items a day and a 3.6-day lead time; limit 4 gives 0.45 a day and 8.6 days; limit 12 gives 0.45 a day and 24.6 days; no limit (all 80 start at once) gives 0.45 a day and 89 days.

The chart is a cumulative flow diagram: the upper line counts items started, the lower counts items finished, and the gap between them is work in progress. A widening gap is a queue building up, usually in front of a bottleneck. Watch what happens as you raise the limit: throughput rises quickly and then stops, while lead time keeps climbing. The sweet spot is the lowest limit that keeps the bottleneck busy. The readout also checks the law: the two sides come out equal, because in a run that starts and ends empty the identity is exact. Real, running systems only approximate it.

There is a human reading as well. A short lead time means feedback arrives while people still remember the work, a stuck item is noticed within days, and a customer who changes their mind has less to throw away. Critics point out that WIP limits are no use if the work is not really divisible, and that the board can become a status ritual. Fair, and the fix is the same: look at where the queue is.

Lean roots, Extreme Programming and the engineering underneath

Agile did not invent its ideas. Lean is the name Western researchers gave to Toyota's production system (the 1990 book The Machine That Changed the World by James Womack, Daniel Jones and Daniel Roos popularised it): remove waste, stop the line to fix defects at the source, produce to demand, and improve continuously. Mary and Tom Poppendieck carried it into software in 2003. Extreme Programming (XP) came from Kent Beck's work on Chrysler's C3 payroll project from 1996; his book Extreme Programming Explained appeared in 1999. Its practices include test-driven development, pair programming, continuous integration, refactoring, small releases, simple design and a sustainable pace.

This matters because Scrum alone says almost nothing about how to build software well. Without automated tests and frequent integration, short sprints just mean shorter piles of unfinished work, and many failed adoptions are exactly that. The engineering practices that make small loops safe are the subject of Shipping Software (36/76).

The same loop turns up outside software. In construction, Glenn Ballard and Greg Howell developed the Last Planner System in the 1990s: crews and foremen meet weekly, look ahead at the next several weeks, clear what blocks the work, and promise only what they are ready to do, then measure how many promises were kept (percent plan complete). It is pull planning on a building site, and it sits inside an ordinary plan-driven contract.

Scaling frameworks, and the cargo cult

One team of ten is easy to describe. Fifty teams working on one product are not, and so frameworks arrived: the Scaled Agile Framework (SAFe; Dean Leffingwell, first released 2011, version 6.0 in 2023), LeSS (Large-Scale Scrum, by Craig Larman and Bas Vodde), and Scrum@Scale (from Jeff Sutherland). Spotify's 2012 description of its squads and tribes is often copied as "the Spotify model", although the people who wrote it described it as a snapshot of one company at one time.

The arguments are real and worth stating fairly. Supporters say large organisations need coordination and a common language, and that a framework gets reluctant executives to start. Critics, among them Ken Schwaber (who called it "unSAFe" in 2013) and some other manifesto signatories, say SAFe is too hierarchical and puts the plan above the team's decisions; its defenders reply that most big companies were never going to adopt something lighter. The honest summary is that no framework has beaten the others in a clean comparison, and that coordinating many teams is mostly about architecture and dependencies, not meeting schedules.

The second criticism is closer to home. Ron Jeffries, a manifesto signatory, described Dark Scrum in 2016: it begins "when people who know their old job, but not their new Scrum job, begin to do the Scrum activities". Managers keep directing the work, and the Daily Scrum becomes a status report to the boss, the sprint becomes a deadline, and velocity becomes a target. People sometimes call this cargo-cult agile: the ceremonies without the learning. Its signs are easy to spot. Retrospectives that never change anything. A Definition of Done that nobody wrote down. Product Owners who relay requests instead of deciding. A sprint that is always slightly too long to finish.

Product writers add a version of their own. Marty Cagan (Inspired, 2017; Empowered, 2020) argues that many teams run perfect sprints on a list of features somebody else chose, which he calls feature teams, and that good product teams are instead given problems to solve and held to outcomes. Whether you agree, the useful test is the same: is the team learning something every sprint that changes what it does next?

Should this be agile? Fit, not fashion

So when does it fit? Ralph Stacey's 1996 matrix, adapted by many agile trainers, sorts problems by how certain the answer is and how much people agree on it: near the certain, agreed corner, you plan and execute; far from it, you probe and adapt. Barry Boehm and Richard Turner's 2003 book Balancing Agility and Discipline is more specific, scoring a project on five factors: team size, how critical failure is, how experienced the people are, how fast requirements change, and whether the culture thrives on order or on freedom. Projects on the agile side of those five suit light, adaptive methods; projects on the other side suit more plan and documentation; most live in between.

The mix is common. A hospital builds a new wing with a fixed design, permits, steel orders and safety inspections, and plans those with critical paths (36/91). The fit-out of the ward inside, or the software that schedules the operating theatres, keeps changing as staff see it. Aerospace and pharma run plan-driven stage gates for good reasons: regulators demand traceability, and a flight control fault cannot be patched next sprint. Teams in those fields often use agile inside a gate, in short loops on prototypes, simulations and software, and plan the parts that cannot be sliced. Nobody should be sneered at for choosing either. The wrong choice is using a method because it is fashionable.

Without JavaScript: a mobile app prototype with shifting requirements, low cost of error and open customers scores about 4.7 of 5 (agile). A bridge with fixed requirements, a high cost of error and unsliceable work scores about 1.7 (plan-driven). Medical device firmware lands in between (about 2.9, hybrid).

Template: a team working agreement and Definition of Done

The cheapest agile tool is a page the whole team wrote and can point at. A working agreement says how you work together; a Definition of Done says what "finished" means. Fill in the fields, edit freely, and copy the text. Nothing is stored: your answers live only in this page until you close it.

What a graduate just sees

• Agile is a bet about uncertainty: short loops are worth it when you cannot know the answer without trying it.
• The Manifesto values the left side more; it never said the right side is worthless, and it never said no planning.
• A Sprint is a fixed container with a goal; the Definition of Done is what stops half-finished work counting as progress.
• Throughput is set by the bottleneck, so more work in progress mostly means longer waits: L = λW.
• A burnup shows scope change; a burndown hides it in the slope.
• Ceremonies without the learning are Dark Scrum; ask what the team changed after the last retrospective.
• Fit decides the method: criticality, change rate, sliceability and trust, not fashion; many real projects are hybrids.

Where it shows up next

• Risk, Cost & Project Control (36/93): the same uncertainty, measured with reserves and earned value, and the reason many agile teams use burnup and forecast ranges.
• Product Strategy, Roadmaps & Prioritization (36/97): what goes at the top of the backlog and why a roadmap should not be a promise.
• Shipping Software (36/76): continuous integration, tests and small releases are what make short loops safe.
• Operations Management (34/52): queues, bottlenecks and variability, the theory under Kanban.
• In the world: a design studio's weekly critique, a hospital's daily bed huddle, a crew's weekly look-ahead on a building site, a city's digital service releasing in small steps.

If you want to go deeper

• 📘 Textbooks. Kenneth Rubin, Essential Scrum (Addison-Wesley, 2012), the standard practical text; and David J. Anderson, Kanban: Successful Evolutionary Change for Your Technology Business (Blue Hole Press, 2010).
• 📘 Gentler. Henrik Kniberg, Scrum and XP from the Trenches (2nd edition, 2015), a short, free account of one team's practice; and Kent Beck with Cynthia Andres, Extreme Programming Explained (2nd edition, 2004).
• 🌐 Free reading. The Scrum Guide (2020) and the Agile Manifesto with its twelve principles.
• 🌐 Free course. MIT OpenCourseWare 1.040, Project Management (Spring 2009), for the plan-driven counterweight; and The Kanban Guide.