Signals & Systems: Every Signal Is a Recipe of Sines

Clap once in an empty stairwell. What you hear afterwards is not the clap: it is the stairwell. The echo, the ring, the way the sound dies away over a second or two, all of that is a fingerprint of the room, and if you recorded it you could, in principle, make any voice sound as if it were speaking from inside that stairwell. Recording studios do exactly this with a recorded "impulse response" of a cathedral.

Now watch a car's wheel in a film as it speeds up. At some speed the spokes seem to slow, stop, and turn backwards. The wheel has not changed. The camera takes only 24 snapshots a second, and a fast wheel looks, in 24 snapshots, exactly like a slow one. The same thing, called aliasing, decides how fast your phone must sample a sound and why the music on a CD tops out at 20 kHz.

A signals-and-systems graduate sees both at once. A signal is a quantity that changes with time (a voltage, a pressure, a pixel row). A system is anything that turns one signal into another (a room, an amplifier, a filter, a bridge, a phone's microphone). The course gives you a very small number of ideas that handle all of them.

A signal is a function; a system is a machine for functions

A sound is air pressure at your ear as a function of time. A photograph is brightness as a function of position. A signal is any such function, and a system takes one in and gives one out: x(t) goes in, y(t) comes out. An amplifier, a guitar pedal, a spring and a mass, a cheap phone microphone are all systems.

Most of engineering rests on a lucky class of systems: linear and time-invariant, "LTI". Linear means that doubling the input doubles the output, and the output for two signals added together is the two outputs added. Time-invariant means that if you play the input a second later, you get the same output a second later. Springs, resistors, capacitors, air in a room and most electronics at small signal sizes behave like this, and for them the whole behaviour compresses into something astonishingly small.

That h is the impulse response: what comes out when you put in one instant of signal (a click). Everything else follows from the two properties above. Any input is a long row of clicks of different sizes at different times. The system answers each click with a scaled, shifted copy of h, and by linearity the output is all those copies added up.

Convolution: an echo you can build

The sum of shifted, scaled copies has a name, convolution, and in discrete form it is only this:

The picture to keep is the echo. A hand clap in a canyon has h = "the clap, then a quieter clap 1.2 seconds later". Shout a word and the output is the word plus a quieter copy of the word 1.2 s later: convolution did nothing mysterious, it only laid down one copy of the input for each spike in h. A room with many reflections has many spikes, and the sound blurs into reverb. A moving-average smoother has a box for h, and it blurs the signal for the same reason.

A worked number: the input below is eight samples. The echo filter has h[0] = 1 and h[d] = g. With d = 10 and g = 0.5, the output is 18 samples long (8 + 11 − 1), the first eight are the input untouched, and starting at sample 10 there is a half-size copy. The tallest part of the copy is 0.5 × 1.0 = 0.5, at sample 12.

Fourier: every signal is a sum of sines

Convolution is easy to understand and tedious to do. The second big idea is a change of viewpoint that turns it into multiplication. A pure sine wave has a remarkable property: put it through any LTI system and a sine of the same frequency comes out, only scaled and delayed. Never a different frequency, never a different shape. Sines are the system's natural shapes, like the pure modes of a guitar string.

Now add Fourier's claim (1807, for heat flow in a solid): nearly any repeating signal, however jagged, is a sum of sines at multiples of its base frequency, each with its own strength and its own delay. So to know what a system does to any signal, break the signal into sines, scale and delay each by what the system does at that frequency, and add them back. The "what the system does at each frequency" is its frequency response, written H(f).

A square wave needs sines at 1, 3, 5, ... times its base frequency with strengths 1, 1/3, 1/5, ... The first sine alone already captures 81% of the square wave's power; the first three capture 93%. But the sharp corners live in the high harmonics: you never fully get them, and a small overshoot of about 9% of the jump sits next to each corner however many sines you add. That is the Gibbs phenomenon, and it is why a sharply filtered signal "rings".

The interactive goes one step past the one in The Fourier Transform (12/12): it also lets you scramble the phases. The strengths (the spectrum you see in a plot) stay identical, but the shape changes. The ear mostly cannot tell, because it hears the strengths; a communications engineer can, because the shape is what a receiver has to detect.

Frequency response and filters

Once you think in frequencies, a filter is just a gain that depends on frequency. Treble and bass knobs are filters. So is the way a thick wall lets a bass note through and stops speech. The simplest electronic filter, a resistor followed by a capacitor to ground, passes slow signals and blocks fast ones (a low-pass). Swap the two parts and you get a high-pass. Its gain has an exact form:

At the cutoff frequency f_c the gain is 1/√2 = 0.707, which is 3 dB lower (a halving of power). Ten times above it, the gain is 0.0995, 20 dB down: every factor of 10 in frequency costs a further factor of 10 in amplitude. Engineers say "20 dB per decade" for this one-part filter and "40 dB per decade" for two parts in a row, and they plot it with logarithmic axes because their ears and their problems both work in ratios. A worked case: a 160 Ω resistor and 1 µF capacitor give f_c = 1/(2π · 160 · 10⁻⁶) = 995 Hz.

The same transform of the impulse response gives the frequency response, so the echo from earlier and this filter are one object seen two ways. An echo of delay d has a response that is a series of peaks and notches (a "comb"), which is why a hall can make a note boom and its neighbour vanish.

Sampling and Nyquist: why the wagon wheel runs backwards

A computer stores a sound as a list of measurements taken at a fixed rate. The sampling theorem says this loses nothing, provided you sample at more than twice the highest frequency present. The idea has several fathers: Nyquist's 1928 work on telegraph rates, Kotelnikov in 1933, Shannon's proof in 1949. Hence CD audio at 44,100 samples per second: human hearing tops out near 20 kHz, and twice that is 40 kHz.

Break the rule and the damage is not a blur but a lie. A frequency above half the sample rate shows up as a lower one, an alias:

Take the film: f_s = 24 frames per second. A one-spoked wheel making 23 turns a second looks like 23 − 24 = −1 turn a second: one slow turn backwards. The same arithmetic makes a helicopter's rotor appear to stand still on video, a ripple pattern appear on a finely striped shirt, and a high-pitched whine alias into the audio band of a badly designed recorder. The cure is to filter out everything above f_s/2 before sampling, an "anti-alias filter", because afterwards the damage cannot be undone.

The FFT and the spectrogram

Finding the sines in a list of N samples directly means N sums of N terms, so about N² multiplications. In 1965 James Cooley and John Tukey published the fast Fourier transform, which does it in about N log₂N steps (Gauss had used the same trick around 1805 on asteroid orbits, but never published it). For a million samples, direct evaluation needs about 1.1 × 10¹² steps and the FFT about 2.1 × 10⁷: roughly 50,000 times fewer. That is why a phone can run a spectrum analyser in real time, and why Wi-Fi and 4G, which use the FFT inside every transmission, are possible at all.

One spectrum describes the whole recording at once, which hides when things happen. A spectrogram chops the signal into short windows and takes the spectrum of each, drawing time across, frequency up, loudness as colour. It is how speech scientists see vowels, how bird-song apps name a bird, and how Shazam-style apps match a tune. The catch is a trade-off built into the maths: a short window tells you when but blurs which frequency; a long window sharpens the frequency but smears the time. Frequency resolution is f_s/N, window length N/f_s, and you can never improve both.

Laplace and z, in words

The Fourier view uses only steady sines. Real systems also grow and decay: a plucked string dies away, a microphone near a speaker squeals louder and louder. The Laplace transform (for continuous systems) and the z-transform (for sampled ones) widen the sines to decaying or growing ones, and in return every LTI system collapses into a handful of numbers called poles.

• A pole in the left half of the Laplace plane is a decay: the system forgets a disturbance. Farther left means faster. A pole on the vertical axis rings forever, and a pole in the right half is growth: instability.
• A pole that has an imaginary part also oscillates. A mass on a spring with ω_n = 10 rad/s and damping ratio ζ = 0.2 (see Vibrations, 34/17) has poles at −2 ± 9.8 j: decaying at rate 2 per second, ringing at 9.8 rad/s.
• For sampled systems the same story is told on a circle: poles inside the unit circle are stable. A digital echo y[n] = x[n] + g·y[n−d] has all its poles of size g^1/d, so for g = 0.9 it fades, for g = 1 it never dies, and for g above 1 it grows without limit: the squeal of a guitar amp.

The whole of Control Systems (34/20) is this idea turned into design: place the poles where you want them, and the machine behaves.

What a graduate just sees

• A system's whole personality is one signal, its reply to a click; every output is a pile of shifted, scaled copies of it.
• Convolution in time is multiplication in frequency, so filters are just gain versus frequency.
• Sines are the natural shapes of linear systems: they come out as sines, only scaled and shifted.
• A sharp edge needs high frequencies; a signal that is brief in time is wide in frequency, and you can't have both narrow.
• Anything above half the sample rate is not lost but disguised as something lower, so filter first, then sample.
• An FFT is not a trick for a few cases: it is the reason frequency thinking is cheap, which is why it is inside audio, radio, MRI and image codecs.
• Left-half-plane poles fade, right-half-plane poles blow up; feedback with too much gain turns a decay into a squeal.

Where it shows up next

• Communication Systems (34/40): a radio station is a message shifted in frequency, a receiver is a filter, and every bandwidth limit is a statement about a spectrum.
• Control Systems (34/20): the Laplace half of this subject, with feedback added to move the poles.
• Embedded Systems (34/41): the anti-alias filter, the sample rate and the digital filter in firmware are all this subject on a chip with a tight budget.
• Machine Learning (34/32): a convolutional layer is literally a convolution with learned h, and speech models read spectrograms.
• The real world: noise-cancelling headphones, MP3 and JPEG, MRI scanners, seismology, radar, the equaliser in your music app, the vibration monitor on a wind turbine.

If you want to go deeper

• 📘 Textbook: Alan V. Oppenheim and Alan S. Willsky (with S. Hamid Nawab), Signals and Systems, 2nd ed. (Pearson, 1997): the standard, used for decades; clear on convolution, Fourier, Laplace and z.
• 📘 Textbook: B. P. Lathi and Roger Green, Linear Systems and Signals, 3rd ed. (Oxford University Press, 2017): more worked examples and a friendlier pace; Oppenheim and Schafer's Discrete-Time Signal Processing is the next step for sampled signals.
• 📘 Gentler: Steven W. Smith, The Scientist and Engineer's Guide to Digital Signal Processing, free to read in full at dspguide.com, written for people who want to use the ideas without heavy maths.
• ▶️ Gentler: 3Blue1Brown, "But what is the Fourier Transform? A visual introduction" (YouTube, 2018).
• ▶️ Free course: MIT OpenCourseWare 6.003 Signals and Systems (Fall 2011), Prof. Dennis Freeman, with lecture videos and problem sets.
• 🌐 Easier first reads in this library: The Fourier Transform (12/12), Vibrations (34/17) and Waves and Sound (13/13).