Add Up Enough and Anything Becomes a Bell
There are countless distributions in the world, yet one shape appears everywhere: the bell, the normal distribution. People's heights, test scores, measurement errors, the dimensions of factory parts, even the sample mean of statistics — from utterly different sources, they all draw the same symmetric bell bulging in the middle, as if by agreement. Why is this one shape so common? The answer is elegant and powerful. Add up many small, independent random factors and, no matter what shape each factor had, their sum converges to a bell. This is the central limit theorem, the deepest and most beautiful result in statistics. Height is normal because thousands of genes and environmental factors add up; measurement error is normal because countless small jitters combine. In E3 we summarized a distribution with two numbers, its center and spread, and the normal distribution is completely determined by exactly those two — the mean and the standard deviation. In this final lesson, see why the bell appears everywhere, and how, with that one ruler, we measure the world.
First, get familiar with the bell itself. The normal distribution is highest at the mean and falls away smoothly and symmetrically on both sides. The remarkable thing is that this curve is completely fixed by just two numbers: the mean μ and the standard deviation σ. μ is the position of the peak (the center), σ is the width of the bell. And the normal has one beautiful rule worth memorizing: the 68-95-99.7 rule. Press the buttons. Within ±1σ of the mean lies about 68% of the whole; within ±2σ, about 95%; within ±3σ, about 99.7%. In other words, a normally distributed value is almost always (99.7%) within 3σ of the mean, and stepping outside 3σ is rare — about three times in a thousand. σ is the natural ruler: "how many σ from the mean" tells you "how rare an event this is." With this one rule you can immediately judge whether a measurement is normal or strange.
Now watch how the bell arises on its own. Throw one die and 1 through 6 are equally likely — a flat rectangle of a distribution, nothing like a bell. But throw several dice and look at the sum, and magic happens. Use the buttons to add more dice. Add just two and the flat distribution becomes a triangle peaking at 7; go to three, four, and the corners round off and it edges ever closer to a bell. Why? For the sum to be very small, every die must come up small at once, which is rare, whereas there are countless ways to reach a middling sum. So the middle values come up overwhelmingly often and a peak forms. The raw material — a single die — is nothing like a bell, yet add only a few and a bell appears. This is the simplest possible example of the central limit theorem at work.
Dice are symmetric to begin with, so you might say we cut them some slack. What about a distribution blatantly skewed to one side? Here is the exponential distribution: most mass near 0, dragging a long tail to the right, not symmetric at all. From this skewed distribution we draw N values, average them, and gather many such averages into a histogram. Use the slider to raise the sample size N. At N=1 it keeps the original skewed shape, but as N grows, astonishingly the distribution of the sample mean becomes more and more a symmetric bell, and at the same time narrows. No matter how lopsided the source, average enough draws and the mean converges to a normal distribution. This is the heart of the central limit theorem. And the width shrinks as 1√N as N grows: quadruple the sample and the scatter of the mean halves. That is why more samples make the estimate of the mean more precise. This is exactly why a poll of only a few thousand people can be quite accurate.
If the normal appears everywhere, it would be convenient to unify every normal under a single standard ruler. That's z-standardization. Take a value X from any normal N(μ, σ) and convert it into how far it is from the mean, in units of σ. As a formula, z = (X − μ)σ. First subtract μ to move the center to 0, then divide by σ to set the width to 1. It's a case of the aX+b transform from E3 (a = 1σ, b = −μσ). However much you change μ and σ with the sliders, watch the z ticks under the curve stay fixed at 0, ±1, ±2. The x values (top row) differ for each distribution, but measured in z (bottom row), every normal becomes the same standard normal N(0, 1). This is powerful because if you hold just the one standard normal as a table (or a function), you can convert the probabilities of every normal in the world into z and read them all off that single table. Whether it's the probability of a height of 175 cm or a stock return, convert to z and you can measure it with the same ruler.
Finally, see how all of this is used in engineering. Measure the same quantity many times and the value comes out a little different each time. That scatter is many small noises added together, so by the central limit theorem it follows a normal distribution. Look at the measurement points: they spread out in a bell around the mean. Apply the 68-95-99.7 rule here. Use the buttons to change the band width. About 95% of the measurements land within ±2σ of the mean. Turn that around and you can be about 95% confident that "the true value lies within ±2σ of the measured mean." This is a confidence interval. The ± in writing a result as "5.0 ± 0.2" is exactly this. In engineering no number ever travels alone without a ±. The tolerance of a part's dimension, the error bars of an experiment, the margin of error of a poll, the precision of a sensor — all are stated as "estimate ± so many σ." So statistics is the language not of "the mean is this," but of "the mean lies in this range, and we are this confident of it." The normal distribution and the central limit theorem put numbers on that confidence. With this we complete one full turn of engineering math: the transformations and eigenvalues of linear algebra, the rotation of complex numbers, the flow and stability of differential equations, and the distributions and estimation of probability — the engineer's toolbox now fits in one hand.