central limit theorem explained

First, get acquainted with the bell itself. The normal distribution is highest at the mean and falls off smoothly and symmetrically on both sides. Remarkably, the curve is completely fixed by just two numbers, the mean μ and the standard deviation σ. μ is the center, the position of the peak, and σ is the width of the bell. The normal distribution also has one handy rule worth memorizing: the 68-95-99.7 rule. Choose a width with the buttons and the shaded area shows that about 68% of the total lies within ±1σ of the mean, about 95% within ±2σ, and about 99.7% within ±3σ. In other words, a normally distributed value is almost always (99.7%) within 3σ of the mean, and straying beyond 3σ is rare, about three times in a thousand. σ is a natural ruler. How many σ a value lies from the mean tells how rare it is. With this single rule you can judge on the spot whether a measurement is normal or strange.

Now watch the bell emerge on its own. Roll one die and 1 through 6 are equally likely, so the distribution is a flat rectangle, nothing like a bell. Roll several dice and look at their sum, though, and something magical happens. Increase the number of dice with the buttons. Add just two and the flat distribution becomes a triangle peaking at 7; go to three or four and the corners round off as it gets closer and closer to a bell. The blue bars are the probabilities of the sum, and the gold curve is the normal distribution with the same mean and spread. Why does this happen? For the sum to be very small, every die must show a low face at once, which is rare, while there are countless ways for the sum to land near the middle. So middle values come up overwhelmingly often and a peak forms. A single die, the raw ingredient, is nothing like a bell, yet adding just a few of them makes one appear. This is the simplest example of the central limit theorem at work.

Dice are symmetric to begin with, so you could say they were an easy case. What about a distribution that is openly lopsided? Here is the exponential distribution: most common near 0, with a long tail trailing off to the right, not symmetric at all. Draw N values from this skewed distribution, average them, collect many such averages, and draw their histogram. Increase the sample size N on the track below. With N=1 you see the original skewed shape, but as N grows, the distribution of the sample mean remarkably turns into a symmetric bell and narrows at the same time. However crooked the source, if you draw enough and average, the average converges to a normal distribution. That is the heart of the central limit theorem. Its width also shrinks as 1√N as N grows, and the value appears below. Draw 4 times as many samples and the scatter of the mean halves. That is why more samples make the estimated mean more precise, and why a poll can be fairly accurate after asking only a few thousand people.

If normal distributions appear everywhere, it is convenient to unify them all with one standard ruler. That is z-standardization. Take a value X from some normal distribution N(μ, σ) and rewrite how far it is from the mean in units of σ. As a formula, z = (X − μ)σ. First subtract μ to move the center to 0, then divide by σ to set the width to 1. It is a case of the aX+b transformation from the last lesson (a=1σ, b=−μσ). However you change μ and σ with the two tracks below, the z tick marks under the curve stay fixed at 0, ±1, ±2. The x values on the gold top row differ from one distribution to another, but measured by the z values on the blue bottom row, every normal distribution becomes the same standard normal N(0, 1). This is powerful because, with just one table or function for the standard normal, you can turn the probability of any normal distribution in the world into z and read it from that single table. Whether it is the probability of a height of 175cm or a stock return, converting to z lets you measure it with the same ruler.

Finally, see how all this is used in engineering. Measure the same quantity many times and the value differs a little each time. That scatter is the sum of countless small noises, so by the central limit theorem it follows a normal distribution. The measurement points in the figure, too, spread around the mean in a bell shape. Apply the 68-95-99.7 rule here. Change the width of the band with the buttons and the points split into gold inside the band and red outside, and about 95% of the measurements fall within ±2σ of the mean. Press "Measure again" to check the same fraction with a fresh set of points. Turn it around and you can be about 95% confident that the true value lies within ±2σ of the measured mean; that is a confidence interval. The ± in a result written as 5.0 ± 0.2 is exactly this. In engineering no number travels alone without its ±. Part tolerances, the error ranges of experimental values, the sampling error of polls, and the precision of sensors are all expressed as estimate ± so many σ. So statistics is not a language for saying "the mean is this much" but for saying "the mean lies in this range, and we are this confident of it." The normal distribution and the central limit theorem put a number on that confidence. This lesson closes the probability and statistics thread. Transformations and eigenvalues from linear algebra, rotation from complex numbers, flow and stability from differential equations, and distributions and estimation from probability: the basic tools in an engineering student's toolbox are now in hand.