A Statistic From a Sample Is Itself Random
Draw one sample and take its average, and you get a single number. Draw a different sample and you get a different number. In other words, a statistic computed from a sample, here the sample mean, is itself a random variable that jumps around from sample to sample. This jumping has its own law. The distribution of the values the sample mean can take is called its sampling distribution, and the width of that distribution is called the standard error. In the engineering-math lesson normal-clt you saw that adding many values yields a bell; here we ask a sharper question, how much the mean itself scatters from sample to sample, and how that scatter shrinks as the sample grows. There is exactly one key formula: the standard error equals σ divided by the square root of n. That single line explains why a poll needs only a few thousand people, why we repeat an experiment many times, and where the ± beside every estimate comes from.
Press the button to draw a fresh sample. Even though every sample comes from the same population and has the same size n, its mean x̄ comes out a little different each time. One draw is one dot, another draw is another dot, and as you keep stacking these sample means into a histogram, what starts out ragged settles into a mound that bulges in the middle. This accumulating pile is the sampling distribution, the distribution that the statistic called the sample mean draws for itself. The key point is that the thing we are measuring, the population mean, is fixed, while the tool we estimate it with, the sample mean, wobbles. Knowing the shape and width of that wobble is the whole of this lesson.
Now look at two distributions side by side. The wide grey shows how the individual raw data are scattered, and its width is the population standard deviation σ. Rising narrowly on top of it, the blue distribution shows how the means of size-n samples are scattered, packed far more tightly toward the center. Because averaging lets large and small values cancel each other, the mean wobbles far less than any single value. This narrow width is the standard error, and although its name resembles the standard deviation σ, it is a completely different quantity: σ says how much individuals differ, while the standard error says how unreliable the average of those individuals is. Drag n upward and watch the grey stay put while only the blue keeps getting narrower.
Now pin that width down with an exact formula: the standard error equals σ divided by the square root of n. The crucial thing is that the denominator holds the square root of n, not n itself. Drag n up and the standard error slides down the curve, dropping steeply at first and then flattening out. You must quadruple n to halve the standard error, and to cut the error to a tenth you must gather a hundred times the sample. Because of this square-root wall, buying more precision costs ever more. So past a certain point, improving the measurement itself beats collecting more samples. More data is always better, but knowing that the payoff grows only at the pace of the square root of n is what real judgment is made of.
So far the population has been a bell, so we have been going easy. What about a distribution blatantly skewed to one side? Here the exponential distribution piles up near 0 and drags a long tail to the right, not symmetric at all. At n equal to 1 the distribution of the sample mean is just this skewed original. But drag n up and something astonishing happens. However lopsided the source, the distribution of the sample mean steadily approaches a symmetric bell and snaps onto the gold normal curve. This is the sampling-distribution face of the central limit theorem. In normal-clt you saw that adding values yields a bell; here you see that the distribution of that sum divided by n, the mean, converges to N(μ, σ/√n) regardless of the original shape. That is why, even without knowing the shape of the population, we can treat the sample mean itself as normal.
Finally, overlay the effect of sample size on the sampling distribution directly. Grey is the distribution of means for a small sample (n=4), spread out wide. Blue is the distribution of means for the large n you set with the slider. The higher you push n, the taller and narrower the blue rises, squeezing in around the true mean. Both distributions share the same center μ, and only the width shrinks, at the pace of σ/√n. What this means in practice is clear: a larger sample makes our estimate land closer to the true value more often, that is, the estimate is more precise. Polls, clinical trials, and quality inspections raise the sample size for exactly this picture. But as block 3 showed, the payoff grows only as the square root of n, so how narrow is narrow enough is always weighed against cost.