Add a Margin to an Estimate and You Get a Confidence Interval
A point estimate from a sample is a single number, so it is almost surely a little off from the truth. A confidence interval admits that uncertainty by surrounding the estimate with a margin of error, giving the range estimate plus or minus z*SE. The subtle part is the reading: 95% confidence is not a statement about this one interval, but about the fraction of the time the procedure catches the true value when sampling is repeated. The parameter is fixed; what is random is the interval. Across five figures we build the interval from a margin, watch 95% of repeated intervals cover the true value, trade confidence against width, shrink the margin as 1/sqrt(n), and separate the correct reading from a common misconception.
A confidence interval starts from the point estimate and adds a margin of error on each side. The margin is z*SE, and the multiplier z comes from the confidence level: 90% gives z = 1.645, 95% gives z = 1.96, and 99% gives z = 2.576. Change the confidence level below and the bracket grows around the same estimate, while the fixed true value stays exactly where it was.
This picture is what 95% really means. Each sample gives a different estimate, so the interval changes every time. Draw many and about 95% of them cover the fixed true value, while the ones that miss are drawn in red. The more you draw, the closer the observed coverage creeps to the confidence level. The true value never moved; the intervals did.
More confidence is not free. To cover the true value more often, you have to cast a wider net. Raising the level shades a larger central area under the curve and at the same time stretches the interval width 2*z*SE. The band for 95% and for 99% differ noticeably, and this tradeoff cannot be avoided.
The margin is z*sigma/sqrt(n), so a larger sample shrinks it, but only at the rate 1/sqrt(n). This is the same standard error you saw in the sampling distribution. To double the precision, that is, to halve the margin, you must quadruple the sample. Diminishing returns are built into the formula from the start.
The last step is a careful reading. The statement, over repeats this procedure catches the true value 95% of the time, is right. But the statement, there is a 95% probability the true value is in this interval, is wrong. Once the interval is drawn, the fixed true value is either inside it or not. Probability lives in the random act of sampling, not in a single interval that has already been realized.