A Distribution Is How Probability Is Spread
Last lesson we said probability is area. But talking only about the probability of a single event is half the picture. What we really want to know is which values come up how often — how probability is spread out over the possible values. The whole shape of that spread is the distribution. The sum of two dice comes up near 7 often, while 2 or 12 are rare. This whole map of "often and rarely" is the distribution, and drawn as a bar chart it shows a peak. When the values are discrete, spaced apart, each bar is a probability; when the values are continuous, the bars become infinitely dense and turn into a smooth curve, a density. And here is the decisive point. For a continuous distribution, the height of the curve is not the probability — the area under the curve is the probability. The "area = probability" of E1 carries straight over. In this lesson you'll draw a distribution from bars to a curve and then to area, learning by eye how probability spreads.
First, the discrete distribution, where values are spaced apart. One die gives 1 through 6 equally, 16 each — six bars side by side at the same height. But look at the sum of two dice and the shape changes sharply. A sum of 7 happens as (1,6)(2,5)(3,4)(4,3)(5,2)(6,1), six ways, while a sum of 2 happens only as (1,1), one way. So 7 is tall at 636 and 2 and 12 are low at 136 — a triangle bulging in the middle. The height of each bar is the probability of that value, and this is called the probability mass function, the PMF. Tap a bar to check each value's probability. The key is that adding up all the bar heights gives exactly 1, because some value must come up. A distribution is precisely "how the total probability of 1 is divided among the possible values."
What if the values are continuous? When outcomes run without gaps, like heights, weights, or measurement errors. Then counting bars one by one is meaningless, because the probability of exactly 173.4825...cm is essentially zero. Instead we gather samples, group them into intervals, and draw a histogram. Use the slider to increase the sample size N. When N is small the bars are jagged and rough, but as N grows the chance bumpiness averages out and vanishes, and the outline of the bars approaches a single smooth curve. This curve is the probability density function. The histogram is an approximation from finite samples, and the ideal shape, with infinitely many samples gathered, is exactly the density curve. So the density curve is "the PMF when values are continuous," a smooth version of bars sliced infinitely fine and joined. The more data you gather, the more the true distribution hiding behind that data reveals itself.
How do you read a probability off the density curve? This is where people get most confused. The height of the curve is not the probability — in the continuous case the probability of a single point is 0. Instead, probability is read as the area under the curve. "The probability that X lies between a and b" is the area under the curve from a to b. Drag the two bounds a and b to widen the interval between them. As the area grows, that probability grows. Include the tall middle under the peak and the area is large for a given width, while the flat tail gives little area for the same width — which is why the middle values come up more often. The area under the entire curve is exactly 1, just as the whole board was 1 in E1. In the end, the probability of a continuous distribution is measuring area — that is, integration. The integral of calculus is used in probability exactly like this.
Distributions are sorted by the shape of their spread, and each shape suits certain situations. Use the tabs to see three representative distributions. The uniform distribution is a flat rectangle of a curve. It fits when anything within a fixed interval is equally likely — for example the random numbers a computer generates, or rounding error. The exponential distribution is highest at the start and drops off sharply to the right. It appears for the waiting time until the next event, the time until a part fails, the gap between arriving customers — cases where "short waits are common and long waits are rare." The normal distribution is the famous bell shape, highest in the middle and falling away smoothly and symmetrically. Heights, test scores, measurement error — almost every natural spread that arises from many small factors added together has this shape. Knowing a distribution's shape lets you anticipate in advance how the data will behave.
Finally, there's another picture that pairs with the density: the cumulative distribution function, the CDF. If the density shows "how much probability is concentrated near each value," the CDF shows "the probability of getting a value less than or equal to x" — the cumulative probability piled up from the far left to x. The picture makes it clear. On top is the density curve, below is the CDF. Drag x, and on top the area up to x gets shaded, while below that area value is marked as a single dot. So CDF(x) is exactly the area under the density curve up to x. Move x from the far left to the far right and you sweep up more and more area, so the CDF starts at 0 and rises to 1, never once going down. Over intervals where the density's peak is tall the CDF rises steeply, and over the tails it rises gently. The two curves are a pair, the same distribution seen two ways: the density is the slope, the CDF is that slope accumulated into area. The relationship of derivative and integral holds here too.