probability distribution explained
The last lesson said probability is area. But talking only about the probability of a single event is half the picture. What we really want to know is how often each value comes up: how probability is spread over the possible values. The overall shape of that spread is the distribution. The sum of two dice often lands near 7, while 2 or 12 are rare. The complete map of "often" and "rarely" is the distribution, and drawn as a bar chart it shows a peak. When the values are discrete, each bar is a probability; when they are continuous, the bars become infinitely fine and turn into a smooth curve, the density. One thing is decisive here. In a continuous distribution, probability is not the height of the curve but the area under it. The "area = probability" of the last lesson carries straight over. In this lesson you draw distributions as bars, then as curves, then as areas, and learn by eye how probability spreads.
Start with a discrete distribution, where the values come in separate steps. One die gives 1 through 6, each with exactly 16, so six bars stand side by side at the same height. Look at the sum of two dice, though, and the shape changes sharply. A sum of 7 can happen six ways, (1,6)(2,5)(3,4)(4,3)(5,2)(6,1), while a sum of 2 happens only one way, (1,1). So 7 stands tall at 636 while 2 and 12 sit low at 136, giving a triangle that bulges in the middle. The height of each bar is the probability of that value, and this is called the probability mass function, PMF. Switch between the two cases with the tabs, and tap a bar to show its probability as a fraction above it. The key point is that all the bar heights add up to exactly 1, because some value is bound to come up. A distribution is the question of how a total probability of 1 is shared out among the possible values.
What if the values are continuous, like height, weight, or measurement error, where the results run on without gaps? Then counting bars one by one is meaningless, because the probability of exactly 173.4825...cm is effectively 0. Instead, collect samples, group them into bins, and draw a histogram. Increase the sample size N on the track below and the change becomes clear. With a small N, the blue bars are ragged, but as N grows the accidental bumps average out and the outline of the bars approaches the single gold curve. That curve is the probability density function. A histogram is an approximation built from a finite sample, and the density curve is its ideal shape when samples pile up without end. So the density curve is the PMF for continuous values: a smooth version made by slicing the bars infinitely thin and joining them. The more data you collect, the more clearly the true distribution behind it emerges.
How do you read probability from a density curve? This is where people get confused most often. The height of the curve is not a probability, because in the continuous case the probability of a single point is 0. Instead, probability is read as the area under the curve. The probability that X lies between a and b is the area under the curve from a to b. Drag the green and orange bounds to widen the interval between them, and the shaded area grows along with the probability shown below. Include the tall middle and the area is large even for the same width; in the flat tails it is small even for the same width. That is why middle values come up more often. The area under the whole curve is exactly 1, just as the whole board was 1 in the last lesson. In the end, probability in a continuous distribution means measuring area, that is, integrating. This is how integration from calculus is used in probability.
Distributions come in kinds according to the shape of their spread, and each shape suits certain situations. Use the tabs to look at three representative distributions. The uniform distribution is a flat rectangle. It fits when any value within an interval is equally likely, as with random numbers generated by a computer or rounding error. The exponential distribution is highest at the start and drops off sharply to the right. It appears when short waits are common and long ones rare: the time until the next event, the time until a part fails, the gaps between arriving customers. The normal distribution is the famous bell, highest in the middle and falling off smoothly and symmetrically on both sides. Most natural scatter that comes from adding up many small factors, such as heights, test scores, and measurement errors, has this shape. For each tab, one line under the figure says where that distribution shows up. Know the shape of a distribution and you can anticipate how the data will behave.
Finally, there is one more picture that pairs with the density: the cumulative distribution function, or CDF. Where the density shows how much probability is packed near each value, this function shows the probability of getting a value less than or equal to a given one: the probability accumulated from the left end up to that point. The picture makes it clear. On top is the gold density curve, and below is the blue cumulative curve. Drag the handle and, on top, the area up to that point is shaded while, below, that area's value is marked as a single dot. So CDF(x) is exactly the area under the density curve up to that point. Move the handle from the left end to the right end and it sweeps up more and more area, so the cumulative curve starts at 0 and rises to 1 without ever going down. Where the density peaks, the cumulative curve climbs steeply; in the tails, it climbs gently. The two curves are a pair, two views of the same distribution. The density is a slope and the cumulative curve is the area that piles up that slope, so the relationship between differentiation and integration holds here too.