PCA Finds the Directions of Most Spread
Imagine scattering a handful of grains on a desk. If they don't land in a round blob but lie stretched out one way, anyone can point and say at a glance, "they spread widest in this direction." That is exactly what Principal Component Analysis (PCA) asks: in a cloud of data, which direction has the most spread? That direction is the first principal component (PC1), and the second-most-spread direction, at a right angle to it, is PC2. Remarkably, these two directions are the eigenvectors of the data's covariance matrix, and how much each one spreads (the variance) is its eigenvalue. The same eigen idea from lesson 8 now applies straight to the spread of data.
Drag the point cloud to spin it around. Grab the handle and rotate, and the whole cloud tilts with it. At any angle, the gold ellipse wrapping the points (the covariance drawn at one sigma) turns along with the cloud. Inside that ellipse the longest axis (green) is PC1, and the short axis at a right angle to it (orange) is PC2. The two axis lengths are the square roots of the variance in each direction, that is, the standard deviations. However you rotate the cloud, PC1 always points exactly along the widest spread. These axes are precisely the eigenvectors of the covariance matrix computed from the points.
Now use the slider to change only the cloud's angle. Two arrows, PC1 (green) and PC2 (orange), follow along the whole time. PC1 is the direction of largest variance, PC2 the second direction at an exact right angle to it. Wherever you set the slider, the two axes always stay 90 degrees apart, because the eigenvectors of a symmetric covariance matrix are always orthogonal to each other. So the principal components break the data down into clean, non-overlapping directions. That means the information on one axis never leaks into the other.
Measure for yourself why PC1 is special. As the slider turns a direction line, every point drops perpendicularly onto that line, leaving a footprint. How spread out those footprints are along the line is the variance in that direction, and the bar shows its value. Swing the line around and the variance rises to a maximum when the line lines up with PC1 and sinks to a minimum when it lines up with PC2. That is why PC1 can also be defined as the direction that maximizes the variance. The most-spread direction is the direction where the most information stays alive.
Now to the real payoff: dimension reduction. Use the toggle to keep only one axis, PC1 or PC2. The points squash flat onto that axis, and the discarded perpendicular direction shows up as dashed stubs, the reconstruction error. The readout is the variance along the dropped axis, that is, how much information you lost. Keeping PC1 leaves the shortest stubs, because you threw away the least-spread direction, so you lost the least. Shrinking 2D down to 1D while keeping the loss as small as possible is exactly how PCA compresses data.
Finally, a tool that asks how many dimensions you truly need. Use the slider to stretch the cloud from round to very elongated. Two bars show the ratio of the eigenvalues: the top is lambda1/(lambda1+lambda2), the bottom lambda2/(lambda1+lambda2). When the cloud is round the two bars are nearly fifty-fifty, so keeping one axis loses half. The more elongated it gets, the closer the PC1 bar climbs to 100 percent and PC2 shrinks toward zero. Then a single axis keeps almost all the information. This explained-variance ratio is the yardstick for how few dimensions you can safely shrink the data to.