Diagonalization Leaves Only Stretching in the Eigenbasis
A matrix usually warps the grid and turns arrows, which is hard to picture. But take the eigenvectors, the special directions that keep their heading under the map and only change length, and make them your new coordinate axes; in those coordinates the matrix suddenly turns simple. Rotation and skew vanish, and all that remains is a diagonal matrix D that stretches each axis by its eigenvalue. That is diagonalization, A = P D P⁻¹, where P holds the eigenvectors as columns. Once split this way, powers Aⁿ become almost free, and for a symmetric matrix those axes come out exactly perpendicular. This one fact threads PageRank, the SVD, and PCA onto a single line.
Toggle the same transformation A between two views. In the standard view the grid warps, and an ordinary arrow that is not an eigenvector changes both its length and its direction. It is a lot to track. Switch to the eigenbasis view and the two eigenvectors become the new coordinate axes. Now A shows up as a diagonal map D = diag(λ1, λ2) that merely stretches the first axis by λ1 and the second by λ2. There is zero rotation. Choose the right basis and a matrix that looked complicated settles into just two scalings; that is the starting point of diagonalization.
Step through how A = P D P⁻¹ acts on one vector. P is the matrix whose columns are the eigenvectors v1, v2. Stage 0 is the starting vector. Stage 1, P⁻¹, carries that vector into eigen coordinates, so you read it as "so much along v1 plus so much along v2." Stage 2, D, scales the first eigen component by λ1 and the second by λ2; pure stretching, the simplest stage. Stage 3, P, maps the result back to the original coordinates. Going in by P⁻¹, stretching by D, coming out by P: those three moves equal one application of A exactly.
The real payoff of diagonalization is powers. Aⁿ = P Dⁿ P⁻¹, and the middle Dⁿ is just the diagonal matrix with each eigenvalue raised to the n. The toil of multiplying the matrix n times disappears. Slide the power k upward. A vector gets stretched by λi each time along each eigen direction, and the eigenvalue with the larger magnitude steadily takes over. So as k grows the vector lines up more and more with the direction of the largest eigenvalue, the dominant eigenvector. This is power iteration, and it is how PageRank and repeated processes settle onto one direction over time.
When a matrix is symmetric (the b slot equals the c slot), something special happens. Drag the slider that controls the off-diagonal term. While A is far from symmetric, its two eigenvector axes sit at a slanted angle. The moment you make A symmetric, the two eigenvectors meet at exactly 90 degrees, perpendicular. This is the spectral theorem: a symmetric matrix always has orthonormal eigenvectors, so it splits as A = Q Λ Qᵀ, where Q is an orthogonal matrix (a rotation) and Λ is the diagonal of eigenvalues. You get to write plain Qᵀ in place of P⁻¹. This orthogonal diagonalization is the geometric heart of the SVD and PCA.
One honest caveat: not every matrix diagonalizes. One side of the toggle is a healthy matrix with two distinct eigen directions, so its eigenvectors span the plane and it diagonalizes. The other side is a shear matrix [[1,1],[0,1]]. Its eigenvalue is a single repeated 1, yet its eigenvector points only along the horizontal axis. You will see the two arrows fold onto the same line. A matrix short of independent eigenvectors like this is called defective, and with too few directions to set up new axes it cannot be diagonalized. The condition for diagonalization is n independent eigenvectors.