diagonalization of a matrix explained
A matrix usually twists the grid and turns arrows, so it is hard to picture. Take the eigenvectors, the directions that keep their direction under the transform and only change length, and use them as the new axes. In those coordinates the matrix suddenly gets simple. The turning and the twist disappear, and only a diagonal matrix D remains, stretching each axis by its eigenvalue. That is the diagonalization A = P D P⁻¹, where P is the matrix with the eigenvectors as columns. Once it is split this way a power is almost free, and for a symmetric matrix those axes meet at a right angle. This one fact threads PageRank, SVD, and PCA on one line.
You can toggle the same transform between two views. In the standard view the grid warps, and an ordinary blue arrow that is not an eigenvector bends in direction as well as length. It is hard to follow in your head. Switch to the eigenbasis view and the two eigenvectors are laid down as the new axes. The first axis is green and the second is orange. Then the transform shows as the diagonal transform D = diag(λ1, λ2), stretching the first axis by λ1 and the second by λ2. The rotation is 0. Choose the right basis and a matrix that looked complicated tidies into two stretches. That is where diagonalization starts.
Walk through how A = P D P⁻¹ acts on one vector, step by step. P is the matrix with eigenvectors v1, v2 as columns. Step 0 is the blue starting vector. Step 1, P⁻¹, moves that vector into eigen coordinates so you read it as so many steps along v1 plus so many along v2, and the arrow turns gold. Step 2, D, stretches the first component by λ1 and the second by λ2 in those coordinates, and the arrow is green. It is only stretching, the simplest step. Step 3, P, sends the result back to the original coordinates, and the arrow is red. Going in by P⁻¹, stretching by D, and coming out by P is exactly one application of A.
The real gain of diagonalization is powers. Aⁿ = P Dⁿ P⁻¹, and the middle Dⁿ is just the diagonal matrix of each eigenvalue raised to n. The work of multiplying the matrix n times disappears. Raise the power k with the slider and the blue vector grows along each eigendirection every time, and the eigenvalue with the larger absolute value gradually dominates. As k grows the vector lines up with the direction of the largest eigenvalue, the dominant eigenvector drawn as a gold dashed line. That is power iteration, and it is why PageRank or a repeated process settles into one direction as time passes.
If the matrix is symmetric, that is if the b position and the c position are the same, something special happens. Move the off-diagonal term with the slider and when it is far from symmetric the two eigenvector axes lean at a slanted angle. The first axis is green and the second is orange. The moment you make it symmetric the two eigenvectors meet at exactly 90 degrees, a right angle, and the center dot turns gold. That is the spectral theorem. A symmetric matrix always has orthonormal eigenvectors, so it splits as A = Q Λ Qᵀ, where Q is an orthogonal matrix, a rotation, and Λ is the diagonal matrix of eigenvalues. You can put a transpose where P⁻¹ sat. This orthogonal diagonalization is the geometric heart of SVD and PCA.
One honest caveat. Not every matrix diagonalizes. One side of the toggle is a healthy matrix with two different eigendirections, so the eigenvectors spread the plane and it diagonalizes. The green arrow and the orange arrow point different ways. The other side is the shear matrix [[1,1],[0,1]]. The eigenvalue repeats as the single value 1, and only one eigenvector comes out, along the horizontal axis. You see the two arrows collapse onto the same line. A matrix short of independent eigenvectors is called defective, and there are not enough directions to set new axes, so diagonalization is impossible. The condition is as many independent eigenvectors as the dimension.