QR decomposition explained
The columns of a matrix are usually skewed. They are not perpendicular and their lengths differ. QR splits those columns into two pieces. First Gram-Schmidt straightens them into perpendicular unit vectors. That piece is Q, a pure rotation or reflection that touches neither length nor angle. What was trimmed along the way, the lengths and the overlaps, is gathered into R, and R is always a triangle filled only on the top. A = QR. One rotation and one triangle. This one line is the stable way to solve least squares, and it is a workhorse of numerical linear algebra. A grid that stays square is the evidence that Q is a rotation. The skewed columns are A and the straightened columns are Q. The line that minimizes the sum of squared errors is the answer QR finds.
Step through with the buttons. Step 0 is the two columns of A, a1 and a2. They lean, and their lengths differ. At step 1 Gram-Schmidt straightens the pair. q1 is the direction of a1 shortened to unit length, and q2 is what remains of a2 after the shadow toward q1 is removed, then scaled to unit length. Now q1 and q2 are exactly perpendicular and both have length 1, so their dot product reads 0. At step 2, R appears. R = Qᵀ A, so it is upper triangular automatically, and the lower-left entry r21 is 0. A is the skewed columns, Q the straightened orthogonal columns, and R the small triangle that joins them. That is A = QR.
The two columns of Q are perpendicular unit vectors. Act with Q on the plane and the grid does not distort. It only turns as a whole. Turn the angle with the slider and the cells neither stretch nor squash. A square stays a square, because length and angle are preserved. Press the reflection button and the figure flips like a mirror, and the columns are still orthogonal units, so the shape does not change. That is the definition of an orthogonal matrix. QᵀQ = I. The QᵀQ shown below stays the identity at every angle. It is the evidence that Q is only a rotation or a reflection.
Why is R filled only as an upper triangle? Each entry is rij = qi · aj, the length of the shadow of the original column j on the straightened direction i. Turn an entry on and the dot product is shown. The point is the entries below the diagonal. r21 = q2 · a1, and q2 is the orthogonal part left after removing a1, so it is exactly perpendicular to a1. Perpendicular means the dot product is 0, so r21 is 0. In general the later direction qi is not used on an earlier column aj, where j is less than i. That column was already expressed by the earlier directions. So everything below the diagonal is 0, and R is upper triangular.
Now hold R and rebuild A the other way. Fix Q as a rotation of 20 degrees and move the three entries of R with the sliders, and A = QR is redrawn at once. r11 sets the length of the first column a1, so raising it lengthens a1. r22 is how far the second direction reaches, the length of the orthogonal part. r12 is the shear, how much the second column leans toward the first direction. These three numbers hold the lengths and the lean, and Q only turns the result by 20 degrees as a whole. R takes the shape and Q takes the direction.
The place QR actually works is least squares. The line that fits the data best is the one that makes the sum of squared residuals, the vertical dashed segments, as small as possible. You can move the slope and the intercept with the sliders and shrink that sum. Textbooks usually build AᵀA and solve the normal equations, and AᵀA is sensitive to error, so it is often numerically unstable. QR skips that step. Put in A = QR and the normal equations shrink to R x = Qᵀ b. R is a triangle, so one back-substitution finishes the job. Press the button and the line jumps to that QR solution, and the sum of squared errors is minimal. You get the same answer more stably without building AᵀA.