An Inner Product Gives a Space Its Geometry
What is length, what is angle, what is a right angle? They feel so obvious they hardly need defining. Yet in a vector space these three are not handed to you for free. A single device, the inner product <u,v>, manufactures all three at once. Length is the square root of a vector's inner product with itself, angle is the inner product of two vectors divided by their lengths, and a right angle is where the inner product is zero. The dot product you met earlier is just one example of this inner product. Change the inner product, say by weighting each axis differently, and the geometry itself bends over the very same vectors. More startling still: functions are vectors too. The inner product of two functions is the integral of their product, and right here Fourier transforms and kernels begin.
Keep the green vector u fixed horizontally and drag the blue vector v. Four numbers track along underneath: the inner product <u,v>, the length of u, the length of v, and the angle between them. Stretch v out the same way as u and the inner product grows; stand it at a right angle and the inner product is zero; turn it the other way and it goes negative. Here is the key: one inner product packs length and angle into a single box. Length, angle, and whether they are perpendicular all unpack from this one quantity, the inner product. From the next block we will pry these three apart one at a time.
First, length. Take a vector's inner product with itself, <v,v>, and that is exactly the length squared. So the length ||v|| is the square root of <v,v>. Drag v and the radius out to the arrow tip is its length, while the dashed unit circle is the set of vectors whose length is exactly 1. Put v on the unit circle and <v,v> reads exactly 1. You never have to define length separately; with just an inner product, length falls out on its own. A vector's inner product with itself is its length squared, and that single sentence is where every norm begins.
Now change the inner product itself. The weighted inner product is <u,v>w = w1·u1·v1 + w2·u2·v2, giving the two axes different weights. Slide w1 and w2. The set of length-1 vectors, the unit ball, is no longer a circle but an ellipse. Its semi-axes are 1/sqrt(w1) and 1/sqrt(w2), so the more weight you pile on an axis, the shorter that direction becomes, because distance along it is now priced more dearly. The vectors are unchanged, yet the geometry bends as the standard for what counts as long shifts. Whenever you handle data whose units differ axis by axis (like height versus weight), you are in fact always using a weighted inner product.
A right angle, too, is ultimately defined by the inner product. Two vectors are orthogonal simply when <u,v>w = 0. Here the weights are fixed (heavier on one axis) and you spin only the angle of v with the slider. The instant the weighted inner product crosses zero, v lights up. But look closely: at that instant u and v do not look 90 degrees apart on screen. The weights have bent the space, so a right angle in the weighted world disagrees with the right angle your eye sees. A right angle is not the visible 90 degrees but the fact that the inner product is zero. Which inner product you use decides what counts as orthogonal.
Finally, the biggest leap: functions are vectors. The inner product of two functions is their product integrated over an interval, <f,g> = ∫ f·g. Here f = sin(x) is fixed, and the slider changes the integer frequency k of g = sin(kx). The area of the product f·g is shaded, and when k is 1 a clear positive area remains. But once k is an integer other than 1, the positive and negative parts cancel exactly, so the integral is 0, meaning orthogonal. Sine waves of different frequencies are at right angles to each other in function space. The Fourier transform, which splits a signal apart frequency by frequency, stands on exactly this orthogonality.