fixed-point iteration explained
Put a number in a calculator and keep pressing cos. The result stops moving near one number, about 0.7391. Feeding a function its own output, again and again, is fixed-point iteration. After x comes g(x), then g(g(x)). It stops at a point where g(x)=x, a point that comes back out unchanged. That point is a fixed point. The phase-plane lesson calls the same point a fixed point of a flow. The surprise is that many starts are pulled to the same point. There is a condition. Near the fixed point the slope |g'| has to be less than 1. Only then does each step shrink the distance, which is a contraction. The idea hides in solving equations, in computer graphics, in economic balance, and in search ranking. Newton's method is a very fast fixed-point iteration. The cobweb picture is that pull, drawn.
The blue curve is g(x)=cos x, and the dashed line is y=x. The green point where they meet is the fixed point. Press one step and a cobweb is drawn. From the current x, go straight up to the curve to read the next value, then go across to the line y=x and take that value as the new x. Move the start x0 anywhere and the cobweb still winds into the crossing. This one picture is the whole of fixed-point iteration. The green point is where the two lines share a height.
Some iterations gather and some scatter. The answer is one slope. Here g is a straight line and the fixed point is 1. The slider a is g'. Keep |a| below 1 and the cobweb turns green and stacks into the fixed point. That is a contraction. Push |a| above 1 and it turns red and is driven away from the fixed point, which is divergence. The closer |a| is to 0, the sooner it finishes. A small slope is fast convergence.
The same straight line g, but now watch the sign of a. The size of the slope decides convergence or divergence, and the sign decides the shape. When a is positive the cobweb is a staircase that approaches the fixed point from one side. When a is negative it becomes a spiral that alternates above and below the fixed point, and the sign flips at every step. Drag the slider across 0 and you see the staircase become a spiral, and the spiral become a staircase.
Convergence can be guaranteed and the speed can still vary wildly. The number of steps to reach a tolerance of 1e-6 is about n ≈ log(tol)/log|g'|. The curve shows that number against |g'|. Set the slider near 0.5 and it finishes within about twenty steps. Push it toward 0.95 and the count explodes into hundreds of steps. A contraction close to 1 does converge, but it is painfully slow. That is why a faster method is wanted.
Here is that faster method. Ordinary fixed-point iteration is linear convergence, so the error shrinks by a fixed ratio at each step, like a half. The bars descend gently. Newton's method is convergence of order 2, so the error is squared at each step, the correct digits double, and the bars drop like a cliff. Switch between them and, for the same precision 1e-10, the linear one takes dozens of steps and the order-2 one takes a handful. Newton is the fastest fixed-point iteration because it designs g so that g'=0.