numerical error and step size
The methods in this set share one promise. The error is about C times the step to the power p, that is error≈C·hᵖ. Here p is the order of the method. Euler is order 1, RK2 is order 2, and RK4 and Simpson are order 4. Plot the error and the step h on log-log axes and you get a straight line whose slope is the order p. Order equals slope. Cut the step in half and an error of order p shrinks by 2 to the power p. For a method of order 4 that is a factor of 16. Shrinking h without end is not the finish. Too small, and rounding error rises off the floor. An explicit method explodes once it crosses its stability limit. The number of steps grows like 1/h, so the cost climbs. The real choice in numerical analysis is the balance between accuracy and cost. The straight lines in the first picture are those orders, drawn as slopes.
The horizontal axis is log h and the vertical axis is log of the error. Take the log of both sides of error=C·hᵖ and you get a straight line whose slope is exactly the order p. The three lines are order 1 in blue, order 2 in orange, and order 4 in green. A button highlights one order, and the h slider moves a point along the line. Shrink h by the same amount and the point on the order-4 line drops much more steeply. A higher order means the same effort cuts the error faster. This one picture is the map of the whole set.
By the order alone, sending h to 0 should send the error to 0. Reality is different. The total error is the truncation C·h² plus a rounding piece ε/h. Shrink h and the first term falls while the second rises. The curve therefore draws a U-shaped valley. Drag the h slider to the far left of the valley and the point turns red, hits the rounding floor, and the error grows again. Nothing is infinitely precise. The bottom of the valley is the best that machine can do.
Accuracy is not the only wall. Take the decay y'=−ky with k=5 and solve it with explicit Euler. The band above is the axis of the step. Green is stable and accurate. Red is unstable. The boundary is h = 2/k = 0.4. Move the slider point inside the green and a smaller step stays closer to the true solution. Cross 0.4 into the red and, whatever the accuracy, the numerical solution flips sign and explodes. A short step is not only precision. It is also the line that stops a blow-up.
Four methods share one plane. The error lines of Euler (order 1), RK2 (order 2), RK4 (order 4), and Simpson (order 4) are drawn together. Pick a method with a button and its line is highlighted, and the h slider moves the point on every line at once. At a large h the gap is small. As you shrink h the higher-order methods become far more accurate. Even at the same order 4, Simpson's line sits a little below RK4 because its coefficient C is smaller. The order sets the slope and the coefficient sets the height.
Last is the practical tug of war. Two bars are shown. The error = C·h² shrinks as you shrink h, and the cost, the number of steps ∝ 1/h, grows as you shrink h. The curve below is their weighted sum, the total loss. Drag the h slider to either end and one end has too much error while the other has too much cost. Somewhere in the middle is the best place. The point turns green there, the step that satisfies precision and cost together. The points at the two ends stay orange. That place is the conclusion the whole set was pointing at.