law of large numbers explained
Toss a coin only a few times and the share of heads jumps around, but toss it a great many times and it settles near 0.5. The sample mean gathering toward the true expected value E[X] as you collect more samples is the law of large numbers. Turn the law around and you can estimate a value that is hard to compute directly, by random samples, and that is the Monte Carlo method. The error shrinks at the speed 1/√n, and the limit is the same even when the seed differs. The more you draw, the more the mean comes home.
Count heads as 1 and tails as 0 and the expected value is E[X] = 0.5, and for a die the average of the faces is E[X] = 3.5. Raise the number of tosses and the running mean up to that point is drawn. The first few tosses swing widely, but as the count grows the curve sticks to the expected value marked by the dotted line. The same seed is used, so opening it at any time gives the same path. The swinging dies down.
Throw darts at random into the square [0,1]×[0,1]. The share of darts that land inside a quarter circle of radius 1 centered at the origin converges to π/4. Multiply the share inside by 4 and you get an estimate of that constant. Raise the number of darts and the points grow dense, and the estimate narrows toward 3.14159. Measuring an area with random samples is the seed of Monte Carlo integration. The points measure the area.
Drawing more does improve things, but how fast? Plot the estimation error against the count and the error shrinks at about the speed 1/√n. Halving the error means you have to multiply the sample by four. Switch the guide lines between 1/√n and 1/n and the real error follows the slow 1/√n, not the fast 1/n. The fact that accuracy is expensive shows itself here. The slow side is the real speed.
One step further, and it is not only the mean that converges. The whole histogram of the sample approaches the probability density curve of the population. When the count is small the bars are bumpy, and when you raise the count the outline of the bars lies on the smooth density curve. You can check this while switching between a bell shape and a flat shape. This is the ground for reading the shape of a distribution from a histogram. The bars come to resemble the curve.
It is random, so how does the same picture keep appearing? Our random numbers are a deterministic sequence fixed by one seed, so the same seed rebuilds the same path. Change the seed and the early swinging differs each time, but every path gathers in the end to the same expected value 0.5. Randomness makes the paths differ, and the law of large numbers makes the destination the same. So an experiment can be reproduced and still give a universal conclusion. The roads differ and the end is the same.