Caching: keep a copy close, use it fast
Say you fetch a book you read often from a far warehouse every single time. Each trip takes ages. But what if you keep one copy on your desk? From then on you just reach out your hand. Keeping things you use often as a copy nearby is caching.
A far warehouse is slow to reach
Earlier we learned a database
jumps straight to the answer with an index.
But even after finding it,
we still have to fetch the data from somewhere.
What if that store sits far away?
The trip is long, so each fetch
takes quite a while.
And when you need the same thing again,
you go down that long road once more.
Tap the data to fetch it from the far warehouse. The path is long, so the round trip takes a while.
One fetch already took a long road.
Once or twice is bearable.
But what if you need the same data
hundreds of times a day?
You would travel that long road hundreds of times.
Just thinking about it is tiring.
If it is something you look at often,
do you really need to go far every time?
Keep a copy close by
The trick is surprisingly simple.
For data you look at often,
fetch it once from the far warehouse
and then keep a copy in a nearby spot.
This nearby holder is called a cache.
Next time you want the same thing
you need not go to the far warehouse.
You pull it from the cache right beside you,
quick, by a short road.
Tap the data to copy it into the nearby cache. The empty cache fills up.
A copy now sits in the nearby cache.
Now when you look for the same data again
you can forget the far warehouse.
You pull it from the cache by a short road.
But there is one thing to mind.
When what you want is in the cache
and when it is not, the work differs.
Let's look at the two cases apart.
A hit and a miss
When finding, there are two branches.
If what you want is already in the cache
you pull it straight by the short road.
We call this a hit. It is fast.
If what you want is not in the cache
you have no choice but to go to the far warehouse.
We call this a miss. It is slow.
On a miss, while you are at it,
you leave a copy in the cache.
Then next time it becomes a hit.
Tap a request. If it is in the cache, a short-path hit; if not, a long-path miss that then leaves a copy in the cache.
At first it missed and was slow,
but once a copy was left behind
the same request hit and got fast.
The more often you look,
the more hits add up and the average speeds up.
This is why a cache is dependable.
But there is one headache.
If the original in the far warehouse changes,
what happens to the copy in the cache?
When the cache goes old
A cache is only a copy.
Even if the original in the far warehouse changes,
the cache does not know
and still holds the old copy it received before.
Then a hit is fast, yes,
but the value it hands out may be old.
We say the cache has gone stale.
Fast, but at the risk of an old value,
that is the shadow of caching.
How do we guard against it?
Tap the original to change its value. The cache still holds the old one. Refilling or dropping the cache makes it match again.
Changing the original threw the cache off.
There are two main ways to guard it.
One is, when the original changes,
to refill the cache with the new value.
The other is to put a time limit on the copy
and drop it once some time passes.
The emptied spot gets refilled
with a fresh copy on the next miss.
Either way it is a balance: fewer old values
while keeping the fast hits alive.
Let's wrap up
Gathered on one line, it is this.
A far store is slow, the road there is long.
Keep often-used things as a copy close by.
That is the cache.
If what you want is in the cache it is a fast hit;
if not, it is a miss and you go to the far place.
After a miss you leave a copy, ready for next time.
If the original changes the cache goes old,
so you refill it or drop it to guard against that.
Keep it close and use it fast, that is caching.
Tap the key points in order to review. (far is slow -> a copy close -> hit and miss -> old when the original changes)
Now you know how to keep a copy close
and reuse the same thing fast.
The stuffiness of going far every time
cleared up with a single copy.
Minding old values for the speed you gained,
this sense of balance is the power of handling data.
Caches hide all over a computer.
Next, where else this wisdom of copies
gets used, let's keep exploring together.