Loss: measuring how wrong the model is

One point, how far it missed

We have drawn one prediction line.
The points are the true answers,
and the line is the model's guess.
Pick out a single point.
Where it should be is the point,
and the line guessed that spot like this.
If the two sit apart,
by that much the model is wrong.
Let's first see this gap with our eyes.

Tap a point. The vertical gap between the point (true) and the line (prediction) shows up.

Each point had a different gap.
Some points cling close to the line
and some sit far away.
A point near the line was guessed well,
a far point missed by a lot.
But just by looking,
it is hard to tell how wrong the whole is.
We had better measure this gap as a number.

Measure the gap as a distance

Turning the gap into a number
is simpler than you might think.
The vertical distance from the point to the line,
just measure that length.
A long distance means a big error,
a short one means a small error.
Stand up one bar per point
and read its height as the distance.
Now each gap has a number on it.

Tap a point. The distance from the point to the line is measured as a bar with its length as a number.

Each point gave a distance as a number.
A near point is a small number,
a far point a big number.
Now we know how wrong each point is
with a clear number.
But there isn't just one or two points.
With a separate number for each point,
it is hard to see at a glance
how wrong the whole thing is.
Let's gather these numbers into one.

Add up all the gaps

To know how wrong the whole is,
take the distance measured at each point
and add every one of them up.
Small gaps and big gaps alike,
gather them all into one bucket.
Then the scattered numbers
merge into a single number.
This merged number is the loss.
It tells in one number
how wrong the model is overall.

Tap the points in turn. Each gap stacks up as a bar and adds into a single loss number.

The scattered gaps
gathered into one loss number.
Now the model is easy to judge.
Whether the line is good or bad,
we only need to look at this one number.
A big loss means the line
strays far from the points,
a small loss means it fits well.
Then what we must do is clear.
Make this number small.

The goal is to make the loss small

Loss is not a fixed number.
Move the line and it changes too.
Fit the line well toward the points
and the gaps shrink, so the loss drops;
move it away from the points
and the gaps grow, so the loss rises.
So move the line this way and that
and find the spot where the loss number
becomes the smallest.
The line at that spot is the best line.

Move the line with the up and down buttons. The gaps and the loss number change together, find where it gets smallest.

Each time you moved the line
the loss number went up and down.
The better it fit the points, the smaller it got.
If you found the smallest spot,
that is the best-fitting line.
Here we realize something important.
Finding a good model
is the same as making the loss small.
This very task is training.
A model learns by shrinking the loss.

Let's wrap up

Gathered on one line, it is this.
For each point there is a gap
between prediction and answer, a vertical distance.
One number that adds up all those distances
is the loss, how wrong the model is.
The smaller the loss the better the model,
and moving the line to shrink the loss
is exactly training.
One number with all of it added up, that is the loss.

Tap the key points in order to review. (gap at each point -> measure as distance -> add up into loss -> shrinking loss is training)

Now you know what loss is
and why we try to shrink it.
The knack of capturing
how wrong a model is in one number
is the first step in teaching a model.
But to shrink the loss,
which way should we move the line?
We can't just move it blindly.
Next let's look together at how to cleverly find
the direction in which the loss drops.