Drawing a line through points: regression

We start from scattered points

Earlier we looked at how spread out
values are, together with the average.
This time we look not at one value
but at two values paired up.
Like hours studied and the score got.
Put time across and score up the side,
and one person becomes one point.
Since everyone differs,
the points do not fall on one neat line
but scatter about like a cloud.

Tap the empty area to scatter points. A faint upward trend to the right starts to show.

Even with the points scattered,
squint and a trend appears.
The further right you go,
the higher it mostly climbs.
It means those who spent more time
tend to have higher scores.
But there are so many points
that naming a new person's score exactly is hard.
If we could draw this trend as one line,
wouldn't it get far clearer?

Draw a line through the points

The easiest way to show the trend as one line
is to draw a single straight line.
Not a wiggly one, a straight one.
This line passes as if cutting
through the middle of the scattered points.
Some points land above the line,
some below it,
but on the whole the line stands for what is between.
This one line becomes the tool for prediction.

Tap the gaps between points to place where the line passes. Tap two spots and a straight line is drawn between them.

Draw one line
and the scattered points look far tidier.
Now pick any time on the across axis
and read the score on the up axis along the line.
That is prediction.
The line gives an answer even where no point was.
But where to place the line and how much to tilt it
was still up to us.
There is a rule that sets the line.

Slope and intercept set the line

A straight line is set fully by two numbers.
The first is the slope.
It says how much it rises up the side
for one step across.
A big slope makes the line steep,
a small one makes it gentle.
The second is the y-intercept.
It sets the height where the line meets the up axis,
that is, the starting position.
Fix these two and the line is fixed to exactly one.
We write this as y=ax+b.

Change slope a and intercept b with the plus and minus buttons. The line steepens, flattens, and slides up or down.

Raise the slope and the line steepened,
lower it and it lay down, as you saw.
Lift the intercept and the whole line slid up,
drop it and it slid down.
With just two numbers
you can move the line at will.
Then for the scattered points,
how do we find the a and b
that fit best?
We need a measure that tells a good line from a bad one.

The line with the smallest mismatch is best

The measure for a good line is simple.
Look at the distance from each point to the line.
The farther a point sits from the line,
the more it has missed.
Add up this distance over all points
and you get how off the line is overall.
The smaller this total mismatch,
the better the line fits the points.
So the best-fitting line
is the one whose total mismatch is smallest.

Nudge the line with the buttons to lower the total mismatch with the points. The smaller it gets, the closer to best fit.

Lowering the total mismatch,
the line settled into the middle of the point cloud.
The spot where it no longer drops much,
that is the best-fitting line.
We found it by eye and estimate,
but a machine repeats this lowering
countless times and finds it very precisely.
What matters is that the method is the same.
Lower the mismatch to find the a and b that fit best.

Let's wrap up

Gathered on one line, it is this.
When paired points are scattered,
draw one best-fitting straight line through them.
The line is set by two numbers, slope a and intercept b,
chosen so the mismatch with the points is smallest.
Feed a new input into that line and you predict the result.
Keep just one thing in mind here.
The a and b that set the line,
these two numbers come back in the neuron we learn next,
under the names weight and bias.

Tap the key points in order to review. (scattered points -> draw a line -> slope a and intercept b -> lower mismatch to best fit -> weight/bias preview)

Now you know how to draw a line through scattered points
and estimate what comes next with that line.
Data that looked complicated
got far clearer with one straight line.
The idea that two numbers, slope and intercept,
make a prediction
is the seed of nearly everything we will learn.
Next, let's keep going and see how this one line
becomes a neuron and makes smarter predictions.