What is linear regression

Starting with scattered points

Up to the last lesson, we looked at the middle and the spread of a single value, and at likelihood. This time we look at two values that come in pairs, such as hours studied and score received. Put time on the horizontal axis and score on the vertical axis, and each person becomes one point.
Because everyone's situation is different, the points do not fall neatly on one line; they scatter like a cloud. Let's place points one at a time below.

Tap the empty spots to scatter points. A faint trend shows: the further right, the higher the points go.

Even with the points scattered, stepping back reveals a trend: the further right, the higher they generally go. That means people who spent more time tend to score higher.
But that alone makes it hard to state a new person's score precisely. If this trend could be drawn as a single line, it would become much clearer.

Drawing a line through the points

The simplest way to show the trend is to draw a single straight line. The line passes as if cutting through the middle of the scattered points. Some points sit above the line and some below, but taken as a whole, the line stands for them.
It is called linear regression because it shows the trend with a straight line. With the line in hand, you can read off values even where there were no points.

Tap one spot on the left and one on the right for the line to pass through. Choose both and a straight line is drawn between them.

With a line drawn, the scattered points look far more orderly. Now pick any time on the horizontal axis, follow it up to the line, and read off the score on the vertical axis. That is prediction.
But so far, where to draw the line and how steeply to tilt it was up to us; the line changed depending on the spots we chose. We need a rule for deciding the line.

Slope and intercept decide the line

A straight line is completely decided by two numbers. One is the slope, which tells how far the line rises vertically for each step horizontally. A large slope makes the line steep; a small one makes it gentle.
The other is the y-intercept, the height where the line meets the vertical axis. Decide these two and there is exactly one line, written as the equation y=ax+b, where a is the slope and b is the intercept.

Change the slope (a) and the intercept (b) with the plus and minus buttons. The line gets steeper or gentler and moves up and down.

Raise the slope and the line steepens; lower it and the line lies down. Raise the intercept and the whole line slides up; lower it and the line slides down. With just two numbers, you can move the line wherever you like.
Then how do you find the slope (a) and intercept (b) that fit the scattered points best? You need a standard that separates a good line from a bad one.

The line with the smallest mismatch is best

The standard is simple. For each point, look at how far apart, vertically, the value the line predicts and the actual value are. This gap is called the residual, and the bigger it is, the further the line missed that point.
Add up the gaps for all the points, and you get how far off the line is overall. The smaller that total, the better the line fits, so the best-fitting line is the one with the smallest total mismatch. The picture below adds up the sizes of the gaps as they are, but ordinary linear regression makes the sum of the squared gaps as small as possible. This is called the method of least squares, and it weighs points that miss by a lot more heavily.

Use the buttons to nudge the line and shrink the total mismatch. The smaller the total, the closer you get to the best-fitting line.

As the total mismatch shrank, the line settled into the middle of the cloud of points. The place where the total will not shrink any further is the best-fitting line.
You found it by estimate, pressing buttons, but a machine repeats this shrinking countless times and finds it very precisely. With least squares, the answer can also be found in one step by solving an equation. Either way, the job is the same: shrink the mismatch to find the slope (a) and intercept (b) that fit best.

Summary

To sum up: when paired points are scattered, you draw the single straight line that fits best through them. The line is decided by two numbers, the slope (a) and the intercept (b), chosen so that the mismatch with the points is as small as possible. Put a new input into that line, and it predicts the result.
Remember one thing: the slope (a) and intercept (b) that decide the line come back later, in the neuron, under the names weight and bias.

Tap the key points in order to review. (scattered points → draw a line → the slope (a) and intercept (b) → shrink the mismatch to the best-fitting line → a preview of weight and bias)

Now you can draw a line through scattered points and use it to estimate what comes next. The idea that two numbers produce a prediction is the seed of almost everything you will learn from here on.
Regression predicts continuous numbers like scores. But often you need to answer which group something belongs to instead, such as whether a mail is spam, or whether a photo shows a cat or a dog. The next lesson looks at classification, which handles problems like these.