What is classification in machine learning
In the last lesson we drew a straight line through points to predict values such as scores. But sometimes the question is different: is this email spam or normal, is this photo a cat or a dog? Instead of hitting a number, you sort things by which group they belong to. Choosing one of a set of fixed groups like this is classification.
Two groups gather apart
The points are scattered again, but this time each point has a color: one color for spam emails and another for normal ones. Each point's position is set by features of the email, for example the number of links across and the number of advertising words up.
Look closely and you will see points of the same color gathered in similar places: spam with spam, normal with normal. That is because things with similar features sit close together.
Tap spread. The mixed-up points gather by color into similar places, and two groups appear.
Two groups are visible: one color gathered on the left, another on the right. A person sees the split at once, but to get a computer to do it, you need a clear rule, not a rough guess.
Draw a line marking where one side ends and the other begins, and the rule becomes clean.
Drawing a boundary between them
Draw a single line between the two groups. This line is called the decision boundary. One side of it is one group's territory, and the other side is the other group's.
A good boundary splits the two groups cleanly, with no point of either color crossing it. Draw the line off to one side and the colors get mixed, so its position matters.
Tap to move the boundary. Off to one side, points of the same color cross the line; placed in the middle, the two groups split cleanly.
The boundary has split the two groups. Now you can say clearly that the left of the line is one group and the right is the other. A rule has replaced the rough guess.
Real data often does not split this cleanly. In that case, you look for the line that sorts the fewest points wrongly, adjusting its slope and position bit by bit to reduce the number of mistakes, the same idea as shrinking the mismatch in the last lesson. So which group should a brand-new point go into?
A new point goes by which side it is on
Say a new email arrives. You do not yet know whether it is spam, but by measuring its features you can tell where it would sit. All you need to check is which side of the boundary that spot is on.
If it lands on the left, it belongs to the left group; on the right, to the right group. Many classifiers go one step further and say that the farther a point is from the boundary, the more likely it is to belong to that group. A point right next to the boundary has a probability close to fifty-fifty, which makes it a confusing case.
Tap an empty spot to place a new point. The side of the boundary it lands on decides its color, that is, its group.
Each new point got its color from where it landed: left of the boundary, the left color; right of it, the right color. The rule is simple and consistent, so even if a hundred new points come in, you sort them one by one with the same rule.
But the world does not always have just two groups. What about three or more, such as cats, dogs, and birds?
Three or more groups need more boundaries
With three groups, one line is not enough, because a single line only splits the space in two. In that case you draw more boundaries and divide the space into three or four regions.
Each region becomes one group's territory, and a new point is sorted in the same way, by which region it lands in. A classifier that answers with probabilities gives each group a probability and picks the biggest. The idea of probability from earlier is used just as it is.
Tap add boundary. The space splits from two regions into three and then four, and each group gets a region of its own.
Even with more regions, the rule stayed the same: draw more boundaries to divide the space, and sort a new point by which region it is in. Whether there are two groups or four, the underlying idea is the same.
The boundary does not have to be a straight line, either. When groups are tangled together in winding shapes, a curved boundary is needed, and the neural networks you will learn about later can draw complicated boundaries like that.
Summary
To sum up: classification decides which of a set of fixed groups something belongs to. Unlike regression, which predicts a value, it answers with the name of a group. You draw a boundary between two groups and sort a new point by which side of the boundary it is on.
With three or more groups, you draw more boundaries. Deciding the group from a single position and, when needed, saying how sure you are with a probability: that is classification.
Tap the key points in order to review. (split into two groups → a boundary between them → a new point goes by its side → more boundaries for three or more)
Now you have seen both kinds of prediction, regression and classification. One answers with a number and the other with a group, but both find a line in the examples and use it on new inputs.
To find a good line, though, you first have to measure how wrong the current line is. The next lesson, on loss, covers how to express that wrongness as a single number.