What does it mean to learn?

A person writes the rules

Earlier we learned a program
follows commands a person wrote.
To make it do anything, you had to write rules.
Think of filtering out spam mail.
If it has the word sale, throw it out.
If it has too many links, throw it out.
Write rules like these one by one
and it seems it should filter just fine, right?

Toggle rules on and off to filter the mail. You will see that rules alone keep missing fresh spam.

Turning on more rules did help a bit.
But spam keeps changing its look.
Instead of sale it writes free gift,
and hides links inside pictures.
Each time you must write a new rule,
so the rules only grow without end.
A person writing each one by hand
cannot keep up with the world's whims.
It looks like we need a different road.

Find the rule from examples

Now let's flip the road.
Instead of writing the rules,
we show heaps of examples with the answers.
This one is spam, this one is fine.
Again, this is spam, this is fine.
See many labeled examples like that
and the machine finds the common thread itself.
What the spam ones share,
that pattern is the rule the machine made.

Show the labeled examples one by one. As they pile up, the common pattern the machine pulls out gets clearer.

As examples piled up, the pattern got clearer.
A person did not write the rule;
the machine found it inside the examples.
What is striking is that even a difference
we can hardly point to in words
gets caught if there are enough examples.
Still, the first pattern it finds is rough.
It will sort a few of them wrong.
So how do we polish this pattern?

It fixes itself bit by bit

Learning is not finished in one go.
The machine sees an example, tries an answer,
and if wrong, checks where it went wrong.
Then it tweaks the pattern just a little.
Wrong on the next example, it fixes again.
Repeat this countless times
and the number of misses keeps dropping.
The more examples you give,
the more the pattern is polished toward fitting well.

Feed one more batch of examples each time. With every tap the machine fixes its pattern and accuracy climbs a notch.

More examples, higher accuracy.
It fixes as much as it gets wrong, again and again.
The pattern that was clumsy at first
has become quite trustworthy by now.
Of course it does not grow perfect forever.
Past some point it improves slowly.
Still, with far less effort than
writing rules by hand, we have come a long way.
So what do we do with this polished pattern?

It predicts new inputs too

The real reward of learning is here.
The examples used for training already have answers.
But the true use comes
when it meets a new input never seen before.
The machine holds it against the pattern it polished
and judges, is this spam or fine.
Because it found a pattern rather than memorized,
it gives an answer even for a first-seen mail.
Guessing ahead like this is called prediction.

Tap a new mail that was not used in training. The machine predicts spam or fine using the pattern it learned.

It sorted even unseen mail with ease.
It did not pull out a memorized answer.
It applied the pattern found in examples to the new input.
So even when the shape is a bit different,
it can judge by what looks alike.
Of course it is wrong sometimes.
The pattern is not perfect.
Still, far more flexibly than a person writing every rule,
it keeps up with the world's changes.

Let's wrap up

Gathered on one line, it is this.
An ordinary program runs on rules a person wrote.
But the world has far too many rules.
So machine learning flips the road.
Show it heaps of examples with answers
and the machine finds the common pattern, the rule.
Wrong, it fixes bit by bit and gets better,
and with that pattern it even predicts new inputs.
Learning from examples instead of writing rules, that is the heart of it.

Tap the key points in order to review. (person writes rules -> find rules from examples -> fix by getting wrong -> predict new inputs)

Now you know the real meaning
of a machine learning.
Instead of a person teaching every bit,
it looks at examples and finds the rule itself.
This one simple idea
is the starting point of today's smart machines
that recognize photos and understand speech.
Next, let's go on to see how this learning
actually runs on the inside.