Memorizing vs truly understanding: overfitting
Imagine a friend who, the night before a test, memorized all the answers in the workbook. If the same questions appear, they get every one right. But the moment a new question shows up with the numbers slightly changed, they fall apart. They only memorized; they never understood the principle. Machine learning has the very same trap. If a model memorizes the training data whole, it scores 100 on what it saw but misses new problems. We call this overfitting. So how do we measure real skill?
Memorize it whole and training score is perfect
Here are some scattered points.
The model's job is to find a smooth curve
that explains these points.
But we could get greedy
and bend the curve up and down
so it passes through every single point.
Then all the training points sit on the curve.
The training score is a flat 100.
It looks perfect, but really it just memorized.
Tap the button to switch on a wiggly curve that memorizes the points whole. It passes through every point, so training accuracy hits 100 percent.
The curve hit every point exactly.
The training score became 100.
Feels great, but we must not stop here.
This curve is close to having simply
memorized where the points are.
It did not catch the real pattern
that flows between the scattered points.
Why memorizing is a problem
shows up the moment we throw it a new question.
When new data arrives, it crumbles
The real test of a memorized curve
is a new point it has never seen.
The new point came from the same pattern
as the training points.
So a smooth curve should roughly
get the new point right too.
But the memorized curve
swings wildly between the training points,
so near a new point it points to an absurd height.
A hundred on what it saw, a zero on the new.
Tap an empty spot to drop in a new point. The memorized curve misses it by a lot, revealing how weak it is on data it never saw.
At the new point the curve missed by a lot.
So perfect on the training points,
yet helpless before a point it never saw.
This is what overfitting really is.
It is just like the friend who memorized
the answer sheet and collapses
when the numbers change.
So what we truly want to know
is not the training score but the score on unseen data.
Split into train, validation, and test
To measure skill on unseen data
we must hide some data away in advance.
So we split what we have into three.
The training data is what the model learns from,
and the validation data lets us check
midway whether things are going well.
The test data we do not touch at all
until the very end, looked at just once.
Only this way can we grade in a way
that mimics truly new problems.
Tap points to sort them into train, validation, and test. Once the three bins fill up evenly, you are ready to evaluate on the unseen test data.
We split the data into three groups.
The key is to never show
the test data during learning.
If you solve the exam questions beforehand,
it is no longer an exam.
Learn with training,
steer midway with validation,
and grade the real score with test at the end.
That score is the model's true skill.
Not too simple, not too complex, just right
Now let's compare three kinds of model.
A model that is too simple
tries to settle for one straight line
and cannot follow the curve the points trace.
That is underfitting, having learned too little.
A model that is too complex
memorizes the points and so is weak on new data.
The just-right model in between
gently catches the pattern that flows through the whole,
rather than each single point.
Tap the three models in turn to compare. The underfit one is too straight, the overfit one wiggles from memorizing, and only the just-right one does well on new data too.
The difference between the three was clear at a glance.
Too simple and it misses the pattern,
too complex and it memorizes and wobbles.
Only the just-right model
did well evenly on both training
and unseen data.
Good learning is not about pushing
the score up to 100,
but doing well steadily on data it has not seen.
Let's wrap up
Gathered on one line, it is this.
A wiggly curve that memorized every point
scores 100 only on training and is weak on new points.
That is overfitting.
True understanding is catching the pattern that runs through the whole.
So we split data into train, validation, and test
and measure skill on data it has not seen.
Neither too simple nor too complex,
the just-right amount generalizes best.
Tap the key points in order to review. (memorize whole and only training is 100 -> fails on new data -> split into train/validation/test -> measure real skill on unseen data)
Now you know the difference between memorizing and truly understanding.
A high training score is not what makes it good.
What matters is not wobbling
even before new data it has never seen.
So we split the data
and grade honestly on data it did not see.
Catch patterns rather than memorize,
this one thing is the most important mindset
for building a clever machine.