Stacking neurons: the neural network

Connect neurons in a stack

Earlier we learned a single neuron.
It gathers inputs by weights and adds them,
and if the sum passes a mark, it fires.
Simple, and its judgment is just as simple.
It can only split things by one line.
But what if we feed one neuron's output
into another neuron's input?
Neurons pass signals back and forth
and can build a bigger picture step by step.

Tap neurons to turn them on and connect them. Lit neurons grow signal lines, so simple neurons gather into a net.

Each neuron is still simple.
What changed is that they are connected.
One neuron's output
becomes the next neuron's input,
so what one alone could not see,
many together begin to see.
But rather than connecting at random,
lining them up in rows is much cleaner.
We call such a row a layer.

Input, hidden, and output layers

Stand neurons in rows and you get layers.
The very front row takes values from outside.
We call it the input layer.
The very back row gives the final answer.
We call it the output layer.
The rows in between
are the hidden layers that do most of the work.
A signal starts at the input layer
and flows through each layer to the output.

Tap the button to flow the signal forward one layer at a time. It lights from input through hidden to output in order.

The signal flowed only forward.
Each layer takes the previous output
and recomputes it in its own way to pass on.
Front layers inspect small pieces,
and back layers gather those pieces
to recognize a bigger shape.
The deeper the layers go,
the more abstract the features they catch.
But there is a hidden trap here.

There must be a bending step between

Just stacking layers seems enough, right?
But there is a trap.
If each layer only adds and multiplies,
that is a straight, linear transform.
No matter how many straight transforms you chain,
the result is just another straight line.
Stack a hundred layers and it equals one layer.
So between layer and layer
we truly need a step that gently bends the signal.

Toggle the bending step on and off to compare. Off, stacked layers collapse into one straight line; on, the bent curves survive and the shape grows rich.

The difference jumped out, right?
With the bending step off,
no matter how many layers, it is one straight line.
What it can do equals a single layer.
With the bending step on,
the signal bends a little at each layer,
and shapes appear that straight lines could never draw.
Thanks to this bending step
stacking layers finally becomes worthwhile.

More layers, more complex patterns

Shall we add more layers with bending steps?
With one layer, just a single straight line
can split the world into two.
Add a layer and bent lines combine
to draw a kinked, curved boundary.
Stack even more
and it learns circles or wiggly boundaries,
more and more complex patterns.
This is why neural networks are powerful.

Tap to add layers. With few layers the boundary is straight; the more you add, the more the dotted boundary curves to wrap a complex pattern.

Each added layer freed the boundary more.
From a straight line
to a kinked line, then to a curved one.
Simple neurons form layers,
and with bending steps tucked between,
a pattern none could dream of alone
they learn together.
This is exactly the skill of a neural network.
Now let's wrap the whole thing up.

Let's wrap up

Gathered on one line, it is this.
Stack simple neurons in layers and connect them.
The signal flows from input layer to output.
Each layer takes the previous output
and catches a bigger feature.
Between layers there must be a bending step.
Otherwise it all collapses into one straight line.
Stack the layers and bend between them,
and the network learns complex patterns. That is it.

Tap the key points in order to review. (stack neurons in layers -> signal flows forward -> need a bending step between -> more layers, more complex patterns)

Now you know how simple neurons gather
to learn complex patterns.
A neuron that alone could draw just one line,
once it forms layers and bends between them,
learned even hard boundaries like curves.
This simple idea, stack and bend,
is the foundation of today's clever machines.
Next, let's keep exploring how this network
gets better on its own.