Everyone learns neural networks in Python. I decided to build a full LLM from scratch in Node.js instead, and write down everything that confused me along the way. Not because TypeScript is better, and not to replace Python. It is just that rebuilding an idea in a different language is what makes it stick.
This is the smallest piece of the whole machine: one tiny network, 3 neurons, 8 learnable numbers. By the end of this post I'll show you something that genuinely fascinated me: your entire network, no matter how deep, collapses into one single equation.
Let's build it.
The smallest network that still counts
Our mini network: an input row with 3 features, then two layers holding 3 neurons in total. Two in the first layer, one in the second.
Every neuron follows the same 4-step recipe. Multiply each input by its own weight, add them up, add a bias, then squash the result:
That's it. That is everything a neuron does. The whole field runs on this recipe repeated at scale.
Run real numbers through it
Take x = [1, 2, 3] and some hand-picked weights:
| Neuron | Weights | Bias | Weighted sum | After ReLU |
|---|---|---|---|---|
| 1 | [0.5, -1, 0.5] | 0.1 | 0.5 - 2 + 1.5 + 0.1 = 0.1 | 0.1 |
| 2 | [-1, 0, -0.5] | 0 | -1 + 0 - 1.5 = -2.5 | 0 |
Notice neuron 2: negative sum, so ReLU flattened it to a hard zero. This is why real network activations are full of zeros.
And the entire network is three calls:
ReLU is not a layer
Quick rant, because this confused me for sometime. ReLU has no weights and nothing to learn. It is an operation applied to each number on its own.
People call it a "layer" because that is how frameworks expose it (looking at you, tf.layers.reLU()), but there are zero parameters inside it.
And here is why it exists at all: without it, stacking layers is pointless. Two weighted sums back to back collapse into one weighted sum. The squash in the middle is the only thing that makes "deep" mean anything.
The moment it clicked for me
Here is the interesting part. Take layer two's equation and plug layer one into it:
The whole network folds into a single equation. Input goes in, one number comes out. The network IS that equation, and every weight and bias inside it is a knob we are allowed to turn.
Once you see this, "training" stops being magic. We know what the network produced (ŷ) and what it should have produced (y).
Measuring the gap is the loss function:
loss = (ŷ - y)²
Training is nothing more than turning the knobs until that gap collapses toward zero. One number, sitting outside every layer, driving the updates inside all of them at once.
Why I care
This tiny network is the same recipe that runs inside GPT, just with 175 billion more knobs. The attention block, the feed-forward block, all of it: weighted sum, bias, squash, repeat.
I'm documenting the whole journey, building an LLM from scratch in Node.js and TypeScript, chapter by chapter. Fully runnable code, zero black boxes:
📖 The free book: book.humanexperiences.io
⭐ The code: github.com/BaraaSoft/node-llm
Next up: LayerNorm, or why stacking these layers makes numbers explode 58× by layer 10, and the elegant fix every transformer uses.
What confused YOU most when you first learned neural networks? Drop it in the comments, there's a good chance I got stuck on the exact same thing.
