How does an LLM work
A large language model, or LLM for short, writes, translates and even writes code. It looks like magic, but inside it does only one thing: it looks at the text so far and guesses what comes next. Chain those guesses together one at a time and you get a sentence, then a long answer. The job is done by a transformer, the stack of attention layers from the last lesson. Let's check, step by step, whether that is really all there is.
Guessing the next word
Think about typing a message on your phone. You type a word and the keyboard offers a few words that might come next. An LLM does essentially the same thing, except that it has learned from far more text and looks at a far longer stretch of what came before.
An LLM reads everything so far and picks one word to come next. It attaches that word, rereads the longer text, and picks the one after that. Strictly speaking it works in tokens, pieces that are often smaller than a word, but here we will simply call them words.
Tap a candidate word to continue the sentence. Each tap shows the next candidates with their probabilities, and the sentence grows one word at a time.
All we did was pick the next word, yet a sentence that makes sense appeared. The secret is the number beside each candidate: its probability, which shows how likely that word is to come next.
Words that fit the grammar and the meaning get high probabilities; odd words get low ones. An LLM works out these probabilities from what it learned, across huge amounts of text, about which words tend to follow which.
Picking one by probability
Probability is not hard. Each word that could come next gets a score for how plausible it is, and the scores are scaled so that together they add up to one hundred percent. After "the sky is clear", a word like "today" gets a high probability and "loud" a low one.
An LLM usually picks a high-probability word. It does not always take the top one, though; often it draws a word in proportion to the probabilities, a bit like a weighted lottery. That is why the same question can get slightly different answers.
At each step, tap the word with the highest probability. The chosen word joins the sentence and you move on to the next step. The candidates are fixed in advance, so you get the same result every time.
Each pick made the sentence one word longer. A real LLM rereads the whole longer sentence and works out fresh candidates for the next word.
Pick, attach, reread, pick again. Repeat this short move countless times and you get paragraphs and long answers. An LLM's text does not arrive all at once; it piles up one step at a time.
Earlier context changes the probabilities
Next-word probabilities are not fixed. They change a lot depending on what came before. After "I was hungry, so I reached for", something to eat sounds right; after "the boat reached the harbor, so I reached for", something from the boat does.
The words right before the gap are the same, yet a different context completely changes what comes next. Picking out the useful words from earlier in the text is the job of attention, from the last lesson.
Tap to change the earlier context. Even though the words right before the gap stay the same, a different context reshuffles the whole set of next-word probability bars.
Changing just the earlier context flipped the next-word probabilities. This is the secret of why an LLM seems smart: it looks over the whole context and picks the next word that fits the situation.
That is also why the same question can get quite different answers depending on the explanation and examples you put in front of it, and why writing a good prompt matters.
Many abilities from one task
Here something surprising happens. The model was trained only to guess the next word well, yet abilities nobody taught it come along. After a long text followed by "to sum it up,", a summary is the most plausible continuation; after "in English, that is", a translation is.
After a description of a problem followed by "as code:", code follows. Abilities like these, which show up on their own as models grow larger, are sometimes called emergent abilities.
Tap to switch the ability. Summary, translation and coding all come from the same next-word guessing. Only the lead-in differs; the machine is doing one job.
Switching buttons made the task look different, but inside it is all the same: look at the lead-in and guess the next word. Polishing this one skill on enormous amounts of text brought many abilities along with it. Acting like a helpful assistant that answers questions comes from a later round of training on people's ratings of its answers.
One point is worth remembering. An LLM does not check facts before it answers; it picks the most plausible next word. So it can state a book title that does not exist, or a wrong number, with complete confidence.
Wrapping up
In short, an LLM is a machine that looks at the context and guesses the next word by probability. It gives each candidate a probability, picks one, and builds long text one word at a time. When the context changes, the probabilities change, and attention decides what to look at in the earlier text.
Getting good at this one task brought along abilities such as summarizing, translating and coding. But what an LLM picks is plausibility, not truth, and that is worth keeping in mind.
Tap the key points in order to review. (guess the next word → pick by probability, one word at a time → changes with context, attention → many abilities come along)
Now you can see what an LLM really is: not magic, but a machine that guesses the next word by probability. Doing this simple job at an enormous scale produced unexpected abilities.
Is text the only thing AI makes, though? There are also AIs that create pictures and music from what they have learned. The next lesson covers generative AI, which uses learned patterns to make something new.