What a language model is actually doing

A large language model does one thing: given some text, it estimates which small piece of text (a token) is likely to come next. Everything else (answering questions, writing code, summarising) is that one step repeated.

1. Text becomes tokens

Models don't read letters or whole words. Text is split into tokens, common chunks of characters. "Learning" might be one token; an unusual name might be several. This is why models sometimes stumble at counting letters: they never saw the letters individually.

2. The model scores every possible next token

For the text so far, the model produces a probability for each token in its vocabulary. After "The capital of France is", the token " Paris" gets a very high probability. After "My favourite colour is", probability is spread across many options.

3. One token is chosen, then repeat

A sampling rule picks a token, often not simply the most likely one. Temperature controls how adventurous that choice is: low temperature gives predictable text, higher temperature more varied text. The chosen token is appended, and the whole process runs again.

prompt:  "The capital of France is"
step 1:  " Paris"   (p ≈ 0.92)
step 2:  "."        (p ≈ 0.71)
step 3:  <end>

Why this explains hallucinations

The model is optimised to produce plausible continuations, not true ones. When it has seen lots of reliable text about a topic, plausible and true usually coincide. When it hasn't, it still produces something fluent. That's why grounding answers in retrieved documents, and checking them, matters so much in real applications.

Try it yourself

• Ask a chatbot the same factual question three times with high temperature, then low temperature, and compare.
• Ask it to count the letter "r" in a long, unusual word, then ask it to spell the word with spaces first.
• Notice which answers change and which stay stable. Stable answers are usually ones the model has seen many times.

Next step: our Generative AI Engineering program turns this understanding into applications that are measurably reliable.