A large language model is a transformer trained to predict the next token in a sequence. That single objective, applied to trillions of tokens, produces a system that can summarise, translate, answer questions and write code — because all of those tasks can be expressed as “continue this text sensibly”.
How it works
- Tokenisation. Text is split into sub-word tokens by a scheme such as byte-pair encoding, so any string — including code and rare words — can be represented by a fixed vocabulary.
- Embedding and position. Each token becomes a vector. Positional information is added (learned embeddings, or rotary embeddings in most current models) because attention itself is order-agnostic.
- Self-attention. Every token computes queries, keys and values, and attends to the tokens it cares about. This is what lets a pronoun resolve to a noun forty lines earlier.
- Feed-forward blocks. Attention layers alternate with position-wise MLPs, wrapped in residual connections and normalisation, repeated dozens to hundreds of times.
- Decoding. The final layer produces a probability distribution over the vocabulary; sampling parameters such as temperature and top-p decide how adventurous the next token is.
Concepts worth knowing
- Context window — how many tokens the model can attend to at once. Larger windows allow whole codebases or long documents in a single prompt, at higher cost.
- Mixture of experts — only a subset of the network's parameters is activated per token, so a model can be very large in total capacity while remaining cheap to run.
- KV caching — keys and values from earlier tokens are reused during generation, which is why the first token of a response is slow and the rest arrive quickly.
- Quantisation — storing weights in 8- or 4-bit formats shrinks memory needs enough to run capable open-weight models on a single GPU or a laptop.
- Reasoning modes — models trained to produce intermediate reasoning before answering trade latency and tokens for accuracy on maths, planning and multi-step problems.
Practical notes
- Cost and latency scale with tokens in plus tokens out; trimming prompt boilerplate is often the cheapest win.
- Models are stateless between calls — conversation memory is just prior turns replayed in the prompt.
- Confident, fluent text is not evidence of correctness. Ground factual claims with retrieval or tool calls.