How it works
Text is split into tokens — subword fragments, roughly three quarters of a word each in English. Each token becomes a vector, and the model passes those vectors through stacked transformer blocks whose attention layers let every position weigh every other position in the context window. The final layer emits a probability distribution over the vocabulary for the next token; a sampler picks one, appends it, and the whole forward pass runs again.
That single mechanism is the entire behaviour at inference time. Pretraining sets the parameters that make those predictions useful. A second stage — instruction tuning and preference optimisation such as RLHF — reshapes the distribution so the model answers rather than continues, and refuses rather than complies with certain requests.
Example
Given "The capital of France is", the model does not consult a geography table. It produces a distribution in which "Paris" holds most of the probability mass because that continuation dominated its training data. Given a question about an obscure company's 2026 revenue, the same machinery produces a confident-looking number with no equivalent support behind it. Nothing in the architecture distinguishes the two cases for the model.
Why it matters
Almost every practical decision about deploying these systems follows from the prediction mechanism. Retrieval exists because the model has no memory it can be trusted to recall. Evaluation is statistical because outputs are sampled rather than computed. Cost scales with tokens because tokens are the unit of work. Teams that model an LLM as a search engine or a database consistently mis-specify their systems.