Large language models are built from many engineering pieces, but the central idea is not mystical. A model receives a sequence of tokens, turns those tokens into vectors, lets each token read information from other tokens, transforms the result through learned layers, and produces scores for what token should come next. The transformer architecture made that process scale unusually well because the main communication step is self-attention rather than recurrence.

In older recurrent neural networks, text was processed one step at a time. The representation after word 12 depended on the representation after word 11, which depended on word 10, and so on. That sequential dependency made long-range context harder to preserve and also limited parallelism during training. Transformers changed the shape of the computation. Instead of passing a hidden state forward through the sentence one token at a time, every token can compare itself with every other token in the same layer. Those comparisons produce attention weights, and the weights decide how much information flows from each token into each other token.

This article focuses on the core mechanism behind that shift. It does not try to cover every detail of production LLMs, training infrastructure, tokenizers, reinforcement learning, or retrieval. The goal is to make the transformer block itself easier to reason about: tokens become vectors, attention mixes information across positions, feed-forward layers refine each position, and the final vector produces next-token probabilities.

Tokens Become Vectors

A transformer does not directly process words as strings. It processes tokens, which are pieces of text from a vocabulary. A token can be a whole word, part of a word, punctuation, whitespace-like structure, or another learned text fragment depending on the tokenizer. For a conceptual explanation, it is fine to start by imagining one word per token, as long as you remember that real LLM tokenization is more granular.

Each token is mapped to an embedding vector. The embedding is a list of numbers learned during training. Nearby directions in this vector space often correspond to useful patterns: topic, grammatical role, style, entity type, or other features that help predict text. The model is not storing one hand-written definition per token. It is learning numerical coordinates that are useful for the task of predicting the next token in many contexts.

The same token can appear in many different contexts, so the initial embedding is only a starting point. The word “bank” in “river bank” and “bank account” begins with the same token embedding, but after attention layers it can carry different contextual information. That contextualization is one of the main jobs of the transformer stack.

Position Has to Be Added

Self-attention compares tokens as a set of vectors. By itself, that comparison does not know whether a token came first, last, or somewhere in the middle. Language depends on order, so transformers add positional information to the token representation.

There are several ways to do this. Some models add learned position embeddings. Some use sinusoidal patterns. Many modern architectures use relative or rotary position methods that affect attention comparisons directly. The implementation details differ, but the reason is the same: the model needs a way to distinguish “the dog bit the man” from “the man bit the dog.” The words are the same, but the positions change the meaning.

Once token and position information are combined, each position has a vector that says roughly: this is the token, and this is where it appears in the sequence. That vector becomes the input to attention.

Attention Lets One Token Read Other Tokens

The simplest way to understand attention is to choose one token and ask what other tokens it should read from. If the sentence is “The cat sat on the mat because it was warm,” the token “it” may need information from “mat” or “cat” depending on the intended meaning. Attention gives the model a flexible way to distribute influence instead of forcing every position to use the same fixed neighborhood.

In the visualization below, click a token in the sentence. The curved lines show how strongly that token reads from the other tokens. The weights shown here are explanatory rather than taken from a trained LLM, but they preserve the structure that matters: for one selected token, the weights over source tokens add up to one, and stronger weights contribute more to the new representation. This first diagram shows what attention weights do after they have been computed. The next section explains how a real attention head produces those weights from query and key comparisons.

This is the core operation: each token builds a weighted mixture of information from the sequence. The selected token does not copy exactly one other token. It receives a blend. Some weight may stay on itself. Some may go to neighboring words. Some may go to a noun, verb, separator, or earlier phrase that helps interpret the current position.

That makes attention different from a fixed local filter. Convolution, discussed in convolution and filtering for images and signals, uses a fixed kernel pattern that slides across the input. Attention computes a new pattern from the content of the tokens themselves. Both are weighted sums, but convolution uses mostly fixed local weights, while attention uses data-dependent weights that can connect distant positions directly.

Query, Key, and Value Vectors

The phrases query, key, and value sound more abstract than the operation really is. The previous visualization showed a row of attention weights as the visible result. It did not use the exact numeric weights from the geometric visualization below; both diagrams use separate simplified examples to show the same mechanism at different levels. For each token, the model learns three projections of its current vector:

• the query asks what this position is looking for
• the key describes what this position can be matched by
• the value contains the information this position will contribute if another token attends to it

For one token reading from the sequence, the model compares its query with every key. A larger query-key match gives a larger score. Those scores are passed through softmax, which turns them into attention weights. The weights are then used to average the value vectors.

Mathematically, a common single-head attention pattern is:

$$\mathrm{Attention}(Q, K, V) = \mathrm{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V$$

The matrix $QK^T$ contains query-key dot products. The scaling term $\sqrt{d_k}$ keeps the scores from becoming too large as vector dimensions grow. The softmax converts each row of scores into a probability-like weight distribution. Multiplying by $V$ then forms the weighted mixture of value vectors.

The next visualization shows the same idea geometrically. Drag the query arrow and watch how its dot product with each key changes. Keys pointing in a similar direction receive larger scores, and softmax turns those scores into the attention weights shown on the right.

The important separation is that keys and values have different jobs. A key is used for deciding where to read from. A value is what gets read and mixed after the decision has been made. This separation lets the model learn matching features and carried information independently. A token might be easy to match because it plays a certain grammatical role, while its value vector may carry semantic details needed later.

Multi-Head Attention Reads Several Kinds of Context

A single attention head produces one set of weights. That is useful, but language often needs several relationships at once. One head might focus on nearby syntax. Another might connect a pronoun to a noun. Another might track punctuation, phrase boundaries, or instruction structure.

Multi-head attention handles this by running several attention mechanisms in parallel. Each head has its own learned query, key, and value projections. The heads produce different value mixtures, and those mixtures are combined into the next representation.

This is not merely a visual convenience. It is one reason transformer layers can represent rich relationships. Instead of asking one attention distribution to capture every dependency in a sentence, the model can split work across heads. The heads are not guaranteed to map cleanly onto human-readable categories, and real models often contain heads whose behavior is hard to summarize. Still, the engineering purpose is clear: parallel attention heads give the layer multiple learned ways to move information across the sequence.

Attention Replaces Recurrence With Parallel Communication

The original transformer design was important because it removed the recurrent bottleneck from sequence modeling. In a recurrent model, token 50 depends on token 49’s hidden state, so training has an unavoidable sequential path through time. In a transformer layer, all token-to-token attention scores for a sequence can be computed with large matrix multiplications. Those matrix multiplications run efficiently on GPUs and other accelerator hardware.

This does not mean transformers are free. Full self-attention over a sequence has a cost that grows roughly with the square of the sequence length because every token can compare with every other token. Long-context models therefore need careful engineering, specialized attention variants, caching during generation, and memory-efficient kernels. But the basic transformer was still a major scaling improvement because it made the dominant training computation highly parallel.

The result is a useful trade: attention spends more computation comparing positions directly, but it exposes that computation in a form hardware can process efficiently. That hardware-friendly structure is one reason transformers became the foundation for modern LLMs.

The Feed-Forward Layer Refines Each Position

Attention is the communication step, but it is not the whole transformer block. After attention mixes information across positions, each position usually passes through a feed-forward network. This part is applied independently to each token position. It does not move information between tokens directly. Instead, it transforms the representation at each position after attention has already brought in relevant context.

A standard transformer block also uses residual connections and normalization. A residual connection adds the block input back to the block output, making optimization easier and helping information flow through many layers. Normalization keeps activations in a manageable range. Those details are not cosmetic; deep transformer stacks rely on them for stable training. But conceptually, the block rhythm is simple: mix information across tokens with attention, then process each position with a learned feed-forward transformation.

Stacking many blocks repeats that rhythm. Early layers may build local syntactic and lexical features. Middle layers may combine phrases and entities. Later layers may form representations that are directly useful for predicting the continuation. This is a rough intuition, not a strict rule, but it explains why a transformer is more than a single attention map. The model repeatedly updates every token representation in context.

From Final Vector to Next-Token Probabilities

Language models are trained to predict tokens. During generation, the model receives the current context and produces a vector at the final position. That vector is projected into one score per vocabulary token. These raw scores are called logits.

Logits are not probabilities yet. A logit can be any real number. Softmax converts the logits into probabilities by exponentiating and normalizing them:

$$P(t_i) = \frac{e^{z_i / T}}{\sum_j e^{z_j / T}}$$

Here $z_i$ is the logit for token $t_i$, and $T$ is temperature. Lower temperature makes the distribution sharper, so the highest-logit options dominate. Higher temperature flattens the distribution, increasing the chance of less likely tokens.

The stepper below follows the whole path at a simplified scale. Click each stage to move from token embeddings to next-token probabilities. The example keeps one context fixed so the visualization can focus on how the representation becomes logits and then probabilities.

Real LLMs use enormous vocabularies and many layers, so the displayed candidate list is tiny compared with production models. The structure is the same, though. The model does not search a database of completed answers. It repeatedly computes a probability distribution for the next token, chooses or samples a token according to a decoding rule, appends that token to the context, and runs the process again.

This repeated next-token prediction can produce long, coherent text because the context keeps growing. Each new token becomes part of the input for the next step. Attention then gives future positions access to earlier words, instructions, names, code symbols, and partial reasoning traces inside the context window.

Causal Masking During Generation

One detail matters for language generation: when predicting the next token, the model must not look into the future. Training examples contain full text sequences, but the prediction at position 10 should only depend on positions 1 through 9. Transformers enforce this with a causal mask. The mask prevents a token from attending to later tokens during language-model training and generation.

Without this mask, the model could cheat during training by reading the answer token directly. With the mask, each position learns to predict the next token from the prefix available before it. That is why decoder-only transformers used for many LLMs are often called causal language models. The attention mechanism is still self-attention, but its allowed connections are restricted so information flows only from earlier positions to later positions.

Encoder-style transformers, such as those used for some classification or embedding tasks, can allow tokens to attend bidirectionally because they are not being trained to generate the next token in the same way. The attention mechanism is shared, but the mask and objective change the behavior.

What Attention Does and Does Not Explain

Attention is the core communication mechanism, but it is not the whole explanation for LLM behavior. Tokenization affects what the model sees. Training data determines what patterns are available to learn. Optimization, scale, architecture details, normalization, activation functions, context length, decoding settings, and post-training all shape the final system.

Still, attention is the right place to start because it explains how the model builds contextual representations. Every token begins as a vector. Position information gives the model sequence order. Query-key comparisons decide how strongly each token reads from other tokens. Value vectors carry the information being mixed. Feed-forward layers refine each position. Logits and softmax turn the final representation into a next-token distribution.

Once that pipeline is clear, many explanations of LLMs become easier to place. “The model uses context” means attention can move information from earlier tokens into the current representation. “The model predicts the next token” means the final vector is converted into logits over the vocabulary. “The model scales well” partly means the transformer replaces a sequential recurrence path with parallel matrix operations over the sequence.

Summary

Transformers made modern LLMs practical because self-attention gives every token a flexible way to read from other tokens while remaining highly parallel during training. The mechanism can be summarized in a few steps:

1. Text is split into tokens.
2. Tokens become embedding vectors, and position information is added.
3. Each token forms query, key, and value vectors.
4. Query-key scores become attention weights.
5. Attention weights mix value vectors into contextual representations.
6. Feed-forward layers refine each position.
7. The final position produces logits, and softmax turns them into next-token probabilities.

The essential mental model is weighted information flow. Attention does not make a token understand the whole sentence by magic. It gives the model a learned, content-dependent way to decide which other tokens matter at each layer. Stack that operation many times, train it over large text corpora, and the result is the central architecture behind today’s large language models.