How Transformer Models Work: Self-Attention, Positional Encoding, and the Architecture Behind Modern LLMs

Every engineer working with LLMs today is operating on top of a transformer. You call an API, stream tokens, parse JSON, and ship. But when something goes wrong — the model loses context, produces hallucinated output at a specific position, behaves differently at high sequence lengths, or fails in ways that don’t match your mental model — the internals matter.

This is not an academic survey. This is the architecture explained at the level of depth you need to debug production systems, make informed decisions about model selection, and understand why scaling laws work the way they do.

The Input Pipeline: From Text to Vectors

Before any attention computation happens, raw text has to become a sequence of numbers.

Tokenization breaks text into tokens. A token is not a word — it is a subword unit produced by an algorithm like Byte Pair Encoding (BPE). “unhappiness” might become ["un", "happi", "ness"]. Each token maps to an integer ID in a vocabulary (typically 32K to 128K tokens for modern models). The exact tokenization scheme matters because it determines what the model treats as a single unit of meaning.

Token embeddings map each integer ID to a dense vector in d_model-dimensional space (512, 768, 1024, or 4096 are common values depending on model size). These vectors are learned parameters — the embedding matrix W_E has shape [vocab_size, d_model]. At initialization they are random; after training they encode semantic relationships between tokens.

Positional encoding is where the transformer has to work around its own architecture. Unlike RNNs, the transformer processes all tokens in parallel with no built-in notion of order. Position must be injected explicitly.

The original paper used fixed sinusoidal encodings:

import numpy as np

def sinusoidal_positional_encoding(seq_len: int, d_model: int) -> np.ndarray:
    """
    Returns shape [seq_len, d_model].
    Even dimensions use sine, odd dimensions use cosine.
    """
    PE = np.zeros((seq_len, d_model))
    positions = np.arange(seq_len)[:, np.newaxis]          # [seq_len, 1]
    div_term = np.exp(
        np.arange(0, d_model, 2) * -(np.log(10000.0) / d_model)
    )                                                       # [d_model/2]

    PE[:, 0::2] = np.sin(positions * div_term)
    PE[:, 1::2] = np.cos(positions * div_term)
    return PE

Each position gets a unique vector with a different frequency pattern per dimension. The model can infer relative distances because sin(pos + k) can always be expressed as a linear function of sin(pos) and cos(pos).

Modern models use learned positional embeddings (GPT-2, BERT) or rotary positional embeddings (RoPE, used in LLaMA, Mistral, and most post-2022 models). RoPE encodes position directly into the query and key vectors inside each attention head rather than adding it to the input — this gives better extrapolation beyond trained context lengths.

The final input to the first transformer block is token_embedding + positional_encoding, a tensor of shape [batch_size, seq_len, d_model].

Self-Attention: The Core Mechanism

Self-attention lets every token look at every other token in the sequence and decide how much to attend to each one. This is what gives transformers their ability to handle long-range dependencies that destroyed RNNs and LSTMs.

Q/K/V Matrices

For each attention head, the model projects the input into three separate spaces using learned weight matrices:

• Q (Query): what this token is looking for
• K (Key): what each token is announcing it contains
• V (Value): what each token actually contributes if selected

import torch
import torch.nn as nn
import math

class SingleHeadAttention(nn.Module):
    def __init__(self, d_model: int, d_k: int):
        super().__init__()
        self.d_k = d_k
        self.W_q = nn.Linear(d_model, d_k, bias=False)
        self.W_k = nn.Linear(d_model, d_k, bias=False)
        self.W_v = nn.Linear(d_model, d_k, bias=False)

    def forward(
        self,
        x: torch.Tensor,        # [batch, seq_len, d_model]
        mask: torch.Tensor | None = None,
    ) -> torch.Tensor:
        Q = self.W_q(x)         # [batch, seq_len, d_k]
        K = self.W_k(x)         # [batch, seq_len, d_k]
        V = self.W_v(x)         # [batch, seq_len, d_k]

        # Scaled dot-product attention
        scores = torch.matmul(Q, K.transpose(-2, -1)) / math.sqrt(self.d_k)
        # scores: [batch, seq_len, seq_len]

        if mask is not None:
            scores = scores.masked_fill(mask == 0, float('-inf'))

        weights = torch.softmax(scores, dim=-1)
        # weights: [batch, seq_len, seq_len] — attention distribution
        return torch.matmul(weights, V)  # [batch, seq_len, d_k]

The critical operation is QK^T / sqrt(d_k). Each row of this matrix is a dot product between one query vector and all key vectors. Tokens with similar vector directions (high dot product) score highly. Dividing by sqrt(d_k) prevents the dot products from growing so large that softmax saturates into near-zero gradients.

After softmax, weights[i][j] is the attention token i pays to token j. This is a probability distribution over the sequence. Multiplying by V produces a weighted sum of value vectors — the output representation for each position.

Causal Masking in Decoder-Only Models

For autoregressive generation (GPT-style), the model must not see future tokens during training. This is enforced by setting all positions where j > i to -inf before softmax, which zeroes out those attention weights. In practice, this is an upper-triangular mask applied to the score matrix.

Multi-Head Attention

Running attention once gives the model one “perspective” on the sequence. Multi-head attention runs h parallel attention heads, each with its own W_q, W_k, W_v projections at reduced dimension d_k = d_model / h. The outputs are concatenated and projected back:

class MultiHeadAttention(nn.Module):
    def __init__(self, d_model: int, num_heads: int):
        super().__init__()
        assert d_model % num_heads == 0
        self.d_k = d_model // num_heads
        self.num_heads = num_heads

        self.W_q = nn.Linear(d_model, d_model, bias=False)
        self.W_k = nn.Linear(d_model, d_model, bias=False)
        self.W_v = nn.Linear(d_model, d_model, bias=False)
        self.W_o = nn.Linear(d_model, d_model, bias=False)

    def split_heads(self, x: torch.Tensor) -> torch.Tensor:
        batch, seq_len, _ = x.shape
        # [batch, seq_len, num_heads, d_k] -> [batch, num_heads, seq_len, d_k]
        return x.view(batch, seq_len, self.num_heads, self.d_k).transpose(1, 2)

    def forward(self, x: torch.Tensor, mask: torch.Tensor | None = None) -> torch.Tensor:
        batch, seq_len, _ = x.shape

        Q = self.split_heads(self.W_q(x))   # [batch, heads, seq_len, d_k]
        K = self.split_heads(self.W_k(x))
        V = self.split_heads(self.W_v(x))

        scores = torch.matmul(Q, K.transpose(-2, -1)) / math.sqrt(self.d_k)
        if mask is not None:
            scores = scores.masked_fill(mask == 0, float('-inf'))

        weights = torch.softmax(scores, dim=-1)
        attended = torch.matmul(weights, V)  # [batch, heads, seq_len, d_k]

        # Recombine heads
        attended = attended.transpose(1, 2).contiguous().view(batch, seq_len, -1)
        return self.W_o(attended)

Different heads learn to specialize. Some attend to syntactic relationships (subject-verb agreement), others to semantic coreference, others to position-relative patterns. This specialization is not designed in — it emerges from training.

Encoder-Decoder vs. Decoder-Only

The original transformer used both an encoder and a decoder. Modern LLMs mostly dropped the encoder. Understanding why requires knowing what each does.

The encoder reads the full input sequence with bidirectional attention (every token can see every other token) and produces a contextualized representation. BERT is an encoder-only model. It works well for classification and extraction tasks where you process a fixed input.

The decoder generates tokens autoregressively with causal (unidirectional) attention. It also has cross-attention layers that attend to encoder outputs. The original transformer decoder for translation looks like this:

1. Causal self-attention over tokens generated so far
2. Cross-attention: queries come from the decoder, keys and values come from encoder output
3. Feed-forward network

GPT-style decoder-only models remove the encoder and the cross-attention. Every layer is just causal self-attention plus a feed-forward block. The entire context (prompt plus generated tokens) is processed in a single forward pass through the stack. This is computationally simpler, scales well, and turns out to be extremely capable with enough parameters and data.

T5 and BART are encoder-decoder models that have a practical advantage for structured tasks: the encoder can process a long document bidirectionally while the decoder generates a shorter summary or translation. For open-ended generation at scale, decoder-only has won.

Feed-Forward Layers and Layer Normalization

Each transformer block contains two sub-layers:

1. Multi-head attention
2. A position-wise feed-forward network (FFN)

The FFN applies two linear transformations with a nonlinearity (ReLU in the original, GELU or SwiGLU in modern models):

class FeedForward(nn.Module):
    def __init__(self, d_model: int, d_ff: int):
        super().__init__()
        # d_ff is typically 4x d_model
        self.linear1 = nn.Linear(d_model, d_ff)
        self.linear2 = nn.Linear(d_ff, d_model)
        self.gelu = nn.GELU()

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        return self.linear2(self.gelu(self.linear1(x)))

The FFN operates on each position independently. Its role is often described as storing factual associations that attention retrieves. Research has found individual neurons in the FFN correspond to specific factual patterns (“Eiffel Tower” triggers neurons associated with “Paris”), though the picture is more distributed in practice.

Layer normalization is applied before each sub-layer (Pre-LN, used in most modern models) or after (Post-LN, the original paper). Pre-LN stabilizes training at depth. Each block follows the residual pattern:

x = x + Attention(LayerNorm(x))
x = x + FFN(LayerNorm(x))

The residual connection means gradients flow directly through the stack. Without it, training deep transformers is unstable.

Training: Next-Token Prediction and Backpropagation

GPT-style models train on next-token prediction. Given a sequence of tokens, the model predicts the distribution over the vocabulary for the next token at every position simultaneously. This is teacher forcing — the correct previous tokens are always provided during training, not the model’s own predictions.

The loss is cross-entropy over the vocabulary:

def next_token_loss(
    logits: torch.Tensor,    # [batch, seq_len, vocab_size]
    targets: torch.Tensor,   # [batch, seq_len]
) -> torch.Tensor:
    # Shift: predict token[i] from context[0..i-1]
    shift_logits = logits[:, :-1, :].contiguous()
    shift_targets = targets[:, 1:].contiguous()

    return nn.CrossEntropyLoss()(
        shift_logits.view(-1, shift_logits.size(-1)),
        shift_targets.view(-1),
    )

The model outputs logits — raw scores over the vocabulary before softmax. Cross-entropy loss maximizes the log-probability assigned to the correct next token. Minimizing this loss over a diverse corpus forces the model to learn grammar, factual associations, reasoning patterns, and anything else predictive of the next token.

Backpropagation through attention is expensive but tractable. The core cost is the QK^T matrix: with sequence length L and d_model dimension D, computing attention is O(L^2 * D) in memory and O(L^2 * D / h) per head. This quadratic scaling in sequence length is the fundamental bottleneck that alternatives like SSMs are designed to address.

Modern training uses Flash Attention, which reorders the attention computation to avoid materializing the full [L, L] score matrix in HBM (GPU high-bandwidth memory), reducing memory usage from O(L^2) to O(L) while producing identical outputs.

Architecture Tradeoffs

Architecture	Context Window	Training Parallelism	Inference Cost	Long-Range Dependencies	Sweet Spot
Transformer	Up to 1M+ tokens (with tricks)	Excellent — full parallelism	O(L) per token with KV cache	Strong through attention	General-purpose LLMs, code generation, reasoning
RNN	Effectively limited by gradient vanishing	Sequential — no parallelism	O(1) per token	Poor beyond ~100 tokens	Deprecated; historically: streaming, edge
LSTM	Better than RNN (~500 tokens practical)	Sequential	O(1) per token	Moderate	Deprecated; historically: time series, NLP pre-2017
SSM (Mamba)	Theoretically unbounded	Parallel in training mode	O(1) per token in inference	Good for long sequences	Long document processing, genomics, low-memory inference
Mixture-of-Experts	Same as base transformer	Same as base	Activates subset of weights	Same as base	Large-capacity models with controlled inference cost (GPT-4, Mixtral)

The SSM/Mamba row deserves more detail. State space models replace the attention matrix with a recurrent state that compresses sequence history. They achieve linear inference cost and handle very long sequences without the quadratic memory pressure. The tradeoff is that selective compression loses information that attention would retain — they perform worse than transformers on tasks requiring precise recall from early context.

Mixture-of-Experts (MoE) addresses a different bottleneck: parameter count versus compute. A 140B MoE model might activate only 20B parameters per token, making inference cost closer to a 20B dense model while retaining 140B capacity. The tradeoff is routing overhead, expert imbalance during training, and higher memory consumption for the full parameter set.

Production Considerations

KV cache sizing is a first-class resource. During inference, keys and values from previous tokens are cached to avoid recomputation. For a model with L layers, H heads, d_k head dimension, and S sequence length, the KV cache for one request uses 2 * L * H * d_k * S * bytes_per_element. At 32K context with a 70B model in FP16, this is several GB per request. This is why inference providers report memory, not just VRAM.

Context window length is not free. Even with Flash Attention, prefill time (processing the prompt) scales quadratically with context length. A 128K-token prompt takes roughly 256x longer to prefill than a 8K-token prompt. If you have long system prompts that are reused across requests, prompt caching (Anthropic, OpenAI both support this) is a concrete cost reduction, not an optimization — it reuses the KV cache for the shared prefix.

Temperature and sampling interact with the output distribution. The model outputs logits, which are scaled before softmax by 1/temperature. Low temperature sharpens the distribution (more deterministic, higher top-token probability). High temperature flattens it (more diverse, higher error rate). Top-p sampling (nucleus sampling) truncates the vocabulary to the smallest set of tokens whose cumulative probability exceeds p, preventing sampling from the long tail. These are not magic knobs — they are direct manipulations of the probability distribution the model has learned.

Attention pattern debugging. When output is unexpectedly poor at specific positions, attention visualization is diagnostic. Some heads learn to attend to tokens that are syntactically related; others to recent tokens; others to the beginning of the sequence. Anomalies in attention weight distribution are correlated with output failures. Tools like BertViz and transformer-specific interpretability libraries surface these patterns.

Model quantization changes the effective attention arithmetic. INT8 and INT4 quantization reduce weights but the attention computation itself is typically kept in FP16 or BF16 for numerical stability. The matrix multiply in QK^T is sensitive to precision loss — quantizing the KV cache too aggressively degrades quality non-uniformly across context lengths. Benchmark on your specific use case, not on generic benchmarks.

Positional encoding choice affects long-context reliability. Models trained with rotary positional embeddings (RoPE) can be extended beyond their training context length using RoPE scaling (YaRN, LongRoPE) without full retraining, though quality degrades in the extended range. If you are retrieving from contexts longer than the model’s native training length, test quality explicitly at those lengths — the model may not tell you it is struggling.

The Architecture Is the Product

Every LLM API call goes through this stack. The context window you pay for is bounded by quadratic attention cost. The temperature you set directly shapes the output distribution. The tokens you generate are the sequential output of a causal decoder stack passing each prediction back in as input for the next.

Knowing the transformer architecture does not make prompt engineering obsolete. But it does make the failure modes legible. When a model “forgets” something you said 50K tokens ago, you know why — the attention scores for that position are competing with everything between then and now. When latency spikes on your longest prompts, you know what to measure. When quantized inference produces degraded output on numeric content, you know where to look.

The architecture is not a black box. It is a documented, understandable system with real tradeoffs. Understanding it is table stakes for engineers who build on top of it.