Autoregressive

Autoregressive generation builds an output one piece at a time. To choose the next token, the model looks only at the tokens already present—like completing a sentence by repeatedly asking, “Given everything written so far, what comes next?”

How the prediction chain works

An autoregressive model represents the probability of a sequence as a chain of next-step predictions. For tokens x₁ through x_T, it learns:

P(x₁, ..., xₜ) = P(x₁) · P(x₂ | x₁) · ... · P(xₜ | x₁, ..., xₜ₋₁)

In a transformer, a causal attention mask enforces this rule: position 7 can attend to positions 1–6, but not position 8. This prevents the network from “seeing the answer” during training. A decoder-only transformer such as GPT uses this arrangement.

Training versus generation

Training is efficient because the model receives an entire known sequence at once and predicts every next token in parallel. The correct earlier tokens are supplied as context, a practice called teacher forcing. Generation is different: the model predicts one token, appends it to its context, then predicts the next.

Greedy decoding picks the most likely next token.
Sampling introduces controlled variation using temperature, top-k, or nucleus sampling.
KV caching stores prior attention keys and values, avoiding repeated computation over earlier tokens.

Why the design matters

The approach gives each output step rich access to its preceding context and provides a simple, stable training objective: next-token cross-entropy. Its central cost is sequential inference: later tokens cannot be generated until earlier ones exist. Errors can also compound, because during generation the model must condition on its own imperfect outputs rather than the correct training history. Excessively sharp sampling can cause repetitive text; overly random sampling can destroy coherence. Autoregression is therefore powerful not because it plans a whole output in one pass, but because it turns generation into many tightly conditioned local decisions.

Autoregressive generation models a sequence as a product of conditional probabilities, predicting each next token from the tokens generated or observed so far. During inference, the model repeatedly feeds its growing prefix back into itself to produce one token at a time. This formulation enables coherent variable-length sequence generation, but makes decoding sequential and allows early errors to influence later predictions.

Think of writing a text message one word at a time. You look at what you have already written, choose the most fitting next word, add it, then repeat. That is autoregressive generation.

An autoregressive AI creates a sequence step by step, using its earlier output as context for the next part. For example, after “Once upon a time,” it might predict “there,” then use the full phrase to decide what comes next. This is how many chatbots, text generators, and code assistants produce their answers.

It matters because language, music, and computer code all unfold in order: each new piece depends on what came before.