Query Vector

A query vector is a learned representation of what one position in a sequence is looking for from other positions. Rather than processing tokens strictly one after another, attention lets each position ask a targeted question: “Which pieces of available information matter to me right now?”

How the query participates in attention
For each input representation x, a transformer applies learned linear projections to create a query (Q), key (K), and value (V). A query is compared with every allowed key using a dot product. Larger scores mean the query and key are more compatible. After scaling and softmax, these scores become attention weights used to blend the value vectors:

Attention(Q, K, V) = softmax(QKᵀ / √dₖ) V

The query supplies the “search request”; keys act like labels that can match it; values contain the information retrieved. The division by √d_k keeps dot-product scores from becoming so large that softmax produces overly sharp, hard-to-train choices.

Self-attention and cross-attention
In self-attention, queries, keys, and values all come from the same sequence, but use different learned projections. A token’s query can therefore seek a distant dependency without losing information through many recurrent steps. In cross-attention, common in encoder–decoder models, queries come from one stream while keys and values come from another. A decoder position, for example, queries the encoder’s representations for source information relevant to its current prediction.

Why query vectors matter in training

• Multi-head attention creates several separate query projections, allowing different heads to search for different relationships in parallel.
• Causal masks restrict which keys a query may inspect, preventing a position from accessing future positions during autoregressive training.
• Poorly scaled queries or keys can make attention nearly uniform (nothing stands out) or nearly one-hot (gradients become brittle). The scaling factor, LayerNorm, residual connections, and careful initialization keep these interactions trainable.
• Computing every query–key comparison costs quadratic memory and compute in sequence length, which is a central limitation of standard attention.