Key Vector

In attention, a token needs a way to ask, “Which other pieces of information are relevant to me?” A key vector is the searchable label each token presents for that purpose: it describes what kind of information the token offers.

Each input representation is transformed into three learned vectors: a query, a key, and a value. The current token’s query is compared with every token’s key using a dot product. A large score means that the query and key align well, so that token deserves more attention. After scaling and a softmax, these scores become weights used to blend the value vectors.

attention(Q, K, V) = softmax((QKᵀ) / √dₖ) V

Here, K is the matrix containing all key vectors, and dₖ is the key dimension. Dividing by √dₖ prevents dot products from becoming excessively large as the vectors get wider; without it, softmax can become too sharp and gradients can fade.

A key answers “How relevant am I?” while a value answers “What information should I provide?” Keeping them separate lets the network learn different representations for matching and for content transfer. For example, one attention head can learn keys that identify syntactic roles, while its values carry the features needed by later layers.

• Queries express what a position is looking for.
• Keys make positions comparable and discoverable.
• Values supply the information selected by attention.

Key vectors are learned through backpropagation along with the projection matrices that create them, such as PyTorch’s nn.MultiheadAttention projections. Poorly scaled key-query scores can make attention nearly uniform, so the model cannot select useful sources, or nearly one-hot, so learning becomes brittle. Multi-head attention helps by giving several independent sets of keys different matching spaces. In a Transformer, masks are applied to the key scores before softmax: a causal mask prevents a token from attending to future keys, while a padding mask prevents attention from being wasted on empty positions.