Attention is a soft dictionary lookup — weightj = softmaxj(β · q·kj), output = Σj weightj · valuej
Every token stores a key vector k and a value (shown as its color). Click a token to make it the query: its q is dotted with all five keys, softmax turns those scores into mixing weights, and the output blends the values.
↑ click a token — the yellow outline marks the current query
mixed value (the attention output for this query)
hard lookup — max weight > 0.95, softmax ≈ argmax≈ uniform average — β small, lookup very soft