All termsTry it in the Attention Visualizer labExplore in the AI Universe
Glossary · Term
Attention
L3 · AdvancedFoundational
In plain terms
A weighting of how much each token attends to others.
Technical definition
A mechanism computing weighted sums of value vectors using query-key similarity scores.