Course
Transformers
Attention, tokens, and the architecture behind every modern language model.
L3 · AdvancedEvolvingKnown~3 h
What you’ll learn
- Explain how text becomes tokens and embeddings and why position must be encoded
- Derive scaled dot-product attention and multi-head attention from queries, keys, and values
- Assemble a transformer block from attention, feed-forward layers, residuals, and normalization
Prerequisites
llm-foundations
Module 1. Inputs
Tokens, embeddings, and positional encoding.
Module 2. The Transformer Block
Attention and the full transformer block.