Language Models are Few-Shot Learners

TL;DR

Shows that a 175-billion-parameter language model can perform many tasks from a few examples in the prompt, without any weight updates, in-context learning.

Why it matters

GPT-3 made scale and prompting central to modern AI: it demonstrated that a big enough next-token predictor generalizes to new tasks from instructions alone. This reframed how people use language models, and set off the prompting era.

Key ideas

• In-context (few-shot) learning: examples in the prompt steer behavior, no fine-tuning.
• Capabilities scale smoothly with model, data, and compute.
• A single general model competes with task-specific systems across benchmarks.
• Documents real limits, factuality, bias, and brittle prompting.

Related concepts