Research

Research Library

Landmark papers that shaped modern AI, each explained in original plain language with the key ideas distilled. Selected papers include a from-paper-to-code walkthrough. Every entry links to the authoritative source.

Transformers

Attention Is All You Need

Walkthrough

Introduces the Transformer, a sequence model built entirely on attention with no recurrence or convolution, the architecture nearly every modern large language model descends from.

ExpertVaswani et al. · 2017 · NeurIPS 2017

Optimization

Adam: A Method for Stochastic Optimization

Walkthrough

Defines Adam, an optimizer that adapts each parameter's step size from running estimates of the gradient's mean and variance, the default choice for training deep networks.

AdvancedKingma et al. · 2015 · ICLR 2015

Computer Vision

Deep Residual Learning for Image Recognition

Introduces residual networks (ResNets), where layers learn a correction to their input via skip connections, making it practical to train networks hundreds of layers deep.

AdvancedHe et al. · 2015 · CVPR 2016

Natural Language Processing

BERT: Pre-training of Deep Bidirectional Transformers

Pre-trains a Transformer encoder to read text in both directions at once by predicting masked-out words, then fine-tunes it for downstream tasks.

ExpertDevlin et al. · 2018 · NAACL 2019

Large Language Models

Language Models are Few-Shot Learners

Shows that a 175-billion-parameter language model can perform many tasks from a few examples in the prompt, without any weight updates, in-context learning.

ExpertBrown et al. · 2020 · NeurIPS 2020

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

Shows that prompting a large model to write out intermediate reasoning steps markedly improves its accuracy on multi-step problems.

ExpertWei et al. · 2022 · NeurIPS 2022

Embeddings

Efficient Estimation of Word Representations in Vector Space

Introduces word2vec, which learns dense word vectors from context so that geometric relationships in the vector space capture meaning and analogy.

AdvancedMikolov et al. · 2013 · ICLR Workshop 2013

Retrieval-Augmented Generation

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Combines a neural retriever with a generator so the model conditions its output on documents fetched from an external corpus rather than on parameters alone.

ExpertLewis et al. · 2020 · NeurIPS 2020

Generative Models

Denoising Diffusion Probabilistic Models

Frames image generation as learning to reverse a gradual noising process: start from pure noise and denoise step by step into a sample.

ResearchHo et al. · 2020 · NeurIPS 2020

Generative Adversarial Nets

Trains two networks against each other, a generator that fabricates samples and a discriminator that tries to spot fakes, until the fakes are convincing.

ExpertGoodfellow et al. · 2014 · NeurIPS 2014

Reinforcement Learning

Playing Atari with Deep Reinforcement Learning

Learns to play Atari games straight from raw pixels by combining Q-learning with a deep convolutional network, the Deep Q-Network (DQN).

ExpertMnih et al. · 2013 · NeurIPS Deep Learning Workshop 2013

Fine-Tuning

LoRA: Low-Rank Adaptation of Large Language Models

Fine-tunes a large model by freezing its weights and training tiny low-rank update matrices instead, cutting the tunable parameters by orders of magnitude.

ExpertHu et al. · 2021 · ICLR 2022

Multimodal AI

Learning Transferable Visual Models From Natural Language Supervision

Trains image and text encoders together on hundreds of millions of image-caption pairs so that matching pictures and descriptions land near each other in one shared space.

ExpertRadford et al. · 2021 · ICML 2021