Loading lesson…
AI Agents / Agent Anatomy
Working memory, long-term memory, and curation.
Reviewed by Yuvaraj
An agent is only ever as aware as the text sitting in its context window at the moment it is called. The model itself is stateless: on each turn it sees exactly the tokens you assembled into the prompt and nothing else. That is fine for a single question, but an agent that plans across dozens of tool calls, or that greets a returning user tomorrow, must know things that happened long ago and far outside that window. Memory is the machinery that decides what an agent carries forward, in what form, and how it gets back into the prompt when it matters.
The cleanest split is between the memory the model reads for free every turn and the memory it has to go and fetch.
Ask about this lesson, or about anything in AI. Answers cite the lessons they draw on.
Finished this lesson?
Mark it complete to earn XP, keep your streak, and schedule a review.
The context window as working memory. Everything the model reasons over on this turn is the prompt: system instructions, the recent dialogue, and any scratchpad notes. It is quick and coherent, but strictly capped by context length.
Summarization and compaction. When the history grows past what fits, older turns are compressed into a running summary. You keep recent turns verbatim for local coherence and replace the tail with a paragraph that preserves its meaning at a fraction of the tokens.
External memory via a vector store. Past events are embedded and saved; when a new situation arises, you embed the query and retrieve the nearest past episodes. This is retrieval-augmented generation applied to the agent's own history rather than to a document corpus.
Structured state. Durable facts get their own explicit slots the agent reads and writes: variables, a task list, or scratchpad files. Because they are addressed by key rather than by similarity, they never get summarized away or lost in the noise.
For a curious beginner
Your desk only fits a few papers. You cannot keep everything in front of you, so you decide what stays on the desk, what gets filed in a drawer, and what you boil down to a sticky note. Memory design is exactly that sorting.
How it is actually used
Every token in the prompt costs latency and money and competes for a fixed budget, so you build a policy: keep recent turns verbatim, compact older ones into a summary, promote durable facts into structured slots, and fetch old episodes from a vector store only on a query match. The prompt you send each turn is an assembled view, not the whole history.
The underlying mechanism
A window admits at most tokens, but a session produces a history with . Memory is a selection-and-compression function: pick a verbatim subset and a summary such that , chosen to maximize information relevant to the next action. Once exceeds there is no lossless option; every policy trades recall against space.
A customer opens a chat about a delayed order. Over forty turns the raw transcript grows past the context window, so the agent cannot simply resend everything. Instead it curates. On turn 41, here is what the assembly step decides to put in the prompt.
The literal wording of turns 1 through 34 is gone; only their summary and the extracted facts survive, plus one relevant past episode fetched back in. The prompt that actually lands looks roughly like this:
[system] role + available tools
[state] order_id=A-4471 goal=refund (package never arrived) tier=premium
[summary] turns 1-34 compressed: troubleshooting done; carrier confirms package lost
[recent] last 6 messages, verbatim
[retrieved] past ticket T-2093 (same carrier-lost pattern) -> resolution: refund + reship
[user] "So what happens now?"
Common mistakes