Memory is not context

Curriculum Module 0 / Lesson 0.2. Lab shape: buffer lab. Runtime: an in-page instrument over a fixed fifty-step run — no Python, no network, no API key.

One agent. Fifty steps. Everything it ever wrote is still in the window, and by step 11 that window is three quarters tool output.

Watch it fail

10|A coding agent is given one job and runs for fifty steps [ILLUSTRATION — every number below is our fixture, and the instrument on this page recomputes all of them]. Each step appends one entry:

Step 1. The instruction: "Fix the failing test in billing/rounding.py. Never touch the migration." — 48 tokens.
Ten steps carry a user turn, 28 tokens each (steps 2, 7, 12, 17, 22, 27, 32, 37, 42, 47).
Thirty-nine steps carry a tool result — a file read, a test run, a grep — 96 tokens each.

20|In numbers: 50 steps, 4,072 tokens written once, 39 tool results holding 3,744 tokens. That is 91.9% of the tokens the run ever produced, from 78.0% of its entries. The instruction is 48 tokens — 1.2% of the run.

Here is what the window holds at the end, under three admission rules and an 800-token budget:

Rule	Entries held	Tokens held	Instruction still there?	Tool results held
No buffer at all	50	4,072	yes	all 39
FIFO, 800 budget	9	728	no — gone at step 11	the last 7
30		Priority + re-read weighting, 800 budget	15	712

Before you scroll: FIFO holds more tokens than the priority rule (728 against 712) and is still the worse buffer. What did it keep?

The picture in words

Picture a desk with a fixed tray. Every step of the run puts one more sheet of paper on the tray. When the tray overflows you have to take something off — and what you take off is the whole design.

40|FIFO takes the oldest sheet, because "oldest" is the easiest thing to compute. On a coding run the oldest sheet is the instruction. So the tidy, cheap, obvious rule quietly deletes the only sheet that says what the job is, and the agent keeps working — confidently, on the wrong task.

The alternative is a rule that ranks sheets: the instruction outranks a user turn, a user turn outranks a tool log, and among tool logs, one the run has already come back to outranks one it never opened again. That is a memory: a decision about what deserves to stay. A window has no such decision in it — it just holds everything until it can't.

The formula, derived

Two numbers decide everything here, and both are sums.

The stream's size is just addition: 50|

S \;=\; \sum_{i=1}^{50} t_i \;=\; 48 + 10 \times 28 + 39 \times 96 \;=\; 4{,}072

The buffer's size is the same sum over the entries the rule admits:

B \;=\; \sum_{i \,\in\, \text{admitted}} t_i \;\le\; 800

and the share of the run that survives into the call is $B / S = 712 / 4{,}072 = 0.1749$ , or 17.5%.

60|The interesting number is neither of those. It is what the model actually reads across the whole run — because the window is re-sent at every step:

R \;=\; \sum_{k=1}^{50} B_k

with $B_k$ the held size at step $k$ . With no buffer, $B_k$ is the cumulative stream, so

R_{\text{unbounded}} \;=\; \sum_{k=1}^{50} \sum_{i \le k} t_i \;=\; 101{,}980

and with the 800-token buffer $R_{\text{bounded}} = 33{,}820$ . The ratio is 3.02× — the same work, a third of the traffic. Nothing about the model changed.

Worked example with the home numbers

70|Walk the first eleven steps by hand. The pool grows by 96 for each tool result and 28 for each user turn:

Step	Entry	Pool	FIFO holds	Priority holds
1	instruction, 48	48	48	48
2	user, 28	76	76	76
7	user, 28	488	488	488
80		10	tool result, 96	776
11	tool result, 96	872	796 — instruction evicted	776 — a log is evicted instead

At step 10 the pool is 776 tokens, under the 800 budget, so nothing has been dropped. At step 11 it is 872. Something must go. FIFO drops everything written earliest, in order, until it fits: the instruction (48) and the step-2 user turn (28) both leave, and the pool settles at 796. The priority rule drops the least important entry instead — the oldest tool log — and keeps the instruction.

By step 50 the difference is total. FIFO holds 9 entries and 728 tokens: seven tool results from steps 43–50 and two user turns from steps 42 and 47. It holds more tokens than the priority window and none of the run's rules. This is the failure the lesson is about, and it is not a size failure.

90|Which four results does the priority rule keep? Not the newest four. The run re-opened four results later — step 9's result at steps 20, 26 and 33; step 41's at 45, 49 and 50; step 28's at 34 and 38; step 16's at 23 and 44. Ten re-reads over four entries. Weighting ties by re-read count keeps exactly those four: 9, 16, 28, 41. Break the ties by recency instead and you keep 46, 48, 49, 50 — the same 15 entries, the same 712 tokens, four completely different results. The tie-break is the memory.

Try it first. A different run, on numbers that do not appear in the lesson: 20 steps, one instruction of 60 tokens at step 1, four user turns of 25 tokens at steps 4, 9, 14 and 19, and fifteen tool results of 80 tokens. The budget is 400 and the rule is FIFO. What has the pool written by step 6, and what does FIFO hold after step 6? (Answer: by step 6 the pool has $60 + 25 + 4 \times 80 = 405$ — the instruction, the step-4 user turn and results at steps 2, 3, 5 and 6. That is five tokens over budget, so FIFO drops the 60-token instruction first and holds $25 + 4 \times 80 = 345$ . The rule that is cheapest to compute is the one that loses the job. For contrast, by step 12 the pool has $60 + 2 \times 25 + 9 \times 80 = 830$ , more than twice the budget, and the run has been running without its instruction since step 6.)

Code that prints the same numbers

The arithmetic is a loop and two sums. The page runs the TypeScript version; this is the same thing in Python so you can check it on your own machine: 100|

INSTRUCTION, USER, LOG = 48, 28, 96
user_steps = {2, 7, 12, 17, 22, 27, 32, 37, 42, 47}
stream = [INSTRUCTION if s == 1 else USER if s in user_steps else LOG
          for s in range(1, 51)]

print(sum(stream))                       # 4072
print(39 * LOG / sum(stream))            # 0.9194499017681729
print(39 / 50)                           # 0.78

def fifo(budget):
    queue, used = [], 0
    for step, size in enumerate(stream, start=1):
        queue.append((step, size)); used += size
        while used > budget:
            used -= queue.pop(0)[1]
        yield used

held = list(fifo(800))
print(held[9], held[10])                 # 776 796
print(len(held), sum(held))              # 50 101980
print(sum(held[:50]), max(held))         # 33820 792

4072, 0.9194499017681729, 0.78, 776 796, 101980, 33820 792 — the same numbers the tables and the instrument show. python3 tools/me-l2-prove-home-numbers/prove-home-numbers.py prints the full set and asserts every one of them.

The lab

110|The instrument below is the whole argument, made draggable. Three controls:

Admission rule — priority, fifo, or no eviction. fifo cuts the window to 9 entries and the instruction disappears; no eviction draws a curve that is the transcript itself, which is what "we have a big window" actually buys.
Token budget — drag it from 256 to 4,096. At the top of the range nothing is evicted and the buffer holds all 50 entries: at that size the buffer is the context, and this lesson's whole distinction evaporates.
Ties: most-re-read first / most-recent first — flips which four tool results survive at identical size. The window does not change shape; the memory does.

Then fill the four blanks (tool-result share, entries held, the step FIFO loses the instruction, the traffic ratio). Every wrong answer comes back with the reason it is wrong, not a red cross. 120|

Interactive lab loads with the lesson app.

The failure mode

Four things this lesson is often mistaken for:

"A bigger window removes the need for memory." Drag the budget to 4,096 and the window holds all 50 entries — for 101,980 re-sent tokens instead of 33,820. The buffer did not become unnecessary; it became the transcript.
"FIFO is a fair approximation." FIFO ends the run holding 728 tokens against the priority rule's 712, and re-sends 33,580 against 33,820 — marginally less traffic, and it lost the instruction at step 11. Being a little cheaper is not the same as being a memory. Fairness to old entries is what deletes the job description.
"Tool output is the detail; the important stuff is small." The important stuff is small: the instruction is 48 tokens, 1.2% of the run. That is exactly why an unweighted rule throws it away first.
"Eviction is a token-budget problem." Every discarded token in the home fixture is a tool result — 3,360 of them. The budget was never the constraint; the ranking was.

One more thing this lab deliberately does not do: it never reads the evicted entries back. A buffer that discards is not a store that persists. Writing evicted entries to somewhere the agent can search again is the difference between a window and a memory, and that is the subject of the next module.

What the published numbers do and do not say

Two published claims carry this lesson, and each is narrow:

With causal masking, every token position attends to itself and every earlier position, so a sequence of $n$ tokens needs $n(n+1)/2$ attention pairs. For the home window that is $712 \times 713 / 2 = 253{,}828$ pairs; for the whole stream it is $4{,}072 \times 4{,}073 / 2 = 8{,}292{,}628$ — 32.7× more, from the same fifty steps. — Arithmetic, not measurement. [REAL: computed by the instrument and the prove script] 140| "Performance is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts." — Liu et al., Lost in the Middle: How Language Models Use Long Contexts, TACL 2024, arXiv:2307.03172. [PUBLISHED]

What the second one does not say: it is a study of retrieval accuracy inside a long input, not a measurement of what a memory system saves, and it says nothing about eviction policies. It is cited here for one narrow reason — the position of a fact inside a long window changes whether it is used, so "it is still in the window" is not the same claim as "the model will act on it". Do not read a cost or a saving into it.

The first is not a benchmark at all. It is the shape of the attention computation, and it is printed because it explains why re-sending a longer window is more expensive than the token count alone suggests. It is arithmetic on our own fixture, labelled as such.

One-line close

Context is what is in the call. Memory is the decision about what gets to be in the call. A window with no ranking rule is not a memory — it is a queue that happens to end at the same place.

What breaks next: you now have a buffer that ranks entries and re-sends less. But why the ranking works — which kinds of memory exist, how they are typed and namespaced, and where each one is supposed to live — is a taxonomy, not a heuristic. That is me-l4, Tiers of memory: the OS metaphor — the first lesson of Module 1, where the ranking rule stops being a guess.

Home numbers / honesty table

160|

Item	Value	Label
Steps in the run	`50`	ILLUSTRATION fixture
Tokens written once	`4,072` (48 + 280 + 3,744)	ILLUSTRATION fixture
Tool results	`39` entries, `3,744` tokens	ILLUSTRATION fixture
Tool-result share of tokens	`91.9%`	ILLUSTRATION fixture, computed by the instrument
Tool-result share of entries	`78.0%`	ILLUSTRATION fixture, computed by the instrument
Buffer budget	`800` tokens	ILLUSTRATION fixture
Home window (priority + re-reads)	`15` entries / `712` tokens / `17.5%`	REAL: computed by the instrument and the prove script
Re-sent tokens, bounded	`33,820`	REAL: computed by the instrument and the prove script
Re-sent tokens, no buffer	`101,980`	REAL: computed by the instrument and the prove script
170		Traffic ratio
FIFO window at 800	`9` entries / `728` tokens	REAL: computed by the instrument and the prove script
FIFO loses the instruction	step `11`	REAL: computed by the instrument and the prove script
Attention pairs, window vs stream	`253,828` vs `8,292,628` (`32.7×`)	REAL: arithmetic on the fixture
Positional degradation inside a long context	qualitative only, no number printed	PUBLISHED: Liu et al., TACL 2024, arXiv:2307.03172
Lesson video	not rendered	No clip is shipped with this lesson

Video

No rendered lesson video is shipped with this lesson. The curriculum gives Module 0 one concept video ("The agent that forgot"), and it belongs to Lesson 1; this lesson is text plus a diagram, and the diagram is the still above. This page does not link a file that does not exist.