How Much Can a Language Model Actually Memorize?

Chat with a large model long enough and you hit a contradiction. Sometimes it quotes a novel almost word for word — and you think it stored the whole training set. Other times you ask for one precise number from a paper and it invents a plausible fake — and you think it remembers nothing. So which is it?

Stop asking only whether it can recite

Older tests often watch the model’s outputs: can it regenerate training text, or can membership inference tell that a snippet was in the set? Both blur two different things. Output is always a mix of rote detail and general skill. A model that answers 2+2 may have learned addition; a model that continues a famous poem may have learned poetic habits — or truly memorized that poem. Failure to recite also does not prove zero leftover information.

The paper’s shift is simple: treat memory as compression. If a model truly kept sample-specific information, using that model as a reference should shorten how you encode the sample. How much shorter is a measure of how much exclusive information about that sample sits inside the weights.

Remove generalization, then measure capacity

To isolate pure memorization, the authors train on uniform random bit strings — data with no grammar, no world knowledge, nothing to generalize. Any improvement can only come from storing the samples themselves. They train GPT-style Transformers from about 500K to 1.5B parameters on random sets of many sizes.

A clean pattern appears. While the dataset is small, total memorized information grows nearly linearly with more data. Past a point, it plateaus. That plateau is the model’s total memory capacity. Across sizes, capacity scales almost linearly with parameter count.

About 3.6 bits per parameter — with a warning label

Under BF16 — the half-precision format common in large-model training — they estimate roughly 3.64 bits of information per parameter. In FP32 the figure rises only to about 3.83 bits. A float32 number has 32 bits on paper; using under four of them for stored information is much less “efficient” than a hard-drive metaphor suggests.

• The number is empirical for GPT-style architecture, training setup, and their compression estimate — not a universal physical constant.
• Gradient descent may miss the global optimum, so the estimate is closer to a lower bound: true capacity may be a bit higher, not wildly higher.
• Do not multiply 7B × 3.6 bits and treat the product as “how many gigabytes of files the model holds.” Parameters are not a disk full of documents.

The striking figure is useful because it turns a fuzzy debate into a scale you can reason about. The deeper point is the method: once you can separate sample-specific memory from shared rules, you can ask better questions about safety, data scale, and what “knowing” even means inside a model.