An LLM has no memory, the chatbot does

In the prompting course I run for non-specialists, one fact comes before any technique: the model behind a chatbot remembers nothing.

A large language model is stateless. Each request stands alone: text goes in, text comes out, and nothing stays in the model. Providers say so in their own documentation: their APIs treat each generation request as independent and stateless, and a multi-turn conversation as a series of requests in which the caller resends the earlier turns.

The model does not remember the conversation. It rereads it, every time

The memory users experience therefore belongs to the chat application. When you send your fifth message, the application sends the model the four earlier messages, the model’s replies, then the new one. The model reads the whole conversation as if for the first time, and continues it.

Three consequences follow, and they explain most surprises.

A long conversation has a limit. A request holds a limited number of tokens, for both what goes in and what comes out. When the conversation outgrows it, the application truncates or summarises it, and whatever falls outside the limit no longer exists for the model, even if it is still on your screen.

A new conversation starts from scratch. The model learned nothing from yesterday’s chat. Features presented as memory are notes the application keeps about you and adds to the prompt: useful, but stored by the application, not learned by the model.

Every word is sent again. A detail shared early in the conversation goes with every request that follows. Before pasting something sensitive, the right question is not whether the AI will remember it, but where the application stores it and how often it sends it.

What the AI remembers is what the application chose to send back