An assistant that followed your rules perfectly for the first twenty messages starts ignoring them. "Reply in Hindi", "always show sources", "answer as JSON" — the rule from message one silently stops applying. No error, no warning; the replies are fluent and confident, only the rule is gone. Restating the rule fixes it for a while, then the drift returns.
Long chat sessions and document-heavy prompts: a support-bot transcript that grew all afternoon, a debugging session with many pasted stack traces, or a prompt that stuffs a whole PDF ahead of a short question. Most common when the team uses a small local model (smaller context window) or leaves an app's default trimming behaviour on without knowing it exists. Linked module: 3.1 (How an LLM works at inference).
The model has no memory between calls; the app re-sends the visible transcript every
turn, and the model can only consider tokens that fit its context window. When the
transcript outgrows the window, something must go, and most apps trim or summarise the
OLDEST content first — which is exactly where your rule lives. So the model is not
disobeying; the instruction no longer arrives. Confirm it: token counts are the
evidence. On a local Ollama endpoint compare prompt_eval_count against the model's
context size (ollama show <model> prints it); in a hosted API compare the request's
prompt tokens against the model's documented window. A second clue: the model still
follows rules given RECENTLY while losing early ones — trimming eats from the top, not
the bottom. Even inside the window, attention weakens for material buried in the middle
of a huge prompt, so partial rule-drift can start before hard overflow.
Restate the rule in the newest message — the end of the prompt is the last place to be trimmed and the best-attended position. For a rule that must always hold, move it into the system prompt or into every request your code builds, instead of trusting one old chat message. If the transcript has grown huge, start a fresh chat and paste a short briefing paragraph: goal, constraints, decisions so far. In code, measure prompt tokens before sending and trim the middle content yourself — keep the instructions and the question, summarise or drop the bulk — rather than letting the app decide blindly.
Front-load critical instructions into the system prompt, not the first user message. Restate hard constraints every few turns in long sessions, and start a new chat when the topic changes. Log prompt token counts in any pipeline you build, with an alert at around 80% of the window. When someone reports "the bot got dumber over time", ask "how long is that chat?" first — it resolves most reports on its own.
Q: "Your chatbot follows the format rule for the first hour, then stops. Users say the model degraded. Walk me through your diagnosis." Strong answer: state that the model is stateless and the app re-sends the transcript, name the context window and oldest-first trimming as the mechanism, point to prompt-token counts versus window size as the confirming evidence, and give the fix ladder — system prompt for standing rules, restatement in long chats, fresh session with a briefing paragraph. Bonus: mention that middle-of-prompt content is attended weakly even before overflow.