When AI still believes what is no longer true
Stéphane Commenge, with Claude ·
The Fact
A University of California, Berkeley study shows that, on logs of 2,048 operations, two of the most advanced models get 9 and 18 of 40 current-state questions right. With a snapshot of the current state in front of them, they answer all 40 correctly.
The Signal
The model doesn't lack memory, it lacks a calendar. When two sentences contradict each other, it doesn't know, on its own, that the second cancels the first.
A useful memory doesn't just keep what it's told. It knows what has been replaced.
The Action
When you change your mind in a conversation with your AI, say so plainly: 'this replaces what I told you on the 12th'.
For a project that lasts, keep a short 'current state' sheet and paste it back in at the start of the conversation.
Series "AI memory", episode 4. Previous episode: having everything in front of you is not knowing what matters.
On October 12, I decided to publish one blog article a month. On October 29, I changed my mind.
My notebook doesn't mind: the first line is crossed out. In an AI's memory, the two sentences sleep side by side, and say opposite things. Which one will it believe?
Three research teams asked that question this year. Their results point the same way.
Berkeley: the old value wins
At the end of September, a team from the University of California, Berkeley published "When Context Changes: Understanding Update Failures in LLMs". The principle: over a long conversation, pieces of information change. Then you ask the model where things stand.
The telling numbers are not in the abstract. They're in section 2.
One error, one cause. On Qwen2.5-7B, 331 errors out of 333 return the old value. The model doesn't ramble. It answers with what was true before. Yet the new value was in the conversation.
The best ones too. On logs of 2,048 operations, two of the most advanced models of the moment get 9 and 18 of 40 current-state questions right. In both cases, more than half the answers are wrong.
The workaround. With a snapshot of the current state in front of them, the same two models answer all 40 questions correctly. Forty out of forty.
The model doesn't lack memory. It lacks a calendar.
UIUC: state drift
In August, a team from the University of Illinois Urbana-Champaign built StateMemBench, a synthetic benchmark for agent memories: 234 scenarios, 322 questions, a situation that evolves over time.
The best conventional memory system gives the up-to-date answer in 20.5% of cases. Feeding the whole history into a long context window, it drops to 14.9%. The authors call the flaw "state drift". The memory keeps the facts. It loses track of what holds today.
STALE: knowing that a memory has expired
In May, the STALE benchmark asked the question more directly: does an agent know when its memories are no longer valid? 400 conflict scenarios, 1,200 questions, contexts of up to 150,000 tokens (the word fragments the model reads). The best model tested reaches 55.2% overall accuracy. A little over half the time.
Why the snapshot works
We do the same, to be fair. Who hasn't called a colleague at their old desk? The difference: we know that one decision can replace another. An AI reads two sentences. It doesn't know, on its own, that the second cancels the first.
The snapshot sorts things out before the question. Instead of the history ("on the 12th, we decide A; on the 29th, we decide B"), the model receives the state: "today, B". It has nothing left to reconstruct.
Reading every amendment to a contract, or reading the consolidated contract. A useful memory doesn't just keep what it's told. It knows what has been replaced.
A tip
When you change your mind in a conversation with your AI, say so plainly: "this replaces what I told you on the 12th". For a project that lasts, keep a short "current state" sheet that you paste back in at the start of the conversation. Berkeley's snapshot, done by hand.
Let's turn the question around
How many of your decisions from three months ago still hold? The vendor chosen in the spring, the price set before summer, the instruction given to the team in July. If your AI still reads them as in force, it's advising you with yesterday's file.
What I'm building with Mycelora
Each memory carries a type, a date and, if needed, its expiry condition.
A replaced or refuted memory drops out of recall. The history stays available.
Since October 10, 2026, with each new memory, a model searches the same project for the one it makes false and marks it as replaced. It's reversible.
If the AI says the opposite of a decision in force, an alert arrives in the conversation.
This sorting is done by a model, project by project. It can miss a case. Hence the available history and the reversible replacement.
The beta is open: mycelora.ai
Next episode
Two AIs, two memories. The next episode is about one memory for all your AIs.
Sources
- Guo, Fang, Gu, Spanos, Demmel, Lavaei (UC Berkeley), "When Context Changes: Understanding Update Failures in LLMs", arXiv, September 30, 2026: arxiv.org/abs/2609.38866
- Fan, Liu, Yang, Ouyang, Han (UIUC), "Can Agent Memory Systems Track Evolving State?", arXiv, August 20, 2026: arxiv.org/abs/2608.19652
- Chao, Bai, Sheng, Li, Sun, "STALE: Can LLM Agents Know When Their Memories Are No Longer Valid?", arXiv, May 7, 2026: arxiv.org/abs/2605.06527