The AI Didn't Forget.
AI doesn’t always forget instructions. Sometimes the context window gets overloaded. Two very different AI workflows reveal the same problem - and why managing context matters.
I Lost the Wrench.
by Jana Diamond, PMP, and Prateek Sharma, MCA
Prateek’s Story:
For months, I thought the AI was forgetting things.
Not little things, either.
I was using a Claude coding assistant every day on production code. During one Android Room migration, I was working through a schema change, a mislabeled migration, and an annotation processor hanging onto a stale cache.
Three hours into the debugging session, we'd accumulated logs, DAO files, entity classes, migration details, and a whole lot of my own thinking out loud.
Then things started getting weird.
The AI forgot the migration number we'd already settled on.
Then it forgot the column we'd renamed.
Then it suggested something we'd already ruled out.
Jana’s Story:
For months, I thought the AI was forgetting things.
Not little things, either.
I was using a ChatGPT AI assistant for a multitude of different tasks. Writing proposals and blogs, even writing a book, working on images for things I’m trying to sell on Zazzle.
We’d start off great. With the prose, it might be two or three hours, then Chat was wandering off-base. Forgetting what I told it to do.
Replying based on earlier context instead of the document I'd just given it two minutes previously, so the cross-references it created were . . . pure fiction.
I even had the same line – Refer to the document – in my instruction set no less than three times.
I was treating Chat like a two-year old, and prefacing every prompt with the same four lines:
Open the document
Read the document
Tell me you read the document
If you don’t still have the document, tell me to re-load it again, do not invent answers.
Images were even worse. My cartoon people kept having human heads and faces on them. Talk about scary-gross!
The solution we each came up with separately?
Start a new session and explain the whole damn thing again.
And again.
We both did this for months.
We both assumed that was simply the price of working with AI.
It wasn't.
And, of course, we didn’t talk to each other about this. But we work together daily.
The AI Hadn't Changed. The Context Had.
An AI conversation doesn't have infinite working space.
Every instruction, file, log, question, response, tool definition, and piece of project information takes up some of that space. As the conversation grows, eventually something has to give.
Some systems summarize older parts of the conversation to make room for new information. Some provide ways to compact or clear the working context. The exact mechanics vary by tool.
But the practical problem is the same:
The conversation you started with is not necessarily the conversation the AI is effectively working from three hours later.
We had been treating the context window like an unlimited notebook.
It isn't.
It's a workbench.
And we had covered every square inch of ours with tools, parts, old coffee cups, three versions of the same schematic, and whatever the heck else we’d thrown on there during the day.
Then we wondered why we couldn't find the wrench.
Prateek Finally Looked at the Workbench
The coding tool Prateek was using had commands for managing all of this.
/context showed what was consuming the available context.
/compact summarized the existing conversation and freed space to keep working.
/clear removed the conversation when it was time to move to unrelated work without throwing away the underlying project setup.
Those commands are specific to the tool Prateek was using. The larger lesson isn't.
Context has to be managed.
That changed how he interpreted the behavior we'd both been seeing.
When an AI appeared to "forget" something we'd established earlier, we had been treating that as a model problem.
Sometimes, what looked like forgetting was really a context problem.
And those require very different fixes.
Starting over worked, technically. But it was the digital equivalent of cleaning my desk by throwing everything into the yard.
Effective?
Sure.
Efficient?
I don't think so, George.
What Actually Belongs in the Conversation?
Once Prateek understood what was happening, another distinction became much more important.
Some information is temporary.
A stack trace. A test result. A question you need answered right now.
Some information needs to survive the entire job.
Project conventions. Constraints. Architecture decisions. Instructions you don't want the AI casually wandering away from halfway through the afternoon.
Those shouldn't necessarily live only in a long conversation.
If something matters throughout the work, it needs to live somewhere the system can reliably retrieve or reload it. Depending on the AI tool, that might be project instructions, memory, configuration, attached reference material, or another persistent source.
We'd been using conversation for both.
That's where we got bit.
Context Management Is Part of the Job
We spend a lot of time talking about prompts when we talk about working effectively with AI.
Prompts matter.
But so does deciding what the AI should be carrying forward, what can be summarized, what should be discarded, and what needs to live somewhere more durable than the conversation.
Once a session becomes long enough, those decisions affect the quality of the work.
So now, when an AI starts contradicting something we settled an hour ago, don't immediately assume the model has suddenly gotten stupid.
Look at the workbench.
Because sometimes the AI didn't forget.
You just buried the wrench.
Originally published on Protovate.AI
Protovate builds practical AI-powered software for complex, real-world environments. Led by Brian Pollack and a global team with more than 30 years of experience, Protovate helps organizations innovate responsibly, improve efficiency, and turn emerging technology into solutions that deliver measurable impact.
Over the decades, the Protovate team has worked with organizations including NASA, Johnson & Johnson, Microsoft, Walmart, Covidien, Singtel, LG, Yahoo, and Lowe’s.
About the Authors
Share article
