Interestingly, the most common error in agentic systems is the OOM error :)
When you build an agent, almost everything you query and everything it needs is held in memory. Things usually work fine locally, but when you put it into production, or just add a little more load - longer conversations, more tool results, bigger documents - the process dies from running out of memory.
To be honest, these systems work perfectly during development, only to fail once conversations get longer or traffic starts increasing. Of course, the issue is not the model here, but the amount of context and state we keep around it. Here are some common culprits:
- tool results held in context
- search results, API responses, and outputs pile up
- no context truncation in place
- external call results accumulate in memory
- entire doc loaded when only a small portion is needed
Also, the fixes are not exotic in any way: truncate aggressively, summarize, intermediate steps, remove what’s not needed from context and mem, retrieve only what is needed, stream instead of buffering, and persist state externally instead of keeping everything in memory.
The key is identifying what truly needs to stay in memory and what can be moved elsewhere. Also, it is equally important to understand how memory usage grows as conversations get longer or agent loops run far beyond what was originally expected.
Given how long-running AI systems are, it is important to design them to handle errors, rate limits, and failure scenarios gracefully, with a well-defined degradation policy.
We have all solved these kinds of problems before. Agentic systems are no different - the constraints are familiar, but the surface area is new.