How Much Memory Does Your Agent Actually Need?

Calibrating Agent Memory

Researchers from IBM Research have published findings that challenge the conventional wisdom on AI agent memory, demonstrating that performance gains depend not on the quantity of experience provided, but on calibrating the 'dose' to the underlying model's capability. The study, which used the ALTK-Evolve framework to distill an agent's past successes and failures into reusable guidelines, found that more memory does not universally lead to better results. This work suggests that optimizing agentic systems is a nuanced task of matching the complexity of guidance to the model's capacity to use it effectively.

Dosage Depends on Capability

The analysis across eight different models, from 30B parameter systems to frontier models, revealed three distinct patterns of behavior when equipped with agentic memory on the AppWorld benchmark. The team tested configurations ranging from no memory to injecting a full set of distilled guidelines or a smaller, curated selection for each task. The results showed a clear correlation between model strength and the optimal memory strategy.

• Strong models with headroom: Models like DeepSeek-V3.2 (671B MoE) showed the most significant improvement (+9.5pp in task completion) when given the entire set of guidelines.
• Weaker models: Systems like gpt-oss-120b (117B MoE) performed best with a curated retrieval approach, gaining +16.1pp in task completion. Providing the full guideline set to these models drowned them in context and yielded lower accuracy.
• Saturated models: Top-performing models that were already near their ceiling on the tasks, such as GLM-5 (745B MoE), showed no measurable gain from the added memory, indicating the guidelines did not address their remaining failure modes.

Cost and Performance Implications

The study carries significant implications for the cost-effective deployment of AI agents in enterprise settings. For weaker or more specialized models, the most accurate memory strategy—curated retrieval—is also the most economical. For instance, gpt-oss-120b achieved its peak performance with only a 5% increase in token overhead per task. This finding decouples performance improvement from high inference costs. For stronger models that benefit from the full guideline set, the researchers note that techniques like prompt caching can make the approach affordable in production by caching the static, shared portions of the context, thereby mitigating the cost of larger inputs.

The research shifts the enterprise focus from merely accumulating agent experience to precisely calibrating the delivery of that experience. This demonstrates that significant performance and cost-efficiency gains can be achieved through sophisticated in-context learning strategies, rather than relying solely on more expensive model updates.