Claude Opus 5.5: 40% cheaper, frontier-grade performance

Data Point

MERIT benchmark measures memory utility for tool agents

Embedding retrieval for agent memory collapses unpredictably (0.30–0.95 across models); structured fact stores and LLM summarization hold at 0.70–1.00, making implementation choice worth 60 points of task success.

Memory backend selection directly impacts agent reliability in multi-turn tasks. Testing with realistic cost accounting ($42.57 across 23,440 episodes) shows full replay is never economical—choosing the right store architecture delivers 2.7–3.9x better utility per dollar than naive approaches.

Replaces vague "long-context" memory assumptions with measurable task-based benchmarking. Requires running MERIT tasks in your domain to validate memory backend choice; structured fact stores outperform embedding retrieval for updated-fact recall. Actionable now: use this to justify switching from vector-only memory to hybrid or write-through stores in production agents.

• “memory lifts dependent-task success from a leak-verified floor of 0.00 to 0.55-1.00”
• “embedding retrieval collapses unpredictably (0.30-0.95 across models; max seed gap 0.45)”
• “agents act on a correctly retrieved value only 55% of the time”
• “update-on-write stores (a structured fact store and, notably, LLM summarization) remain at 0.70-1.00”
• “Swapping a memory's implementation moves task success by up to 60 points”
• “the best condition per domain delivers 2.7-3.9x its marginal utility per dollar”