MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks

In realistic settings, memorization and action are tightly coupled: agents acquire memory while interacting with the environment, and subsequently rely on that memory to solve future tasks.

We argue that agent memory should be evaluated by treating memorization and action as inseparable components of agentic behavior.

Despite their strong performance on existing memory benchmarks, these agents exhibit low task completion rates in MEMORYARENA, revealing persistent difficulties in maintaining and exploiting latent task state across sessions.

MEMORYARENA is the first one designed to assess agent memory using sequential subtasks with causal dependencies across sessions.

Later purchases depend on recalling attributes of earlier items to ensure compatibility and preference consistency.

Solving each session requires the agent to recall prior purchases, identify compatibility constraints, discard negative options, and select a valid product.

I want to stay at a hotel with at least a two-level higher rating than Rebecca’s.

Verifying a single claim often requires pages of derivations and careful reuse of previously established conclusions, making this setting a natural testbed for evaluating long-term memory and multi-step formal reasoning.

Task-relevant information must be selectively stored and retrieved through a persistent memory system in order to support decision-making in later subtasks.

The decay trend indicates agents cannot sustain execution as dependencies span more sessions.

This pattern suggests that while agents can make some progress on individual subtasks, they fail to integrate these partial successes into globally consistent solutions dramatically.

Consequently, pairing strong long-context agents with external memory does not reliably produce a “1 + 1>2” effect.

Across sessions, the agent never directly observes the full underlying task state (e.g., the latent bundle specification, the evolving set of group constraints, or the intermediate dependencies required by later subtasks).

small errors in the agent’s implicit state estimate accumulate across sessions and eventually dominate downstream decisions

Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks

Memgpt: Towards llms as operating systems.

Compatibility notes: Gel pairs well with Astringent. Foam pairs well with Pore. Salicylic pairs well with Exfoliating.

A person who received their B.A. in a university different from where they received their postgraduate degrees got to name a location and was set to participate, at least up to July 2021, at a convention held by a society founded, up to April 2021, more than one decade ago but less than four decades ago.

The candidate matches educational disparity, location naming, and multi-author paper participation within the Brunswich/Geobiology context.

We selected 2 items that are logically mutually exclusive (satisfying the reject map) to serve as “hard negative” samples, thereby testing the model’s understanding of constraints.

Evaluate All: Never pick the first option; compare all candidates.

Every subquery i must build upon the memory context of all preceding steps (1. . . i−1).

Each task necessitates the agent to sequentially complete multiple purchase sub-goals (e.g., 6 items) within a single shopping scenario, while simultaneously satisfying global constraints.

Subqueries MUST be completely self-contained and answerable independently- do not use pronouns or references like "this person", "the author", "these conditions", "they", "the movie", etc.

GPT-5.1-mini exhibits 'satisficing' behavior, purchasing the first relevant result (Option 1) immediately.

Massive token input (20k+ chars) causes ”Lost in the Middle” and instruction drift.

A memory retrieval failure causes drift from the finalized seed plan (wrong date/origin in flight search) and has a downstream constraint violation when selecting dinner.

I was unable to find any reliable source that ties all of those specific biographical and discographic constraints to a single identifiable founding member.

XML-wrapped history contains noise regarding 19th-century maritime disasters (Schooner Abraham Newland 1801; Capt. Morgan).

With probability at least 1−δ/2 there exists an error term ε(depending on the VC dimension dofH, the sample size m, k and|L| andδ) such that for all i∈[k], ℓ∈ L and all h∈H we have |Lℓi(h)−ˆLℓi(h)| ≤ε.

Running Hedge across T rounds with step size eta and applying the Hedge regret bound gives that the average regret against the best fixed loss/distribution is small.

Using the uniform deviation bound on both sides of the inequality in step 2 we get with probability at least1−δ...

In other words, the policy output by Algorithm 2 achieves the near-optimal worst-case expected loss across all distributions and losses, within an additive (ε) margin, with high probability, provided the sample complexity exceeds the above threshold.

In realistic settings, memorization and action are tightly coupled: agents acquire memory while interacting with the environment, and subsequently rely on that memory to solve future tasks.

Despite their strong performance on existing memory benchmarks, these agents exhibit low task completion rates in MEMORYARENA, revealing persistent difficulties in maintaining and exploiting latent task state across sessions.

This pattern suggests that while agents can make some progress on individual subtasks, they fail to integrate these partial successes into globally consistent solutions dramatically.

small errors in the agent’s implicit state estimate accumulate across sessions and eventually dominate downstream decisions