Broad coverage
Textual and coding memory tasks cover long conversations, changing user states, and reusable engineering experience.
Open · Fair · Reproducible
One rulebook for
memory.
AML measures how well memory systems store, retrieve, and support the use of long-term information. Research methods and commercial products are compared under one unified, traceable evaluation contract.
The first official leaderboard is now live.
Leaderboard released
AMC / 2026
Agent Memory Challenge 2026
The first leaderboard
is now live
02Evaluation types
10+Text datasets
≈5KText questions
04Evaluation stages
01 / Why AML
A benchmark system,
not another isolated score.
Memory systems are often reported with different datasets, answer models, and judges. AML brings public benchmarks, controlled evaluation data, a standard interface, unified answering, multi-agent judging, version review, and public governance into one long-running evaluation system.
Controlled comparison
Datasets, prompts, Answer models, judges, aggregation, and resource rules remain fixed within a release.
Actionable diagnosis
Capability-level profiles accompany the overall rank, making strengths and failure modes easier to understand.
02 / Unified evaluation
Your system owns memory.
The platform controls the rest.
Candidates expose only two memory operations. The platform fixes generation and scoring, making it easier to attribute performance differences to memory storage and retrieval.
Read the evaluation documentation- 01 / StoreAddCandidate system
- 02 / RetrieveSearchCandidate system
- 03 / GenerateAnswerOfficial platform
- 04 / ScoreEvalOfficial platform
03 / Benchmark overview
Two memories,
two separate leaderboards.
Textual and coding memory use different tasks and primary metrics. Academic methods and commercial products are also published separately, so unlike systems are never mixed into one rank.
01
Textual
Textual Memory
Measure what a system can reliably remember and use across long, evolving histories.
≈150Mhistory characters
7capability dimensions
- LoCoMo-Refined
- ScriptMem
- LongMemEval
- CLBench
- PersonaMem
- BEAM
02
Coding
Coding Memory
Measure whether historical engineering context helps an agent solve the software task in front of it.
12source repositories
1,290annotated history tasks
- 01Debug MemoryDiagnosis, fixes, and validation
- 02Development MemoryArchitecture, patterns, and conventions
04 / Capability profile
Explain the rank,
not only the winner.
Textual results are mapped into seven shared dimensions. The profile reveals where a system succeeds, where it fails, and whether it respects changing state, evidence, and privacy boundaries.
05 / Open evaluation release
AML-memory / agent-memory-leaderboard
Inspect the contract behind every comparable score.
The public repository exposes per-benchmark evaluation contracts, shared runtime configuration, and documentation for methodological review. Protected evaluation data, participant traces, and production infrastructure remain private to preserve benchmark integrity.
Browse the public repositoryPublic Answer and scoring behavior can be reviewed against the reported release.
Comparable results bind to a benchmark bundle, pipeline revision, and model configuration.
Held-out questions, gold answers, participant traces, and private annotations are not distributed.
Every published row identifies the evaluated method, product, commit, image, or API version.
06 / Participant divisions
Open methods and commercial products,
evaluated without false equivalence.
Academic Methods
For papers, research projects, and open-source systems. Results bind to public source, configuration, and reproducibility evidence.
Commercial Products
For stable hosted products and APIs. Internal implementation may remain closed, while the evaluated product and API version stay verifiable.
Agent Memory Challenge · Leaderboard released
Memory decides.
First verified leaderboard expected August 12, 2026.Public rankings
Agent Memory
Leaderboard
Release 01 · Expected August 12, 2026
Combined top-ten results from the two supplied workbooks, ranked together by total micro score. Division tags are shown inside each row for quick scanning.
Ranking view
Textual Memory · Academic Methods
Overall
Aggregate performance across the fixed textual evaluation suite.
All academic leaderboard results are evaluated using the same GPT-4o mini base model.
🥇 Best
🥈 Second
🥉 Third per metric
Textual and coding memory never share a ranking. Academic and commercial entries are also published separately.
Textual Memory ranks by Overall; Coding Memory ranks by Task Solve (%), with supporting detail retained.
Each public row identifies the evaluated source, product, image, commit, or API version and its review status.