Mycelora benchmarks
This English version is provided for convenience. In case of any discrepancy between the English and French versions, the French version prevails.
Version: 1.0
Updated: November 6, 2026
On BEAM, the most demanding conversational memory benchmark published to date, Mycelora scores 75.6% on 100,000-token histories and 66.1% on one million tokens. Among systems whose protocol is published and gives the model no part of the answer key, this is the best published score at both sizes.
BEAM: score by history size
| System | 100K | 1M | Answer model | Judge | Questions |
|---|---|---|---|---|---|
| Mycelora | 75.6% | 66.1% | Claude Opus 5.5 | gpt-5, Mem0's judge prompt | 340 per size (17 conversations) |
| Mem0 | not published | 64.1% | not stated | gpt-5 by default in its harness, Mem0's judge prompt | not stated |
| Honcho | 63.0% | 61.8% | not stated for BEAM | not named, described as the paper's judge | 400 at 100K (20 conversations) |
These three systems score the answer of a model that receives only the question and what the memory serves it. Judges partly differ: the gap with Honcho should be read as an order of magnitude. With Mem0, judge and judge prompt are the same.
Share of questions scored 0.5 or more: 85.0% at 100K, 73.2% at 1M (Mem0 reports 70.1% at 1M on the same measure).
Mycelora's 1M score was measured on the October 27, 2026 version, before the November 5, 2026 improvements; it will be measured again.
BEAM 100K: score by ability
| Ability | Mycelora (n = 34) | Honcho |
|---|---|---|
| Contradiction resolution | 90.4% | 70.6% |
| Instruction following | 90.4% | 84.4% |
| Preference following | 87.5% | 85.6% |
| Information extraction | 83.6% | 78.4% |
| Temporal reasoning | 80.9% | 64.4% |
| Multi-session reasoning | 69.2% | 63.1% |
| Knowledge update | 68.4% | 46.3% |
| Abstention | 67.6% | 36.3% |
| Event ordering | 63.3% | 52.3% |
| Summarization | 54.4% | 49.0% |
| Overall | 75.6% | 63.0% |
BEAM 1M: score by ability
| Ability | Mycelora (n = 34) | Mem0 |
|---|---|---|
| Instruction following | 86.8% | 85.2% |
| Preference following | 80.4% | 88.3% |
| Knowledge update | 73.5% | 65.0% |
| Contradiction resolution | 66.9% | 35.7% |
| Temporal reasoning | 66.9% | 61.8% |
| Abstention | 66.2% | 52.5% |
| Information extraction | 63.2% | 70.0% |
| Multi-session reasoning | 55.3% | 65.2% |
| Event ordering | 53.3% | 53.6% |
| Summarization | 48.6% | 63.5% |
| Overall | 66.1% | 64.1% |
Mycelora's method
| Item | Choice |
|---|---|
| Benchmark | BEAM (ICLR 2026), official code and data, 100K and 1M splits |
| Conversations scored | 100K: 17 of 20; conversations 1, 5 and 14 were used for tuning and are excluded from the score. 1M: 17 conversations run with the same settings |
| Memory tested | Mycelora in production, as a user experiences it, with no benchmark-specific tuning |
| Path tested | Each session is closed like a real work thread; for each question, the model receives Mycelora's opening block, then its recall |
| What the answer model receives | The question and what Mycelora serves it, inside BEAM's official answer prompt (answer_generation_for_rag), unchanged; nothing else: no question type, no annotation from the dataset |
| Answer model | Claude Opus 5.5, no tools |
| Judge | gpt-5 with Mem0's judge prompt; never the answer model, because a model does not grade its own answers |
| Scoring | Criteria scored 0, ½ or 1; mean per ability, then mean of the ten abilities |
| Runs | One run, memory emptied before each conversation |
Why this judge. On 23 criteria checked by hand, gpt-5 with Mem0's judge prompt agrees with the truth in 98% of cases; the BEAM paper's judge (gpt-4.1-mini) in 57 to 67%, and it changes its verdict from one run to the next.
Known weaknesses
- Summarization and event ordering are our weakest abilities. Most of the material is served to the model but not fully used in the answer.
- Abstention: a product choice. When the requested information does not exist, Mycelora has the model answer that nothing is recorded on that point, then give what comes closest. BEAM counts this addition as a mistake; for a real user, it is most often useful. We keep it.
Limits
- Noise. Two identical runs differ by 1 to 5 points on the overall score, more per ability.
- One run per score, on 17 conversations per size.
- English conversations, on topics generated by BEAM.
- A given model: the score measures Mycelora with Claude Opus 5.5.
Other published results, not used in the comparison
These results come from a protocol that gives the answer model parts of the answer key, or that is not documented enough to be compared. They are reported as published.
| System | Published figure | Why it is not compared |
|---|---|---|
| past.dev | 100K: 92.1%; 1M: 90.7% | Exabase's answer prompt, reused unchanged and described in past.dev's own repository: depending on the question type, the model receives the topics to order, the points the summary must cover, the preference or instruction being tested, the reason a question cannot be answered, the key dates and a calculation hint. The same model answers and judges |
| Exabase M-1 | 100K: 76.9% | Same answer prompt as above; the same model answers and judges |
| Hindsight | 100K: 73.4%; 1M: 73.9% | AMB harness, published by Hindsight's company; the version that produced the 100K figure gave the model parts of the answer key, removed on October 9, 2026 |
| Cognee | 100K: 79% | A single conversation run end to end; judge and answer model not published |
Reproduce
Raw answers, criteria scored by the judge with its reason, and the answer prompt, for 100K and 1M: github.com/EvidencAI/mycelora-benchmarks, under CC BY-SA 4.0, the license of the BEAM dataset.
Sources
- BEAM: Tavakoli et al., ICLR 2026, arXiv 2510.27246.
- Mem0: docs.mem0.ai; harness mem0ai/memory-benchmarks.
- Honcho: honcho.dev/evals; results plastic-labs/honcho-benchmarks.
- past.dev: past.dev/benchmarks; pastdotdev/benchmarks.
- Exabase: exabase.io.
- Hindsight: hindsight.vectorize.io; vectorize-io/agent-memory-benchmark.
- Cognee: cognee.ai.