Research
Four long-memory benchmarks, five results.
- 98.4% LongMemEval-S Claude Opus 5.5
- 96.7% LoCoMo Claude Opus 5.5
- 76.2 BEAM 128K Claude Opus 5.5
- 74.7 BEAM 1M Claude Opus 5.5
- 60.2 BEAM 10M Claude Opus 5.5
Claude Opus 5.5 runs use deep search, Renor’s default. Markdown · Every run, for agents
Results by benchmark
LongMemEval-S LoCoMo BEAM 128K BEAM 1M BEAM 10M
LongMemEval-S
98.4%
Answers judged correct
Read by Claude Opus 5.5, deep search (the default)
8,722
tokens in the mean reader prompt
from a history of about 115,000 tokens
Questions about long chat histories: what was said, facts that changed, dates and order, preferences, and when the answer is not there.
- What the user said 69/70
- What the assistant said 54/56
- Preferences 30/30
- Knowledge updates 78/78
- Across several sessions 131/133
- Dates and order 130/133
How it ran
Dataset
500 questions, each over a chat history of about 115,000 tokens
Score
492 of 500 correct (95 percent Wilson interval 96.9 to 99.2)
Reader
Claude Opus 5.5 (xhigh effort), Message Batches
Retrieval
Deep search through the public CLI (top-K 50, 4,000-token budget); mean page 5,537 tokens
Judge
gpt-4o-2024-08-06 with the official LongMemEval prompt
Draws
One. Not held out.
Date
27 Oct 2026
Record
Evaluation ledger, section BM
LoCoMo
96.7%
Answers judged correct
Read by Claude Opus 5.5, deep search (the default)
12,599
tokens in the mean reader prompt
Questions about long conversations between two people over many sessions.
- Single-hop 98.0
- Multi-hop 96.5
- Open-domain 94.8
- Temporal 94.1
How it ran
Dataset
1,540 questions, categories 1 to 4
Score
1,489 of 1,540 correct (95 percent Wilson interval 95.7 to 97.5)
Reader
Claude Opus 5.5 (xhigh effort), Message Batches
Retrieval
Deep search through the public CLI (top-K 50, 4,000-token budget); a 16,000-token context cap, picture turns kept as their captions
Judge
gpt-4o-mini with the J prompt of the Mem0 paper
Draws
One. Not held out.
Date
28 Oct 2026
Record
Evaluation ledger, section BT
BEAM 128K
76.2
Nugget score out of 100
Read by Claude Opus 5.5, deep search (the default)
9,363
tokens in the mean reader prompt
from the 128K-token tier
Ten memory abilities over very long chats, from extracting a detail to ordering events and following an instruction given long ago.
- Extracting details 87.3
- Knowledge updates 85.0
- Preferences 95.0
- Following instructions 78.8
- Contradictions 80.9
- Saying it does not know 77.5
- Dates and time 83.8
- Across sessions 66.7
- Ordering events 54.2
- Summarizing 54.5
How it ran
Dataset
400 questions over 20 chats, the 128K tier
Score
Nugget score 0.762; the one reply the scorer could not parse counts as 0 (0.764 over the 399 scored); no interval computed
Reader
Claude Opus 5.5 (xhigh effort), Message Batches
Retrieval
Deep search through the public CLI (top-K 50, 4,000-token budget)
Judge
DeepSeek V4.1 Flash on OpenCode Go with the BEAM rubric, a stand-in for BEAM’s official evaluator
Draws
One. Not held out.
Date
28 Oct 2026
Record
Evaluation ledger, section BT
BEAM 1M
74.7
Nugget score out of 100
Read by Claude Opus 5.5, deep search (the default)
9,694
tokens in the mean reader prompt
from the 1M-token tier
Ten memory abilities over very long chats, from extracting a detail to ordering events and following an instruction given long ago.
- Extracting details 81.6
- Knowledge updates 91.4
- Preferences 87.1
- Following instructions 91.0
- Contradictions 75.0
- Saying it does not know 70.0
- Dates and time 70.4
- Across sessions 70.4
- Ordering events 60.3
- Summarizing 49.5
How it ran
Dataset
700 questions over 35 chats, the 1M tier
Score
Nugget score 0.747 over 2,415 nuggets (95 percent bootstrap interval 0.722 to 0.771)
Reader
Claude Opus 5.5 (xhigh effort), Message Batches
Retrieval
Deep search through the public CLI (top-K 50, 4,000-token budget)
Judge
DeepSeek V4.1 Flash on OpenCode Go with the BEAM rubric, a stand-in for BEAM’s official evaluator
Draws
One. Not held out.
Date
28 Oct 2026
Record
Evaluation ledger, section BT
BEAM 10M
60.2
Nugget score out of 100
Read by Claude Opus 5.5, deep search (the default)
11,251
tokens in the mean reader prompt
from the 10M-token tier
Ten memory abilities over very long chats, from extracting a detail to ordering events and following an instruction given long ago.
- Extracting details 78.8
- Knowledge updates 83.8
- Preferences 71.3
- Following instructions 76.3
- Contradictions 58.8
- Saying it does not know 65.0
- Dates and time 46.3
- Across sessions 37.7
- Ordering events 36.5
- Summarizing 48.0
How it ran
Dataset
200 questions over 10 chats (208,696 turns), the full 10M tier
Score
Nugget score 0.602 over 621 nuggets (95 percent bootstrap interval 0.545 to 0.659); one empty reply counts as a miss
Reader
Claude Opus 5.5 (xhigh effort), Message Batches
Retrieval
Deep search through the public CLI (top-K 50, 4,000-token budget)
Judge
DeepSeek V4.1 Flash on OpenCode Go with the BEAM rubric, a stand-in for BEAM’s official evaluator
Draws
One. Not held out.
Date
28 Oct 2026
Record
Evaluation ledger, section BT
Why the numbers look this way
- Facts keep their history A new value replaces the old one as the same fact. The newest answers; the earlier ones stay, with their dates. Knowledge updates: 78/78 on LongMemEval-S
- Two searches, one ranking Facts first, then notes found by meaning and by exact words, ranked together. Deep search gives the conversation around each match more room. Every answer turn was on the page for 463 of 470 answerable LongMemEval-S questions
- Small pages The reader gets a page of a few thousand tokens, never the whole history, so a strong model reads only what matters. A mean prompt of 11,251 tokens on BEAM 10M
How Renor works
Save An agent saves a fact or a note. Search Renor ranks what matches. Recall The agent answers from a small page.
Save. A fact is a subject, a predicate and a value; a note is your words. Both are Markdown, and the search index is rebuilt from them.
Search. Current facts first, then notes by meaning (Vertex embeddings) and by words (BM25), cut to a token budget.
Recall. The page holds the current facts and the closest notes. If nothing matches, it says so instead of guessing.
Questions
What is deep search, and what does lean mode score?
Deep search asks for 50 matches with a 4,000-token budget instead of 20 with 1,300. It is Renor’s default. Lean mode (RENOR_DEEP_SEARCH=0) is the smaller page that was the default before 9 Nov 2026. With lean mode and the same reader, LongMemEval-S scores 94.4% (472 of 500).
Why is the score of record 60.4%?
It is the official nightly run of 19 Oct 2026, with that day’s defaults and a much cheaper reader, gpt-4o-mini; regressions are tracked against it. The reader matters: on identical pages, Claude Haiku 5.5 scored 83.9% and Claude Opus 5.5 94.4% on the same lean-mode pages. Three earlier official runs scored 61.2% to 62.4%.
Are these held-out scores?
No. Renor’s defaults were developed on these same 500 questions, so these are not held-out scores, and the deep-search setting was chosen reading them. On the 400 questions outside the 100-question development set, deep search scored 394/400.
Why one run each?
Each strong-model run is paid, so each benchmark ran once with no replicate. Read every score with its interval, under How it ran.
Why is BEAM out of 100?
BEAM scores each answer against key points from 0 to 1; we show it times 100. The judge is a stand-in, not BEAM’s official evaluator, so the figures hold for that judge only.
Can I compare these with other memory systems?
Only with the same reader and the same judge. Earlier runs, such as LoCoMo 78.8% with DeepSeek V4.1 Flash, are in the details for agents.
The evaluation code is open source.
The scripts behind these runs are in Renor’s open-source engine.
Renor home Durable memory for AI. Your agents stop starting from zero.
Product How it works Docs Research Culture Press Install Blog Pricing Status
Account Get started Sign in
Legal Security Privacy Data protection Terms Refunds Support
© 2026 Renor