Research

Four long-memory benchmarks, five results.

Claude Opus 5.5 runs use deep search, Renor’s default. Markdown · Every run, for agents

Results by benchmark

LongMemEval-S LoCoMo BEAM 128K BEAM 1M BEAM 10M

LongMemEval-S

98.4%

Answers judged correct

Read by Claude Opus 5.5, deep search (the default)

8,722

tokens in the mean reader prompt

from a history of about 115,000 tokens

Questions about long chat histories: what was said, facts that changed, dates and order, preferences, and when the answer is not there.

How it ran

Dataset

500 questions, each over a chat history of about 115,000 tokens

Score

492 of 500 correct (95 percent Wilson interval 96.9 to 99.2)

Reader

Claude Opus 5.5 (xhigh effort), Message Batches

Retrieval

Deep search through the public CLI (top-K 50, 4,000-token budget); mean page 5,537 tokens

Judge

gpt-4o-2024-08-06 with the official LongMemEval prompt

Draws

One. Not held out.

Date

27 Oct 2026

Record

Evaluation ledger, section BM

LoCoMo

96.7%

Answers judged correct

Read by Claude Opus 5.5, deep search (the default)

12,599

tokens in the mean reader prompt

Questions about long conversations between two people over many sessions.

How it ran

Dataset

1,540 questions, categories 1 to 4

Score

1,489 of 1,540 correct (95 percent Wilson interval 95.7 to 97.5)

Reader

Claude Opus 5.5 (xhigh effort), Message Batches

Retrieval

Deep search through the public CLI (top-K 50, 4,000-token budget); a 16,000-token context cap, picture turns kept as their captions

Judge

gpt-4o-mini with the J prompt of the Mem0 paper

Draws

One. Not held out.

Date

28 Oct 2026

Record

Evaluation ledger, section BT

BEAM 128K

76.2

Nugget score out of 100

Read by Claude Opus 5.5, deep search (the default)

9,363

tokens in the mean reader prompt

from the 128K-token tier

Ten memory abilities over very long chats, from extracting a detail to ordering events and following an instruction given long ago.

How it ran

Dataset

400 questions over 20 chats, the 128K tier

Score

Nugget score 0.762; the one reply the scorer could not parse counts as 0 (0.764 over the 399 scored); no interval computed

Reader

Claude Opus 5.5 (xhigh effort), Message Batches

Retrieval

Deep search through the public CLI (top-K 50, 4,000-token budget)

Judge

DeepSeek V4.1 Flash on OpenCode Go with the BEAM rubric, a stand-in for BEAM’s official evaluator

Draws

One. Not held out.

Date

28 Oct 2026

Record

Evaluation ledger, section BT

BEAM 1M

74.7

Nugget score out of 100

Read by Claude Opus 5.5, deep search (the default)

9,694

tokens in the mean reader prompt

from the 1M-token tier

Ten memory abilities over very long chats, from extracting a detail to ordering events and following an instruction given long ago.

How it ran

Dataset

700 questions over 35 chats, the 1M tier

Score

Nugget score 0.747 over 2,415 nuggets (95 percent bootstrap interval 0.722 to 0.771)

Reader

Claude Opus 5.5 (xhigh effort), Message Batches

Retrieval

Deep search through the public CLI (top-K 50, 4,000-token budget)

Judge

DeepSeek V4.1 Flash on OpenCode Go with the BEAM rubric, a stand-in for BEAM’s official evaluator

Draws

One. Not held out.

Date

28 Oct 2026

Record

Evaluation ledger, section BT

BEAM 10M

60.2

Nugget score out of 100

Read by Claude Opus 5.5, deep search (the default)

11,251

tokens in the mean reader prompt

from the 10M-token tier

Ten memory abilities over very long chats, from extracting a detail to ordering events and following an instruction given long ago.

How it ran

Dataset

200 questions over 10 chats (208,696 turns), the full 10M tier

Score

Nugget score 0.602 over 621 nuggets (95 percent bootstrap interval 0.545 to 0.659); one empty reply counts as a miss

Reader

Claude Opus 5.5 (xhigh effort), Message Batches

Retrieval

Deep search through the public CLI (top-K 50, 4,000-token budget)

Judge

DeepSeek V4.1 Flash on OpenCode Go with the BEAM rubric, a stand-in for BEAM’s official evaluator

Draws

One. Not held out.

Date

28 Oct 2026

Record

Evaluation ledger, section BT

Why the numbers look this way

How Renor works

Save An agent saves a fact or a note. Search Renor ranks what matches. Recall The agent answers from a small page.

Save. A fact is a subject, a predicate and a value; a note is your words. Both are Markdown, and the search index is rebuilt from them.

Search. Current facts first, then notes by meaning (Vertex embeddings) and by words (BM25), cut to a token budget.

Recall. The page holds the current facts and the closest notes. If nothing matches, it says so instead of guessing.

Questions

What is deep search, and what does lean mode score?

Deep search asks for 50 matches with a 4,000-token budget instead of 20 with 1,300. It is Renor’s default. Lean mode (RENOR_DEEP_SEARCH=0) is the smaller page that was the default before 9 Nov 2026. With lean mode and the same reader, LongMemEval-S scores 94.4% (472 of 500).

Why is the score of record 60.4%?

It is the official nightly run of 19 Oct 2026, with that day’s defaults and a much cheaper reader, gpt-4o-mini; regressions are tracked against it. The reader matters: on identical pages, Claude Haiku 5.5 scored 83.9% and Claude Opus 5.5 94.4% on the same lean-mode pages. Three earlier official runs scored 61.2% to 62.4%.

Are these held-out scores?

No. Renor’s defaults were developed on these same 500 questions, so these are not held-out scores, and the deep-search setting was chosen reading them. On the 400 questions outside the 100-question development set, deep search scored 394/400.

Why one run each?

Each strong-model run is paid, so each benchmark ran once with no replicate. Read every score with its interval, under How it ran.

Why is BEAM out of 100?

BEAM scores each answer against key points from 0 to 1; we show it times 100. The judge is a stand-in, not BEAM’s official evaluator, so the figures hold for that judge only.

Can I compare these with other memory systems?

Only with the same reader and the same judge. Earlier runs, such as LoCoMo 78.8% with DeepSeek V4.1 Flash, are in the details for agents.

The evaluation code is open source.

The scripts behind these runs are in Renor’s open-source engine.

View on GitHub

Renor home Durable memory for AI. Your agents stop starting from zero.

Product How it works Docs Research Culture Press Install Blog Pricing Status

Account Get started Sign in

Legal Security Privacy Data protection Terms Refunds Support

© 2026 Renor