# Renor research: every result, with its setup and caveats This file is for agents and anyone who wants the details. https://renor.cloud/research shows a person one figure per benchmark, and the home page the same LongMemEval-S figure; this file says what each figure is, and lists every other result Renor reports. Each figure is one run unless it says otherwise. A comparison holds only within one reader and one grader. The per-question records are in Renor's evaluation ledger, which is not public; the evaluation code is in the open-source engine: https://github.com/renorai/renor-engine/tree/main/scripts/eval ## What /research and the home page show - LongMemEval-S 98.4% (492/500, Wilson 96.9 to 99.2). Reader Claude Opus 5.5 at xhigh effort; grader gpt-4o-2024-08-06 with the official LongMemEval prompt. Pages from Renor main 209c0b344 through the public CLI (`renor index`, then `renor search --json`) with `--top-k 50` and a 4,000-token ranked-expansion budget (RENOR_TOPK_SCALES_BUDGET=1), a deeper search. That switch is off by default: this is not what a default install does. Renor's default settings with the same reader score 94.4% (472/500, below). One run, 2026-10-27, no replicate. Not a held-out score: the setting was chosen reading all 500 questions, and Renor's defaults were developed on them too; on the 400 questions outside the 100-question development set it scored 394/400. Ledger section BM. - LoCoMo 96.7% (1,489/1,540, categories 1 to 4, Wilson 95.7 to 97.5), deep search. Reader Claude Opus 5.5 at xhigh effort; judge gpt-4o-mini with the Mem0 paper's J prompt. Pages through the public CLI with `--top-k 50` and the 4,000-token budget (RENOR_TOPK_SCALES_BUDGET=1, off by default), a 16,000-token context cap, the picture turns kept as captions. One run, 2026-10-28, no replicate. Not held out: the setting was chosen on the LongMemEval-S pages (ledger section BM, 27 Oct). Ledger section BT. The 78.8% row below is an earlier configuration. - Text read per question, LongMemEval-S, mean tokens: Renor 5,537 (the page the reader got in the 98.4% run, all 500 questions; the default page is 2,508); Mem0 6,787, its own figure at its top-200 budget (https://docs.mem0.ai/core-concepts/memory-evaluation); the whole conversation about 115,000 (full-context average, Table 2 of the Zep paper, arXiv 2501.13956). ## LongMemEval-S, every other result 500 questions about long chat histories (about 115,000 tokens each). Grader gpt-4o-2024-08-06 with the official prompt unless stated. None of these is held out: Renor's defaults and levers were developed on these 500 questions. - Score of record: 60.4% (302/500, Wilson 56.0 to 64.6). The official nightly run of 19 Oct 2026, Renor main e36333cf with its defaults of that day, public CLI, top-K 20, reader gpt-4o-mini. It is the number regressions are tracked against. Three earlier official runs of the defaults, 16 to 18 Oct, scored 61.2%, 61.8% and 62.4%. Ledger sections Z and T. - 94.4% (472/500, Wilson 92.0 to 96.1): Renor's shipped default pages (main c9b68c685, top-K 20, ranked-expansion budget 1,300), reader Claude Opus 5.5 at xhigh effort, 2026-10-26, one run. On the 400 questions outside the development set: 380/400 = 95.0%. Ledger section BG. - 83.9%: the same shipped default pages with Claude Haiku 5.5 as the reader, mean of three runs (418, 417 and 423 correct); 37 questions flip between runs. On these identical pages the gap to Opus (83.9% against 94.4%) is the reader alone. Ledger section BG. - 64.4% (322/500): the score of record's setup plus the event and duration lines, which are on by default since; measured 19 Oct on a tree before they merged. Today's main has not had an official gpt-4o-mini run. Ledger section Z. - 77.2% (386/500): research configuration of 19 Oct (summaries written when a note is saved, off by default, plus the event and duration lines), reader gpt-4o-mini. Every lever was chosen on these 500 questions. With Claude Haiku 5.5 reading the same pages: 87.8% (439/500, ledger section AG); the 26 Oct three-run mean of that set with Haiku is 87.5%. RESEARCH-ONLY. Ledger section AC. - 86.2% (431.0 correct, mean of three runs): public CLI default with DeepSeek V4.1 Flash on OpenCode Go as the reader and the same model as grader (it agrees with the official grader on 491 of 500 answers). A screening row, not the official instrument. Ledger sections BI and BK. Why these scores differ: the reader, the page and the product version. On identical pages the reader alone took 83.9% (Haiku) to 94.4% (Opus). The deeper page took Opus from 472 to 492 correct (23 questions gained, 3 lost; the default pages of the newer tree are byte for byte the 94.4% run's). The 302/500 run used gpt-4o-mini on main e36333cf (18 Oct); the Claude runs used main c9b68c685 (22 Oct) or later, so both the reader and the product changed between them, and that gap is not the reader alone. ## LoCoMo, every other result - 78.8% (1,213/1,540, categories 1 to 4, Wilson 76.7 to 80.7): earlier configuration of 18 Oct 2026, Renor's shipped CLI with the picture turns kept as captions, top-K 20. Reader DeepSeek V4.1 Flash (Fireworks); grader gemma4:31b, which is not the grader published LoCoMo scores use, so it is not comparable with them. One run. Adversarial questions, scored separately: "I don't know" on 137/446 = 30.7%. Ledger section Y. - 88.1% (1,357/1,540, the same 2026-10-28 answers): the judge changed, not the answers. DeepSeek V4.1 Flash on OpenCode Go as a continuity judge, five unparsed replies counted wrong. Shows how much the judge moves LoCoMo. Ledger section BT. - 75.2% (mean of three runs, 75.6, 75.2 and 74.8; categories 1 to 4): byte-identical pages to the 78.8% row, read by DeepSeek V4.1 Flash on OpenCode Go and graded by the same model, 2026-10-27. The reader host and the grader both differ from the 78.8% row and the cause of the 3.55-point gap is not separated. Adversarial "I don't know" 20.0%. A screening row. Ledger section BO. ## BEAM BEAM scores are nugget scores from 0 to 1, graded by a stand-in judge, not BEAM's official evaluator. Each figure is one run, not held out. The 2026-10-28 rows read deep pages (`--top-k 50`, the 4,000-token budget) with Claude Opus 5.5 at xhigh effort, judged by DeepSeek V4.1 Flash on OpenCode Go. Ledger section BT. - BEAM 128K (400 questions, 399 scored): 0.762. The one reply the scorer could not parse counts as 0; over the 399 scored it is 0.764. 2026-10-28. - BEAM 1M (700 questions): 0.747 (bootstrap 95% 0.722 to 0.771). 2026-10-28. - BEAM 10M (200 questions, all ten chats): 0.602 (bootstrap 95% 0.545 to 0.659). One question returned no text and counts as a miss. 2026-10-28. Earlier rows, with the shipped CLI and DeepSeek readers (OpenCode Go or Nebius), not comparable with the rows above: - BEAM 128K (400 questions): 0.498 (mean of three runs, lowest reasoning) and 0.524 (one run, default reasoning). Shipped CLI, 6,000-token page, reader DeepSeek V4.1 Flash on OpenCode Go, judge gemma4:31b. Weakest abilities: summarization and temporal reasoning. A screening row, 2026-10-27. Ledger section BP. - BEAM 1M (700 questions): 0.468, one run, shipped CLI, reader DeepSeek V4.1 Flash on OpenCode Go, judged by DeepSeek V4.1 Flash on Nebius (not comparable with the gemma4:31b rows). A screening row, 2026-10-27. Ledger section BR. - BEAM 10M (200 questions): 0.292, a lower bound only: 68 of 200 answers came back empty and scored 0. Development row, 2026-10-20; no rerun yet. ## The coding-agent test Claude Code (Claude Haiku 5.5) does 50 small coding tasks, three times each, where the right answer was settled in an earlier session; scored exactly against each task's answer, no model grader. With Renor's hosted configuration (hybrid search with real embeddings): 97% (96 to 98), 2026-10-25. With Renor's keyword-only server: 81% (78 to 84). Without any memory: 45%. With hand-written notes: 96%. With everything in CLAUDE.md: 99%. These come from separate runs, not one paired table, so Renor is not shown to differ from notes or CLAUDE.md. With memory 30 times larger (42,870 entries): Renor hybrid 95%, notes 96%. The records of this test are not in the evaluation ledger. ## What is not here Competitor accuracy figures appear only as each provider's own published result, each with its source link. Answer models differ between providers, so compare the figures with care. Mem0's token figure is used only because it is the same quantity, the text sent to the answering model per question.