> ## Documentation Index > Fetch the complete documentation index at: https://docs.cognee.ai/llms.txt > Use this file to discover all available pages before exploring further. # Accuracy Benchmarks > Cognee's published accuracy results on the BEAM and LoCoMo long-term memory benchmarks, with datasets, metrics, models and the commands to reproduce them. Cognee publishes its memory accuracy results on two long-term memory benchmarks. It scores **0.79 on BEAM 100K** and about **91% on LoCoMo**. Each result has a technical report, and its answers, judge verdicts and configs are published in the [Cognee repository](https://github.com/topoteretes/cognee/tree/dev/cognee/eval_framework), so you can check every number without rerunning anything, or rerun it yourself. ## Results | Benchmark | Dataset | Metric | Score | Runs | | - | - | - | -: | - | | BEAM 100K | 1 held-out conversation, 20 questions ([Mohammadta/BEAM](https://huggingface.co/datasets/Mohammadta/BEAM)) | Mean BEAM rubric score, 0–1, graded by an LLM judge | **0.79** | 4, std 0.005 | | BEAM 10M | 1 conversation, 20 questions ([Mohammadta/BEAM-10M](https://huggingface.co/datasets/Mohammadta/BEAM-10M)) | Mean BEAM rubric score, 0–1, graded by an LLM judge | **0.67** | 5, std 0.020 | | LoCoMo | All 10 conversations, 1,762 questions ([snap-research/locomo](https://github.com/snap-research/locomo)) | LLM-judge accuracy | **91.3%** | 2 answer passes: 90.7% and 91.3% | * **BEAM:** [technical report](https://github.com/topoteretes/cognee/blob/dev/cognee/eval_framework/beam/REPORT.md) with the full method, per-ability scores and limitations, and the [per-question results](https://github.com/topoteretes/cognee/tree/dev/cognee/eval_framework/beam/report_artifacts). The report is marked as a preliminary version. * **LoCoMo:** [technical report](https://github.com/topoteretes/cognee/blob/dev/cognee/eval_framework/locomo/REPORT.md) with the method, grading rules and audits, and the [per-question results](https://github.com/topoteretes/cognee/tree/dev/cognee/eval_framework/locomo/report_artifacts). ## BEAM [BEAM](https://arxiv.org/abs/2510.27246) tests memory over long, multi-session user–assistant conversations, with relevant evidence scattered across many turns. Its questions cover ten abilities: information extraction, multi-hop reasoning, knowledge updates, temporal reasoning, summarization, preference following, abstention, contradiction resolution, event ordering and instruction following. An LLM judge scores each rubric criterion 0, 0.5 or 1, and a question's score is the average over its criteria. **Method.** Each conversation is ingested with `add` and `cognify`, the two lower-level steps that `remember()` runs, so the benchmark can set its own chunking and session handling: one document per conversation batch and one chunk per user–assistant turn, plus the global context index and session memory (the 100K configuration builds the index but doesn't query it). Questions are answered with `hybrid_completion` retrieval, which combines chunk, graph and summary search. Answer prompts are chosen by BEAM question type, following the convention of other published BEAM results. * **100K** is the primary result. Retrieval settings (chunk top-k 20, entity top-k 20) and prompts were tuned on two conversations, checked on two more, and scored on a held-out conversation. * **10M** is an exploratory scale check. It routes each question type to different retrieval settings, which were selected on the same 20 questions that were scored, so it is not a held-out estimate. | Stage | 100K | 10M | | - | - | - | | Embeddings | `openai/text-embedding-3-large` | `openai/text-embedding-3-large` | | Ingestion | `openai/gpt-5` | `openai/gpt-5-mini` | | Answers | `openai/gpt-5` | `openai/gpt-5` | | Judge | `openai/gpt-5` | `openai/gpt-5` | ## LoCoMo [LoCoMo](https://github.com/snap-research/locomo) tests memory over ten long conversations between two people, with single-hop, multi-hop, temporal, open-domain and adversarial questions. **Method.** Each conversation is ingested through `remember()` as dated windows of 6 turns, one chunk per window, with Cognee's standard extraction and summary prompts. Every question uses `hybrid_completion` retrieval with 20 passages, 20 entities and 20 facts, and the same answer prompt; the question's category is never shown to the answering model. Answers come from `gpt-5-mini` and are graded by a `gpt-5.1` judge. The runs used Cognee 1.5.4 with Ladybug, LanceDB and SQLite. Two answer passes over the same memories scored 90.7% (1,598 of 1,762) and 91.3% (1,608 of 1,762). By question type, the 91.3% pass scored: | Question type | Questions | Accuracy | | - | -: | -: | | Single-hop | 796 | 92.5% | | Multi-hop | 236 | 89.8% | | Temporal | 284 | 90.1% | | Adversarial | 446 | 90.6% | | **Total** | **1,762** | **91.3%** | The total leaves out the 96 open-domain questions, which need general knowledge, and the 143 questions whose gold answer an audit flags, 224 questions in all, since 15 are in both groups. The 143 flagged questions are listed with each audit's reason in `report_artifacts/gold_exclusions.json`; open-domain questions are left out by category. At the hybrid retriever's own default depth (5 passages, 5 entities and 5 facts, below `recall()`'s default of 10 each), with the global context index, resolved dates and adapted extraction and summary prompts, one run scored 89.7% (1,580 of 1,761 scored questions). ## Limitations * **BEAM scores come from one conversation per scale, 20 questions each.** Per-ability scores rest on two questions each and are directional. * **BEAM is fully public.** Its questions and rubrics informed the prompts and retrieval settings, and there is no hidden test set. The 100K split reduces this risk; the 10M result has no such split. * **The 100K and 10M scores are not a controlled comparison.** The conversations, ingestion models and retrieval selection differ between them. * **LoCoMo is not held out.** Its ten conversations were used to develop the setup. * **LLM judges vary between runs.** Read small differences between scores with caution. * **The 10M ingestion can't be rerun exactly as published.** It ran in Cognee's distributed mode, and its BEAM-specific orchestration is not in the public code. ## Reproduce the results Run the benchmarks with Cognee's built-in evaluation framework. Both runs are LLM-cost-intensive. For BEAM 100K, preprocess and ingest the conversations, run the evaluation with the packaged config, then aggregate the runs into the mean score and standard deviation: ```bash theme={null} uv run python -m cognee.eval_framework.beam.preprocessing.preprocess \ --dataset beam --splits 100K --max-conversations 2 --execute-compressions \ --output-dir temp/beam_preprocessed_documents uv run python -m cognee.eval_framework.beam.local_ingest \ temp/beam_preprocessed_documents/100k/conversation_000001_id_ \ --dataset-name beam_100k_local --prune-first uv run python -m cognee.eval_framework.beam.eval.run_sweep \ --split 100K --conversation-index 1 --num-runs 4 \ --config-json-path cognee/eval_framework/beam/report_artifacts/100k_fixed/beam_hybrid_completion_20_20_qa_v1_config.json uv run python -m cognee.eval_framework.beam.eval.aggregate_cross_run \ --output-dir \ --conversation-index 1 --retriever hybrid_completion_20_20_qa_v1 ``` Without `--execute-compressions`, preprocessing only reports what it would compress. For LoCoMo, one command ingests, answers and grades all ten conversations: ```bash theme={null} uv run python -m cognee.eval_framework.locomo.run_locomo_eval --conversations 0-9 ``` See [Built-in Evaluation Framework](/integrations/eval-framework) for setup, options and how to benchmark retrieval on your own data. ## Next steps > **[Evaluation framework](/integrations/eval-framework)** > > Run these benchmarks and your own > **[Token efficiency](/core-concepts/evaluation-and-efficiency/token-efficiency-and-context-compression)** > > What memory costs per call, measured This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.