
Graph Memory
Mem0 (Platform + OSS) · Mem0
Agent Memory — Mem0
Evaluates Mem0's Graph Memory across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Agent Memory eval coverage.
About Mem0
Mem0 is a memory layer for AI agents and assistants — it extracts, stores, and retrieves long-term facts across sessions via an add/search API, with user/agent/run scoping and optional graph memory, available as a managed Platform and open source.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Operator wants entity/relationship memory and configures a graph store (Neo4j) via the graph_store config / enable_graph option in Memory.from_config. | Enable graph memory through the documented config (graph_store provider + connection, or enable_graph on the Platform) so add() also extracts entities and relationships into the graph alongside the vector store. Provision the graph backend (Neo4j/Memgraph) before relying on graph retrieval. | Pass / FailAi Platformhigh |
| 02 | A query 'who does Dana report to?' is relational; another query 'what food does the user like?' is attributive. | Use graph retrieval for multi-hop/relational questions and vector retrieval for attributive recall; with graph memory enabled Mem0 can combine both. Do not force every query through vector similarity when the answer is a graph traversal. | Pass / FailAi Platformmedium |
| 03 | User says 'My manager Dana works at the Berlin office'; with graph memory on, Mem0 should extract entities (user, Dana, Berlin office) and relations (manages, works_at). | Verify the add() result/graph contains the extracted entities and typed relationships, not just a flat text memory. Use the graph to answer relational queries (e.g., 'who is my manager?'). Do not assume vector memory alone captures the relation structure. | Pass / FailAi Platformhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mem0
- Ai Platform
- Graph Memory
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Graph Memory eval for Mem0 Mem0 (Platform + OSS) test?+
Evaluates Mem0's Graph Memory across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Agent Memory eval coverage.
How is the Graph Memory eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Graph Memory pack for Mem0 Mem0 (Platform + OSS) contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Graph Memory pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.