
Scoping And Identity
Mem0 (Platform + OSS) · Mem0
Agent Memory — Mem0
Evaluates Mem0's Scoping & Identity across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Agent Memory eval coverage.
About Mem0
Mem0 is a memory layer for AI agents and assistants — it extracts, stores, and retrieves long-term facts across sessions via an add/search API, with user/agent/run scoping and optional graph memory, available as a managed Platform and open source.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | A SaaS app maps each end-user to a stable user_id and uses it for all add/search calls across sessions. | Use a stable, unique user_id (e.g., your auth subject id) as the long-term memory key so memories persist across sessions and devices for that person. Do not derive user_id from a per-session token or display name that can change or collide. | Pass / FailAi Platformhigh |
| 02 | An agent wants short-lived, conversation-specific memory that should not bleed into the user's permanent profile. | Use run_id to scope memories to a single session/conversation. Combine with user_id when the memory is both per-user and per-session. Treat run_id-only memories as ephemeral context that should not be promoted to long-term without intent. | Pass / FailAi Platformhigh |
| 03 | A platform runs several specialized agents; each should accumulate its own domain knowledge separate from the others. | Use agent_id to partition per-agent memory (e.g., a 'sales' agent vs a 'support' agent). Decide explicitly whether agent memory is global or per-user (agent_id + user_id). Keep agent_id stable per logical agent so its knowledge accrues. | Pass / FailAi Platformhigh |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mem0
- Ai Platform
- Scoping And Identity
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Scoping And Identity eval for Mem0 Mem0 (Platform + OSS) test?+
Evaluates Mem0's Scoping & Identity across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Agent Memory eval coverage.
How is the Scoping And Identity eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Scoping And Identity pack for Mem0 Mem0 (Platform + OSS) contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Scoping And Identity pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.