Platform Org Project Webhooks Config
Mem0 (Platform + OSS) · Mem0
Agent Memory — Mem0
Mem0 evals — Platform: Org/Project, Webhooks & Config (relift v3 InfraRed)
About Mem0
Mem0 is a memory layer for AI agents and assistants — it extracts, stores, and retrieves long-term facts across sessions via an add/search API, with user/agent/run scoping and optional graph memory, available as a managed Platform and open source.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Integrator calls the managed Mem0 Platform and must authenticate the MemoryClient with the MEM0_API_KEY. | Authenticate with the documented scheme — an 'Authorization: Token <MEM0_API_KEY>' header (or MEM0_API_KEY via MemoryClient). Keep the key in a secret store, never in client-side code or logs, and rotate on exposure. | Pass / FailAi Platformcritical |
| 02 | Operator registers a webhook to mirror memory changes and expects events for memory_add, memory_update, and memory_delete. | Subscribe to the documented memory lifecycle events and handle each type explicitly to keep an external mirror in sync (create on add, modify on update, remove on delete). Do not assume only adds fire; updates and deletes must be applied too. | Pass / FailAi Platformhigh |
| 03 | An OSS operator switches the embedder model after memories were already stored with the previous embedder. | Recognize that existing vectors were produced by the old embedder; changing the embedder generally requires re-embedding existing memories (or the new model and old vectors will be incompatible/lower-quality). Plan a migration rather than swapping models silently. | Pass / FailAi Platformmedium |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mem0
- Ai Platform
- Platform Org Project Webhooks Config
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.