Eval Library
M
For Mem0AI Platform

Platform Org Project Webhooks Config

Mem0 (Platform + OSS) · Mem0

Agent Memory — Mem0

Mem0 evals — Platform: Org/Project, Webhooks & Config (relift v3 InfraRed)

About Mem0

Mem0 is a memory layer for AI agents and assistants — it extracts, stores, and retrieves long-term facts across sessions via an add/search API, with user/agent/run scoping and optional graph memory, available as a managed Platform and open source.

Employees

~30

Industry

Agent Memory

Headquarters

San Francisco, CA

Website

mem0.ai

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Integrator calls the managed Mem0 Platform and must authenticate the MemoryClient with the MEM0_API_KEY.

Authenticate with the documented scheme — an 'Authorization: Token <MEM0_API_KEY>' header (or MEM0_API_KEY via MemoryClient). Keep the key in a secret store, never in client-side code or logs, and rotate on exposure.

Pass / FailAi Platformcritical
02

Operator registers a webhook to mirror memory changes and expects events for memory_add, memory_update, and memory_delete.

Subscribe to the documented memory lifecycle events and handle each type explicitly to keep an external mirror in sync (create on add, modify on update, remove on delete). Do not assume only adds fire; updates and deletes must be applied too.

Pass / FailAi Platformhigh
03

An OSS operator switches the embedder model after memories were already stored with the previous embedder.

Recognize that existing vectors were produced by the old embedder; changing the embedder generally requires re-embedding existing memories (or the new model and old vectors will be incompatible/lower-quality). Plan a migration rather than swapping models silently.

Pass / FailAi Platformmedium

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Mem0
  • Ai Platform
  • Platform Org Project Webhooks Config

Recommended for

Mem0 (Platform + OSS)Mem0 customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.