Platform Org Project Webhooks Config
Mem0 (Platform + OSS) · Mem0
Agent Memory — Mem0
Evaluates Mem0's Platform: Org/Project, Webhooks & Config across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Agent Memory eval coverage.
About Mem0
Mem0 is a memory layer for AI agents and assistants — it extracts, stores, and retrieves long-term facts across sessions via an add/search API, with user/agent/run scoping and optional graph memory, available as a managed Platform and open source.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Integrator calls the managed Mem0 Platform and must authenticate the MemoryClient with the MEM0_API_KEY. | Authenticate with the documented scheme — an 'Authorization: Token <MEM0_API_KEY>' header (or MEM0_API_KEY via MemoryClient). Keep the key in a secret store, never in client-side code or logs, and rotate on exposure. | Pass / FailAi Platformcritical |
| 02 | Operator registers a webhook to mirror memory changes and expects events for memory_add, memory_update, and memory_delete. | Subscribe to the documented memory lifecycle events and handle each type explicitly to keep an external mirror in sync (create on add, modify on update, remove on delete). Do not assume only adds fire; updates and deletes must be applied too. | Pass / FailAi Platformhigh |
| 03 | A vendor runs multiple customers as separate projects and instantiates MemoryClient(org_id=..., project_id=...) per customer. | Scope Platform calls with the correct org_id/project_id so each customer's memories live in their own project and never mix. Map your tenant to the right project on every call; do not share one project across customers. | Pass / FailAi Platformcritical |
How this eval is graded
Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
Rubric criteria
- Mem0
- Ai Platform
- Platform Org Project Webhooks Config
Recommended for
Works with
Related evals
Claude API
Evaluates Anthropic's Batch API across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Extended Thinking across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
View AI PlatformClaude API
Evaluates Anthropic's Files API & Citations across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Foundation Model & API eval coverage.
ViewFrequently asked questions
What does the Platform Org Project Webhooks Config eval for Mem0 Mem0 (Platform + OSS) test?+
Evaluates Mem0's Platform: Org/Project, Webhooks & Config across 9 scenario-based test cases, each graded against an expected-behavior rubric by an LLM judge, from Corsac's Agent Memory eval coverage.
How is the Platform Org Project Webhooks Config eval scored?+
The judge rubric: Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.
How many test cases does this eval pack include?+
The Platform Org Project Webhooks Config pack for Mem0 Mem0 (Platform + OSS) contains 9 test cases. 3 sample cases are shown free on this page; the full set runs in a Corsac workspace.
How do I run this eval?+
Sign up for Corsac, connect your model or agent endpoint, and run the Platform Org Project Webhooks Config pack as-is or after customizing thresholds. Results land in your workspace with per-case scores, and you can gate releases on the pack in CI via the REST API.
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.