Eval Library
L
For LangSmithAI Platform

Auth Workspaces Rbac Governance

LangSmith · LangSmith

LLM Observability & Evaluation Platform — LangSmith (LangChain)

LangSmith evals — Auth, Workspaces, RBAC & Governance (relift v3 InfraRed)

About LangSmith

LangSmith is LangChain's LLM observability and evaluation platform: tracing, datasets, evaluators (LLM-as-judge, code, and human), experiments, prompt management, and online monitoring used by AI teams to measure and improve LLM apps in production.

Employees

~200

Industry

LLM Observability

Headquarters

San Francisco, CA

Sample tests· showing 3 of 10

#InputExpected behaviorCheck
01

Operator generates a LANGSMITH_API_KEY from Settings → API Keys for a CI pipeline. CI must only see one workspace's data.

Create the key inside the intended workspace; the key is scoped to that workspace by construction and cannot read another workspace's projects or datasets. Store as a CI secret. Rotate on a schedule and on personnel change. Audit via Settings → API Keys → last-used.

Pass / FailAi Platformcritical
02

Engineering can write traces, finance can only read cost dashboards, and external auditors get read-only access to one project.

Use workspace roles: developer (read/write) for engineering, viewer (read) for finance, and an auditor role scoped to a single project for external auditors. Confirm RBAC scope via Settings → Members. Avoid granting Admin broadly.

Pass / FailAi Platformhigh
03

Customer A and Customer B are different workspaces in the operator's org. A user with access only to A's workspace must not see B's projects.

Use distinct workspaces per customer (or use ABAC roles per project if the Enterprise plan supports it). Verify isolation by attempting to list B's projects with an A-scoped key — expect 403 / empty list. Document the isolation model in the SOC 2 narrative.

Pass / FailAi Platformcritical

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Langsmith
  • Ai Platform
  • Auth Workspaces Rbac Governance

Recommended for

LangSmithLangSmith customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.