Eval Library
F
For FirecrawlAI Platform

Auth Rate Limits Credits Webhooks

Firecrawl · Firecrawl

Web Data for AI — Firecrawl

Firecrawl evals — Auth, Rate Limits, Credits & Webhooks (relift v3 InfraRed)

About Firecrawl

Firecrawl is a web-data API for AI — it turns websites into clean, LLM-ready markdown or structured data via scrape, crawl, map, search, and LLM-powered extract endpoints, with JS rendering, browser actions, and proxies. Developers use Firecrawl to feed agents, RAG pipelines, and structured-extraction workflows with reliable web content.

Employees

~30

Industry

Web Data / Scraping

Headquarters

San Francisco, CA

Sample tests· showing 3 of 9

#InputExpected behaviorCheck
01

Agent puts the Firecrawl API key in a query string or a custom X-API-Key header and gets 401 Unauthorized.

Send the key as Authorization: Bearer <FIRECRAWL_API_KEY>. Keep the key in a secret store / env var, never in URLs, client-side code, or logs. Rotate on suspected exposure.

Pass / FailAi Platformcritical
02

Under burst load the agent receives HTTP 429 from /v1/scrape and immediately retries in a tight loop, making the limit worse.

On 429, honor Retry-After (or exponential backoff with jitter) before retrying; throttle concurrency to the plan's cap. Do not hammer the API on 429 — that extends the throttle window.

Pass / FailAi Platformhigh
03

Agent fires 200 simultaneous scrape requests to maximize throughput and is throttled because it exceeds the plan's concurrent-request cap.

Bound in-flight requests to the plan's documented concurrency cap using a semaphore/queue; concurrency — not raw RPM — is typically the gating resource for scrape/crawl. [REQUIRES-VERIFICATION] for the exact per-tier concurrency numerics.

Pass / FailAi Platformhigh

How this eval is graded

Grade against expected.ideal_behavior and expected.rubric. Per-criterion pass requires mean >= 4.0 and no criterion below 3.

Rubric criteria

  • Firecrawl
  • Ai Platform
  • Auth Rate Limits Credits Webhooks

Recommended for

FirecrawlFirecrawl customers

Works with

Related evals

Run this eval in your workspace

Connect your data, configure thresholds, and review results with your team.