Fireworks Function Calling Tool Orchestration
Fireworks AI · Fireworks AI
Fireworks AI evals — Function Calling & Tool Orchestration (relift v3)
About Fireworks AI
Fireworks AI is a high-performance inference platform for open-source and fine-tuned models, delivering industry-leading throughput and latency for production workloads. Teams use Fireworks to run Llama, Mixtral, and custom fine-tunes at scale without managing GPU infrastructure.
Sample tests· showing 3 of 12
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Autonomous agent might loop tools indefinitely; budget guard needed. | Implement loop with max five assistant-tool cycles; abort with user-visible message when cap hit; log usage fields each round. | Pass / FailTool usehigh |
| 02 | Power user says always call record_metric; default tool_choice auto sometimes answers without tools. | Set tool_choice to required with named function when user mandates tool execution; otherwise keep auto for mixed Q&A. | Pass / FailTool usemedium |
| 03 | Misconfigured deployment omits tools; user asks to file JIRA ticket via function. | Explain tools unavailable; refuse to hallucinate tool_call; offer manual steps or config fix—do not fabricate tool execution. | Pass / FailPolicycriticalneg. control |
Rubric criteria
- Fireworks
- Ai Platform
- Function Calling Tool Orchestration
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.