Model Catalog Routing
Together AI · Together AI
Together AI evals — Model Catalog & Routing (relift v3)
About Together AI
Together AI is an enterprise AI inference cloud providing fast, scalable access to leading open-source models via an OpenAI-compatible API. Teams use Together for production inference, fine-tuning, and dedicated GPU deployments without the complexity of self-managed infrastructure.
Sample tests· showing 3 of 9
| # | Input | Expected behavior | Check |
|---|---|---|---|
| 01 | Integration uses a pinned slug from last quarter; Together returns HTTP 404 model_not_found while meta-llama/Llama-3.3-70B-Instruct-Turbo is listed on docs.together.ai serverless chat table. | Map 404 to a catalog refresh: select a current API model string from the serverless models page, update config, and log the replacement mapping—do not retry the deprecated slug indefinitely. | Pass / FailAi Platformhigh |
| 02 | Agent defaults to google/gemma-3n-E4B-it which lacks function calling in the serverless table while Qwen/Qwen3.5-9B supports tools and structured outputs. | Select Qwen/Qwen3.5-9B (or another catalog row with Function calling Yes) before sending tools; validate supported_parameters mentally against the serverless models table. | Pass / FailAi Platformcritical |
| 03 | User uploads warehouse label photo; agent must not send image_url parts to meta-llama/Meta-Llama-3-8B-Instruct-Lite (text-only context 8192). | Choose Qwen/Qwen3.5-397B-A17B or google/gemma-4-31B-it from the Vision models table and format user content as multimodal array with image_url. | Pass / FailAi Platformmedium |
Rubric criteria
- Together Ai
- Ai Platform
- Model Catalog Routing
Recommended for
Works with
Related evals
Run this eval in your workspace
Connect your data, configure thresholds, and review results with your team.