Chat Completions Openai Compatible

Operator hardcodes model='sonar' for every workload — quick factual lookups, deep multi-hop research, and structured-output extractions — to 'keep things simple.'

Route by workload: sonar for low-latency single-hop factual answers; sonar-pro for higher-quality multi-source synthesis with deeper grounding; sonar-reasoning for chain-of-thought heavy tasks where latency budget allows. Encode the choice per route, do not hardcode.

Client sets max_tokens=128 for a multi-source comparison question. Response is truncated mid-sentence; finish_reason='length'.

Detect finish_reason='length' and either (a) raise to caller as a partial answer, or (b) re-issue with larger max_tokens. Do not present the truncated answer as complete. max_tokens caps OUTPUT only — retrieved web context is server-side and not bounded by it.

Response carries usage={prompt_tokens, completion_tokens, total_tokens}; finance team rolls up per-tenant cost.

Read usage.prompt_tokens and usage.completion_tokens directly. Do not estimate from text length. For sonar-reasoning, completion_tokens includes reasoning tokens that count toward output billing — treat the full completion_tokens as billable output.