Responses And Chat Completions

Agent streams /v1/chat/completions with max_tokens=256 for a long answer; the final SSE chunk has finish_reason='length'.

Detect finish_reason='length' on the terminal chunk and treat the answer as truncated — either raise a partial-completion to the caller or continue by appending the assistant text and re-requesting. Never present truncated output as complete.

Responses API multi-turn flow stores previous_response_id from turn 1 and references it on turn 2; the referenced response has since been deleted/expired.

Handle the missing-reference error by rebuilding the conversation from the integrator's own message store rather than retrying verbatim; do not assume server-side state is durable.

Agent sets n=4 on chat completions to get candidate diversity but reports cost as a single completion.

Account for n completions: output tokens are billed per candidate. Surface all n choices or pick one deterministically, and attribute cost as sum across choices.