Streaming Stt Realtime

Browser captures audio at 48 kHz via getUserMedia but the WebSocket is opened to wss://api.assemblyai.com/v2/realtime/ws?sample_rate=16000 and frames stream without resampling.

sample_rate query param must match the actual stream. Either resample client-side to 16 kHz (Universal-streaming's preferred rate) before sending, OR pass sample_rate=48000 and accept higher bandwidth. A mismatch degrades accuracy silently — there is no server-side rejection.

Voice-agent application sets end_utterance_silence_threshold=2000 ms. End-users complain the agent feels slow because turn-end is detected too late.

end_utterance_silence_threshold trades latency against premature endpointing. Lower (e.g., 500-800 ms) for snappier voice agents; raise for dictation where long pauses are expected. Tune per workload and verify FinalTranscript boundaries land where expected. Document the tradeoff.

Browser code opens wss://api.assemblyai.com/v2/realtime/ws?token=<account_key> using the long-lived AssemblyAI API key directly so 'it just works in dev.'

Mint a short-TTL streaming token server-side via POST /v2/realtime/token and pass it as the token query param. The account key must never appear in client JS. Rotate the token mint on key revocation. Treat any client-visible account key as a compromised secret.