Caching Simple And Semantic

Config cache={mode:'simple', max_age:3600}. Caller sends two requests with identical messages[] and model; one has temperature=0, the other temperature=0.1.

Simple cache keys on the full request payload — different temperature values are different keys and miss. Verify with x-portkey-cache-status: MISS on both requests. To collapse them, normalize the request body client-side before sending.

Config cache={mode:'semantic', similarity_threshold:0.95, max_age:3600}. Caller sends 'how do I reset my password?' then 'how can I reset my password?'

At threshold 0.95 these near-duplicate queries should hit; verify via x-portkey-cache-status: SEMANTIC_HIT. Tune similarity_threshold against a calibration set — lower thresholds increase recall but raise the chance of returning semantically wrong cached responses.

Operator wants to attribute cost savings to caching and reads response body to infer cache hits.

Inspect x-portkey-cache-status response header (values include HIT, MISS, SEMANTIC_HIT, REFRESHED per docs) — do not infer hits from latency or from the absence of provider attribution. Aggregate by header value into cost-savings dashboards.