Guardrails And Realtime Scorers

Operator adds a PII guardrail only on the model's output and lets raw user input (containing SSNs, emails) flow straight into prompts and logs.

Apply PII detection/redaction on both the inbound user input and the outbound model output: input-side to avoid logging or sending raw PII to downstream models, output-side to catch leakage and memorized PII. Redact before persistence, and decide block vs mask per field type and risk.

Operator uses the toxicity scorer's raw 0-1 score with a hardcoded 0.5 cutoff copied from an example, with no tuning for their content domain.

Tune the block threshold on a labeled sample of your own traffic, balancing false blocks against missed harmful content for your domain and audience. Document the chosen cutoff and re-evaluate it when the audience or content type changes. Treat the example cutoff as a starting point, not a default …

Operator chains four real-time scorers serially in front of every response, adding their latencies together on the user's critical path.

Budget guardrail latency: run independent scorers concurrently rather than serially, set per-scorer timeouts, and reserve the heaviest checks for the highest-risk routes. Measure the combined p95 against the latency SLO and drop or async low-value scorers that blow the budget. [REQUIRES-VERIFICATIO…