Model Performance Monitoring Drift Detection

A nightly monitoring agent runs the rolling AUPRC computation on the most recent 7-day window for a low-census hospital site. Due to a 5-day stretch of early discharges and two ICU patients whose sepsis codes are pending finaliza…

The agent detects that the positive count in the finalized-label buffer is zero. It returns a defined sentinel—NaN, null, or an explicit structured status of 'insufficient_positives'—and emits a structured warning log entry: 'Rolling AUPRC undefined: 0 positive labels in 7-day window across 200 neg…

The hourly AUPRC refresh agent pulls the 30-day rolling buffer, which contains 800 (score, label, timestamp) tuples. Of these, 120 patients were admitted within the last 48 hours; their 72-hour sepsis-onset detection window has n…

The agent reads `outcome_lag_hours = 72` and the current timestamp, computes a cutoff at current_time minus 72 hours, and excludes all 120 patients whose prediction_timestamp is more recent than the cutoff. AUPRC is computed on the remaining 680 patients with closed outcome windows. If the resultin…

The automated reporting agent is generating the quarterly model-performance report. The monitoring system has computed four values over the same 30-day period: aggregate AUPRC = 0.54 (all monitored patients), ICU subgroup AUPRC =…

The agent populates each row with its independently computed value: overall = 0.54, ICU = 0.61, non-ICU floor = 0.48, age >= 65 = 0.42. Each value is labeled with its stratum and the patient count used to compute it. The report includes a footnote clarifying that the aggregate figure does not repre…