Monitoring and drift detection

Data drift — the distribution of the model’s input features changes over time relative to what it was trained on (a demographic shift in the user base, a new product category appearing, a seasonal pattern), while the true underlying relationship between inputs and outcome stays the same. Concept drift — the actual relationship between inputs and the true outcome changes (fraud patterns evolve to evade a fraud model’s learned signal, a user preference shift makes a recommendation model’s learned associations stale), even if the input distribution itself looks unchanged. They call for different detection and different fixes: data drift is detectable by comparing input feature distributions over time (no ground-truth labels needed, so it can be monitored in near-real-time) and fixed by retraining on more representative recent data or by input-side handling; concept drift is only detectable by comparing predictions against actual outcomes, which usually requires waiting for ground truth to arrive (sometimes with real lag) and is fixed by retraining with updated labels reflecting the new relationship, not just fresher inputs. A dashboard tracking only feature-distribution statistics will miss concept drift entirely, and one tracking only prediction-vs-outcome accuracy will react to concept drift only after the lag for ground truth to arrive — genuinely different monitoring, not one dashboard serving both.

Application error rate is available immediately and automatically — a request either errored or did not, known the instant it happens. Model accuracy requires the true label/outcome, which is frequently unavailable at prediction time and often arrives late or not at all: a fraud model’s prediction is checked against a chargeback that may not occur (or get reported) for weeks; a recommendation model’s “was this actually a good recommendation” outcome may never have a clean ground-truth label at all; a churn-prediction model’s true outcome is only known months later, when the customer has or has not actually churned. This is why production ML monitoring leans heavily on proxy signals available immediately (input distribution stability, prediction distribution stability, confidence/calibration metrics) as the fast-feedback layer, with true accuracy tracked separately as a slower, delayed-feedback layer once labels do arrive — treating the two as complementary rather than expecting one dashboard to give an instant accuracy readout the way an uptime monitor gives an instant error rate.

• Input feature distribution statistics (mean/variance/percentiles per feature, or a distance metric like population stability index or KL divergence between current and training-time distributions) — flags data drift directly and immediately, no labels needed.
• Prediction distribution (the model’s own output distribution — proportion of each predicted class, or the distribution of predicted scores/probabilities) — a sudden shift (a classifier that used to predict 5% positive now predicting 30% positive) is a strong signal something changed, even without yet knowing whether the new predictions are more or less accurate.
• Confidence/calibration — is the model’s stated confidence still well-calibrated (of the cases it says it is 90% confident about, are roughly 90% actually correct, checked against whatever ground truth eventually does arrive) — a sharp confidence shift without a corresponding known cause is worth investigating even before full ground-truth accuracy is available.
• Missing/null rate and schema changes on inputs — a pipeline change upstream (a source system altering a field, a new client version omitting a feature) is often the actual root cause behind what first appears to be “drift,” and this is usually the cheapest, fastest signal to check.
• Volume/latency of predictions themselves — a sudden change in request volume or in prediction latency, while not directly about accuracy, often correlates with an upstream system or traffic change worth understanding regardless.

None of these substitutes for real ground-truth accuracy once it is available — they are the early-warning layer that lets a team investigate before the slower, definitive signal confirms whether there is an actual problem.

• Real-time / low-lag layer — automated alerts on input-distribution shift beyond a defined threshold, prediction-distribution shift, missing/null-rate spikes, and basic operational health (latency, error rate, request volume) — tuned to page or ticket the on-call owner for genuinely actionable anomalies, not every minor statistical wobble (over-alerting on noise trains people to ignore the alerts, defeating the purpose).
• Delayed / ground-truth layer — scheduled jobs that join predictions against outcomes as labels arrive, computing real accuracy/precision/recall/business-cost metrics on a rolling window, compared against a baseline from the model’s validation period, alerting on sustained (not single-point-noisy) degradation.
• Segmented, not just aggregate, monitoring — a model’s overall accuracy can look stable while degrading badly on a specific, meaningful segment (a particular customer cohort, a particular input range, a new geography) that an aggregate number averages away; monitor key segments explicitly, not only the whole-population rollup.
• Dashboards for human review, distinct from automated alerts — trend lines over weeks/months that a human periodically reviews for slow, sub-threshold drift that no single alert would trigger but that accumulates into a real problem over a longer horizon than any individual alert’s window covers.
• A clear, pre-defined escalation and retraining trigger — what specific alert conditions actually initiate a retraining pipeline (automatically, or as a human-triggered decision), so detecting drift has a defined next action rather than just generating a notification nobody has an established playbook for.

Aggregating across a whole population can mathematically cancel out opposing segment-level changes — a model’s accuracy improving on a large, easy majority segment while degrading on a smaller but important minority segment can leave the overall aggregate number essentially flat, hiding a real and possibly serious regression from anyone only watching the top-line metric. Concrete example: a lending or fraud model whose overall accuracy stays stable while its false-positive rate rises specifically for a newly-onboarded customer segment (a new geographic market, a new product line) that is currently a small fraction of total volume but growing — the aggregate metric will not flag this until that segment grows large enough to move the overall number, by which point real business harm (or a fairness/compliance problem) may have been accumulating unnoticed for a long time. Deliberate segmented monitoring — tracking key metrics broken out by the dimensions most likely to hide meaningful heterogeneity (customer cohort, input range, geography, model-version rollout percentage during a canary) — is the direct mitigation, and is worth the added dashboard complexity specifically because the aggregate view is known to be unreliable for exactly this failure mode.

Retraining is not free (compute cost, validation effort, deployment risk per the deployment subtopic), so triggering it on every detected fluctuation is its own failure mode. A reasoned decision considers: magnitude and persistence — a brief, small distribution shift that reverts within days is more likely transient (a short-lived traffic anomaly, a temporary upstream data issue) than a sustained, growing shift that persists over weeks; whether it correlates with an actual metric regression, once ground truth is available, rather than treating input drift alone as automatically requiring action — some drift is genuinely benign if the model remains accurate despite it; root cause — drift caused by an upstream pipeline bug (a broken feature computation) should be fixed at the source, not “solved” by retraining a model on now-permanently-corrupted data, which would bake the bug into the model itself; and cost-benefit — for a low-stakes model, tolerating mild, plausibly-transient drift and re-checking later is often more sensible than an automatic retrain-on-every-signal policy that burns compute and introduces deployment risk (per the deployment subtopic) for marginal or unconfirmed benefit. The mature setup treats detected drift as a trigger to investigate, with retraining as one possible outcome of that investigation rather than an automatic reflex response to the alert firing.