Guides/Model monitoring & drift
Post-launch model monitoring and drift
Shipping a model is the start of its lifecycle, not the end — the world keeps changing after launch, and a model that scored well on day one can degrade silently for months before anyone notices. Instrument drift signals, separate data drift from concept drift, and define who owns the retrain decision before you need it.
The monitoring loop
Treat this as a standing operational loop with a named owner, not a one-time launch task.
Observemetrics + logs
Detectdrift alert fires
Diagnosedata vs concept vs pipeline
Respondretrain, rules, or rollback
Redeployre-baseline metrics
Four kinds of drift
| Drift type | What changes | Typical signal |
|---|---|---|
| Data drift | Input feature distribution shifts (new user segment, seasonality) | Feature distributions diverge from the training baseline |
| Concept drift | The relationship between inputs and the correct output changes | Accuracy drops on fresh labeled samples even though inputs look normal |
| Feedback-loop drift | The model's own predictions shape the data it's later trained on | Metrics look great short-term, but diversity or coverage collapses over time |
| Upstream schema drift | An upstream system changes a field, format, or logging behavior | Silent nulls, type mismatches, or a feature suddenly going flat |