Observability, Monitoring and Drift in Production

This chapter explains how to keep an AI system trustworthy after go-live. The evaluation in Chapter 20 shows that a system is fit for purpose on the day it is released; this chapter addresses the days after, when the incoming data changes, the relationship between inputs and the right answer shifts, and the metrics accepted at release quietly decay. Its observability stack, drift detection, quality monitoring, and incident process are connected to the monitoring obligations that Regulation (EU) 2024/1689, the AI Act, places on providers under Article 72 and on deployers under Article 26 (1, European Union, 2024), and to the GitOps principle from Architecture as Code that observed state must be compared continuously with desired state (72, Nordqvist, 2025).

A deployed model is a frozen summary of the past, while the world it was trained on continues to move. Data drift, or covariate shift, occurs when the distribution of inputs changes while the relationship between inputs and outputs stays the same: a claims classifier receives a new form layout after a policy change. Concept drift occurs when the relationship itself changes: the criteria for a benefit are amended, so the same claim should now be routed differently. Performance decay is the observable consequence of either, and it is what the organisation notices, usually late.

Gama et al. (69, Gama et al., 2014) offered a taxonomy that remains useful for architects. Drift can be sudden, incremental, gradual, or recurring, and each pattern calls for a different detection window and response. They distinguished real drift, where the conditional distribution of the target changes, from virtual drift, where only the input distribution changes. Huyen (74, Huyen, 2022) brought this into production practice by separating the signals an operator can observe: raw inputs, derived features, predictions, and, where labels arrive, accuracy-related metrics. The crucial constraint is that labels often arrive late or never. A fraud model learns the truth weeks after a transaction; a benefits eligibility model may never learn it unless a decision is appealed.

Primary topics explored

1. Why models decay
2. The observability stack for AI
3. Tracing agent actions
4. Detecting drift
5. Quality monitoring with sampled human review
6. Service level objectives for probabilistic services