Definition
Drift detection
Drift detection is monitoring for slow, cumulative divergence between two things that should stay aligned — a model's behavior versus its training distribution, a deployed schema versus the code that defines it, a process versus its intended baseline. Unlike anomaly detection's sudden spikes, drift is gradual, so it's caught by comparing state against a reference rather than by watching for a jump.
Key takeaways
- Drift detection monitors for slow, cumulative divergence between two things that should stay aligned.
- Unlike anomaly detection's sudden spikes, drift is gradual — caught by comparing state to a reference, not watching for a jump.
- It is a reconciliation: enumerate what should be true and check reality against it, flagging gaps before they break.
- Planoda's schema-drift check compares the code's data model to the live database and fails the deploy gate on a mismatch.
Some failures arrive as a spike; others creep. A machine-learning model degrades as the world it predicts shifts away from its training data (data drift, concept drift). A database falls behind the schema its application expects, one forgotten migration at a time. A team's process erodes from the standard it agreed to. None of these trip a threshold on any single day — they accumulate — so detecting them means periodically comparing the live state to a declared reference and reporting the delta.
Drift detection is fundamentally a reconciliation, not an alarm. It enumerates what should be true (the schema's columns, the model's expected input distribution, the SLA's baseline) and checks reality against it, flagging gaps before they cause a failure. The payoff is catching a problem in its slow phase, when it's cheap to fix, instead of in the acute phase, when it has already broken something.
Planoda guards against schema drift directly: a check enumerates every column the code's data model defines and compares it against the live database, failing the deploy gate with exact reconciliation hints when the database is missing something the code expects — so a slow divergence can't ship a 500 to production.
Related terms
- Anomaly detectionAnomaly detection is automatically flagging data points that deviate meaningfully from an expected pattern — a sudden spike in bug inflow, a cycle whose throughput collapses, an SLA breaching its trend. Instead of waiting for someone to notice on a dashboard, the system continuously compares live signals against a baseline and surfaces the outliers worth a human's attention.
- Model EvaluationModel evaluation is the practice of measuring how well an AI model or system performs against defined criteria, using test cases, scoring methods, and metrics. It replaces 'it seems to work' with evidence — catching regressions, comparing models, and verifying quality before and after deployment. For generative systems, evaluation is harder because outputs rarely have one correct answer.
- ObservabilityObservability is the degree to which you can understand a system's internal state from the data it emits — logs, metrics, and traces. A system is observable when you can answer new questions about its behavior without shipping new code, letting you debug unknown failures in production rather than only the ones you anticipated.
- Single Source of TruthA single source of truth is a design principle where every piece of data has exactly one authoritative, canonical home, and all other views derive from it. Instead of the same fact living in several places that can drift apart, one record is definitive. It eliminates contradictory copies, so there is never ambiguity about which value is correct.
- Fine-TuningFine-tuning is the process of further training a pretrained language model on a smaller, task-specific dataset so it adapts to a particular domain, format, or style. Rather than training from scratch, it adjusts the existing model's weights on curated examples, producing a specialized variant that performs a narrow task more reliably than the base model.