Distribution shift

A condition in which the statistical properties of inputs to a deployed AI system differ meaningfully from those of the training data, often leading to degraded performance. Distribution shift is one of the principal anomalies that the Article 14(4)(a) oversight mechanism must be designed to detect, through performance dashboard alerts or input monitoring. Sustained undetected distribution shift is a common pathway from launch-time compliance to operational non-compliance in AI systems.

The change over time in the statistical patterns a model's inputs follow relative to what it was trained on, caused by new vendors, new sensors, new customer segments, seasonal patterns, or any other operational change; the reason data-centric diagnostics are a standing practice rather than a one-time project, since a slice that was well covered last year can become a coverage gap this year.

A gradual change in the operating conditions a fleet actually encounters (a change in task mix, terrain, or seasonal conditions) that can make performance look different over time for reasons unrelated to anything the engineering team changed, distinct from an ordinary confound because it reflects the world changing rather than the test being poorly controlled.

The technical fact that an AI model's performance can change when the population or conditions it encounters shift away from the data it was trained and tested on; the underlying reason healthcare's per-product FDA review and defense's dual-use verification layer both exist.

See data drift. The movement of input data away from the training distribution; a leading indicator that accuracy is about to fall.