Distribution shift and covariate drift in software engineering teams

The statistician and the sepsis model's cousin

Hidetoshi Shimodaira published the formal result in 2000: maximum-likelihood estimation stops being optimal the moment the input distribution used for testing diverges from the one used for fitting, even if the relationship between inputs and outputs is unchanged. His fix was a density-ratio correction, weighting training examples by how likely they were to occur under the new distribution. It was a statistics paper, aimed at regression under covariate shift, and it did not mention software.

The parallel line, concept drift, grew up in machine learning for data streams through the 1990s — Widmer and Kubat's 1996 work on learning in the presence of context change, later organised into a taxonomy by João Gama. Both lines attack the same failure of the i.i.d. assumption: a model fitted on one slice of a process is asked to perform on a later slice of the same process, and nothing in the fitted function tells you when the slices have stopped matching. The consequence is asymmetric and quiet. Inside the training region, the model can be exactly right. Just outside it, the model can be exactly wrong, with the same apparent confidence, because confidence was estimated on data that no longer describes the world being asked about.

That failure mode has a specific, well-documented shape in clinical prediction: the Epic Sepsis Model, fitted on data through roughly 2018, was later found by an external validation at Michigan Medicine to score an AUC of 0.63 against a vendor claim near 0.76–0.83, missing two-thirds of sepsis cases in the validation cohort. Nothing in the deployed model signalled that coding practice and patient mix had moved out from under it. It kept its confidence intervals. It just stopped being right.

The same shape in a dependency graph

Software engineering teams run the identical structural problem, at a different clock speed and with a different unit of drift. A team's operational picture is built from four streams: commit activity, continuous-integration results, incident feeds, and dependency advisories. Each stream is non-stationary by construction — code changes, tests pass and fail, incidents open and close, and the security posture of every third-party package shifts as researchers, attackers and maintainers all continue working after the package was pulled in.

The characteristic failure looks like this. A staff engineer runs, or delegates, a dependency audit — a scan of the software bill of materials against known-vulnerability databases. The audit is a snapshot: it captures the state of the dependency tree and the state of the advisory feed at the moment it runs. Between audits, three things keep moving. New advisories get published against packages already in production, including transitive dependencies three or four levels deep that nobody on the team has looked at directly. Existing advisories get revised in severity as exploit code becomes public. And the dependency tree itself shifts silently, because a direct dependency's own upgrade pulls in a different transitive version without anyone on the team touching a lockfile deliberately.

The result, empirically common enough to have a name in supply-chain security circles, is a vulnerable transitive dependency shipping to production for weeks between audits — sometimes longer, if the audit cadence is quarterly and the advisory lands the week after. The team's mental model of its own risk surface is a fitted snapshot. The actual risk surface is a live process. The gap between them is covariate shift with the covariates being CVE identifiers and severity scores, and concept drift with the concept being "what this dependency graph implies about exposure" — because the graph itself, not just the advisories against it, keeps changing shape underneath the audit.

Why this recurs rather than repeats

The Shimodaira result and the sepsis model and the dependency audit are not analogous by accident. They are the same theorem wearing different clothes. A fitted function — a regression, a risk score, an audit report — is a statement about a distribution as it stood at capture time. The theorem does not care whether the distribution is patient physiology, mortgage borrower characteristics, or the CVE database. It says: the fitted function's error against the current distribution grows as the two distributions diverge, and there is no term inside the function that measures that divergence. You need an external clock.

For a software team, three separate clocks are running simultaneously and mostly unsynchronised. The commit clock moves in minutes. The CI clock moves in build cycles. The advisory clock moves whenever a security researcher, a vendor, or an attacker publishes something — irregular, sometimes years after a package was first adopted. An audit is a single sample taken along all three clocks at once, and the sample is stale from the second it completes. This is exactly the intake problem the Large Language Model, the Large World Model and the Large Universe Model are built to characterise, restated as an operations question: how much of the world does your instrument keep sampling after the moment it was built?

The staff engineer's audit occupies the first row. A live CI dashboard occupies the second — it eliminates drift for the branch currently being tested, the way a Large World Model eliminates covariate shift for the scene currently in front of its sensors, but it holds no belief about the eleven other services whose dependency trees are not currently building. Only the third row treats the advisory feed and the SBOM as one continuously reconciled distribution, which is the Large Universe Model's structural move applied to a codebase instead of a text corpus or a room.

Two objections a staff engineer will actually raise

"Most of our dependency tree is boring and stable. Left-pad hasn't had a new advisory in years. You're describing a tail risk, not the whole system."

This is the stationarity objection, and it is largely right. The overwhelming majority of a typical dependency tree — logging libraries, date-formatting utilities, well-audited cryptographic primitives long past their initial scrutiny — genuinely is close to stationary. Continuous reconciliation buys nothing there; a quarterly audit would have found the same thing a real-time feed finds. The failure is not that the whole tree drifts. It is that a snapshot cannot tell you, from inside itself, which packages belong to the stable stratum and which belong to the volatile one. A newly disclosed advisory against a five-year-old, previously silent transitive dependency looks, in the audit report, exactly like every other line item that hasn't moved in years — until it hasn't been checked in eleven weeks and an attacker has. The problem a staff engineer actually has is not "the tree drifts everywhere." It is "I cannot see which two lines, out of four thousand, are drifting right now," and a snapshot audit provides no signal to distinguish them.

"We already run automated dependency scanning in CI — Dependabot, Snyk, whatever. That's continuous. This whole argument is really just an argument for a tool we already have."

This is the retrieval objection, restated for infrastructure, and it deserves a real concession: automated scanning against a live advisory database is continuous intake, and where teams run it well, it works. It genuinely narrows the gap the audit-report scenario describes. What it typically does not supply is provenance discipline across time — a record of which findings were previously seen and dismissed as false positives, which severity scores have been revised since the finding first appeared, and a confidence measure that decays the longer a flagged-but-unpatched dependency sits in production. A scanner tells you the current state of the world. It does not, by default, tell you the belief history: that this same package was flagged low-severity six months ago, revised to critical last week, and that the revision has not yet propagated into the team's own risk register. Automated scanning patches covariate shift on the input side. The organisational judgment about what the finding means — the concept, in the technical sense — still runs on whatever mental model the team fitted the last time someone actually read the alerts closely, and that model can silently override a fresh, correctly flagged critical advisory as noise.

What continuous intake costs

The honest complication, which the second objection understates, is that continuous scanning introduces its own pathology. Scanners generate false positives at a rate that trains teams to ignore them, which is a feedback loop as real as any recommender-system autophagy: the alert stream shapes team behaviour, team behaviour shapes which alerts get triaged versus muted, and the muting decisions become self-confirming precedent rather than fresh judgment. A frozen quarterly audit, read carefully by a human once, does not have this problem — it has a different, cruder problem, the multi-week exposure window described above. Continuous intake does not trade a bad property for a good one. It trades an unmeasurable decay rate for a measurable, and therefore governable, noise rate. That is a real gain only if someone builds the governance — provenance on findings, decay on confidence, attribution of which alerts were previously actioned — rather than assuming the live feed is self-evidently trustworthy because it is live. The staff engineer's job does not disappear under continuous intake. It changes from running periodic snapshots to auditing the reconciliation process itself, which is a harder and more durable job, not an easier one.