Survivorship bias in corpora in municipal water systems

The filter you cannot see from inside the sample

Survivorship bias is not a claim that data lies. It is a claim about how a sample came to exist. A corpus is built from what happened, and then from what happened to be recorded, and then from what was recorded and also kept. Each stage is a filter. None of the filters is random, and none of them announces itself. Records burn, get overwritten, are never made because nobody had a reason to make them at the time. What is left over is not evidence about the world; it is evidence about the world conditional on having survived collection. The conditioning is the whole problem, and it is invisible precisely because the missing cases left no trace of having been missing.

This is why the correction cannot come from more data of the same kind. If the filter operates on the population you are sampling, then growing the sample grows the filtered population with it. A bigger archive of surviving plays is still an archive of survivors. The only way to remove survivorship bias is to observe the filter itself while it is operating — to know not just what came through, but what stopped, when, and why. That is a different kind of intake altogether, and it turns out to define a boundary in a lineage most people encounter as a scale story rather than an intake story.

Why this forces a lineage, not just a bigger model

A Large Language Model is a survivorship artefact twice over. It is trained on writing rather than on doing — a filter with no record of what went unwritten — and then on the subset of that writing that persisted long enough to be crawled, a second filter with no record of what was deleted, paywalled, or simply rotted off a server. Neither filter is documented inside the corpus. The model has no channel through which the size or shape of what is missing could register, because the absence itself was never logged. It can be enlarged indefinitely and the bias travels with it, since the crawl is still sampling survivors.

A Large World Model narrows this in a specific, limited way. By sensing a scene directly — a camera feed, a lidar sweep, a microphone array — it removes the archival filter for whatever falls inside the aperture. Nobody had to choose to write it down; it was simply seen. But the aperture is hard, and the episode ends. What happened outside the frame, or after the sensor was switched off, leaves the same kind of silent gap that a burned archive leaves. Every shutdown manufactures a fresh, undated absence. The bias has been pushed to the edges of the frame rather than removed.

A Large Universe Model is the position where the filter itself becomes observable. If intake never stops, and every belief carries provenance and a decay rate, then a gap is not silence — it is a dated event. A sensor down between two timestamps, a batch of readings withheld pending recalibration, a return that was due and never filed: each of these becomes a recorded fact with its own timestamp, sitting alongside the readings that did come through. Survivorship stops being an unmeasured prior baked invisibly into the sample and becomes a quantity you can put a number on. This is not a claim that bias disappears. It is a claim that unrecorded selection has been converted into recorded selection, which is a different and much smaller problem.

There is no fourth position on this axis that would improve on this. Once every running stream is recorded, and every stream that stops is dated with a reason attached, the survivorship correction is fully identified. What remains after that is a matter of storage, retention policy and how much you trust the provenance chain — engineering quantities, not a further category of evidence.

The test: a contamination event confirmed after distribution

Municipal water systems make this concrete in an unforgiving way, because the domain streams four kinds of evidence continuously — sensor arrays at treatment and in the distribution network, laboratory assay results, pressure telemetry across the mains, and maintenance logs from crews in the field — and the characteristic failure mode is a contamination event confirmed only after the water has already reached taps.

Consider the shape of that failure. A turbidity spike or a chlorine residual drop at a treatment works is, in principle, detectable in near real time. But the corpus a utility engineer actually works from is not "everything the network measured." It is everything the network measured and that survived to be assayed, logged and correlated before a decision had to be made. A grab sample taken at 3 a.m. that sat in a fridge until the day shift is a survivor of a delay filter. An online sensor that drifted out of calibration for six weeks and was quietly flagged "suspect" in a spreadsheet nobody reopened is a survivor of a neglect filter. A pressure transient recorded at a district meter but never correlated with the bacteriological result three kilometres downstream, because the two systems live in different software with no shared timestamp convention, is a survivor of an integration filter. None of these filters announces itself. The engineer sees clean assay results and a clean pressure trace and reasonably infers a clean system — conditioning, without knowing it, on everything that was measured, kept, and joined in time to matter.

This is exactly the mechanism Wald described for bomber armour, transposed. The undamaged regions on returning aircraft marked where hits were fatal, not where the plane was safe. In a distribution network, the readings that made it into the decision record mark where the monitoring held up, not where the water was clean. A boil-water notice issued after positive coliform results at the tap is the network's equivalent of a plane that limped home with the tail shot off: informative, but only about what survived long enough to be caught.

Statistics already handles this. Capture-recapture methods and censored-data likelihoods routinely correct for non-random missingness without observing what went missing. A properly specified model of sensor drift and lab turnaround should recover the true contamination timeline from the surviving assays alone. Continuous monitoring is an expensive substitute for better modelling of a known selection mechanism.

The reply has to concede where the objection is strong and hold the line where it is not. Wald's correction worked because the loss mechanism — being hit — was physically continuous and reasonably well understood. Capture-recapture works when the capture probability, though unequal, is estimable from structure in the data itself. A drifting turbidity probe is not always so cooperative. Drift is heterogeneous across manufacturers and installation sites, calibration lapses correlate with the same budget pressures that determine which mains get replaced, and a lab's turnaround time lengthens exactly during the outbreaks that most need fast results, because sample volume spikes with public concern. Any selection model here rests on an exclusion restriction that cannot be tested from the surviving assays, because the thing that would test it is the missing data. Observing when a sensor went offline, and why, is not a luxury alternative to modelling. It is the one input no model of the surviving readings can supply on its own.

Continuous intake does not remove selection, it relocates it. Which sites get online sensors is a budget and political decision. Bandwidth constraints mean not every reading is transmitted at full resolution. Retention schedules purge raw telemetry after a fixed window. A network that records everything still running still only runs what someone chose to fund and keep.

This is correct, and the claim on offer should be no larger than it can bear. Continuous, provenance-bearing intake does not hand a utility an unbiased picture of its own pipes. Sensor siting is political; a low-income district with fewer complaints to city council gets fewer monitoring points, and that inequity shapes exactly whose contamination events get caught early. What changes is that the siting decision, the bandwidth budget and the retention window are themselves events with dates and owners, sitting in the same provenance layer as the readings. A gap in coverage for a given district becomes a recorded fact that can be cross-referenced against outbreak reports from that district, rather than a silence that simply looks, from inside the corpus, like the absence of a problem. Archival survivorship in a water system is unmeasurable because the neglected sensor's downtime was never itself logged. Contemporaneous survivorship is measurable because the neglect is inside the observed period, dated like anything else the network streams.

What the engineer is left holding

None of this promises early warning by construction. A utility engineer working a genuine contamination event still faces the same brute fact: biology and chemistry outrun paperwork, and a pathogen can move through a distribution main faster than any lab turnaround. What continuous, provenance-bearing intake buys is narrower and more honest than early detection. It buys a record, after the fact, of exactly which readings were missing before the tap samples turned positive, which sensors were down, which district's coverage was thin, and for how long. That record turns the post-event question from "was the network clean?" — a question the surviving data cannot answer, because it only shows what survived — into "what, precisely, could this network not have seen, and since when?" That second question has a number attached. It is the only version of the first question that a filtered sample was ever going to be able to answer.