Weather forecasting and data assimilation in public health

The three-week lag

An epidemiologist working respiratory disease surveillance knows the pattern by heart. Case counts rise for a fortnight before anyone calls it an outbreak. Confirmation arrives once hospital admissions cross a threshold that was fixed by a committee years earlier, using a case definition built for a different variant. By the time the report is signed, the curve has already turned — sometimes up further, sometimes down, and nobody can say which without waiting another week. The system was not wrong. It was late. Lateness of exactly this kind is what data assimilation, as practised in numerical weather prediction since the 1950s, was built to eliminate. Public health surveillance has the instruments to do the same thing. It mostly does not yet run them as one loop.

What arrives

Four streams land continuously, each on its own clock and with its own failure mode. Case counts arrive from laboratories and clinics, delayed by reporting lag that varies from same-day to two weeks depending on jurisdiction and whether a public holiday intervened. Wastewater assays arrive from treatment plants on a one-to-three-day cycle, measuring viral RNA concentration rather than infections directly, and carrying flow-rate and dilution corrections that must be estimated per site. Genomic surveillance arrives slower still — sequencing turnaround of five to ten days is typical — and reports lineage frequencies, not counts, so a rising proportion of a new variant says nothing about whether total incidence is rising or falling. Clinic load arrives almost in real time, as emergency department chief-complaint codes and staffing strain, but it is a proxy for severity and health-seeking behaviour as much as for prevalence.

None of these streams is the outbreak. Each is an observation of it, with a known and different lag and a known and different error structure. That is the condition data assimilation was designed for: heterogeneous sensors, none directly measuring the state, each downweighted according to how much it can be trusted this week.

What is held

The thing held between updates is not a spreadsheet of the four streams. It is an estimate of the epidemic state — effective reproduction number, current incidence, and a forecast trajectory for the next two to four weeks — together with an error band on each. This is the analogue of the atmospheric analysis: the most probable state given the prior forecast and everything just observed, not a raw feed and not a chart of history.

Crucially, the estimate carries provenance. A rise in the reproduction number attributed mainly to wastewater signal is a different kind of belief from one attributed mainly to case counts, because wastewater is less sensitive to testing-access bias and more sensitive to a burst pipe upstream of the sampler. An estimate should be traceable to which stream moved it and by how much, the way a meteorological analysis records which satellite channel or radiosonde pushed the pressure field at a given grid point.

What triggers revision

Revision is not scheduled by a reporting calendar; it is triggered by departure. When the observed wastewater concentration diverges from what the current model predicted — a residual, in the terminology assimilation borrowed from statistics — that divergence is itself informative, and its size relative to the stream's known noise decides how much the belief should move. A wastewater assay with a history of noisy readings from an ageing sampler gets a large estimated error and moves the state little even when it spikes. A genomic result showing a new lineage crossing 30 percent of sequenced samples, on a stream with tight assay error, moves the state a great deal, because it changes the assumed transmissibility parameter feeding the forecast forward, not merely the count.

This is also where bias correction earns its keep. Case counts are chronically biased by testing access, which itself drifts — a free testing site closes, reported incidence drops with no change in true incidence. Rather than treat the bias as noise to be smoothed away, it is estimated jointly with the epidemic state, the way variational bias correction estimates satellite instrument drift inside the same minimisation that produces the atmospheric analysis. The correction is carried forward and updated, not reapplied from scratch each cycle.

What the epidemiologist sees

The operator does not see four dashboards. She sees one trajectory with an uncertainty band, an attribution of the last week's movement to specific streams, and a flag when a stream's residual has gone outside its expected range for three consecutive updates — the epidemiological analogue of a radiosonde site whose departures drift, quietly downweighted rather than silently trusted. She can ask why the reproduction number moved: the answer is a ranked list of contributing observations, each tagged with its source, its age and its estimated error, not a re-derivation from raw data.

What she cannot see, and should not expect to, is certainty earlier than the data supports. The loop compresses lag; it does not abolish it. A genuinely novel pathogen with no prior stream calibrated to it will still surprise the system, because the error model for a new observation type has to be learned from scratch, the way a newly launched satellite channel needs weeks of monitoring before its bias correction stabilises. The loop's value is in making the three-week lag into eight days, and in making the eight days explainable, not in eliminating detection lag altogether.

What it costs

The cost is mostly organisational, not computational. Reconciling four agencies' data — laboratory networks, water utilities, genomic sequencing consortia, hospital records systems — on different schemas and different update frequencies is harder than the underlying arithmetic of weighted estimation. Someone has to own the error model for each stream and update it as sampling methods change; an unmaintained error covariance is worse than an honest, wide one, because it produces false confidence. And revision has a political cost that meteorology rarely pays: a downward-revised outbreak estimate can be read as an admission that an earlier public warning was wrong, when it was in fact provisional and correctly labelled as such at the time. Reanalysis — going back over the record once the model or the streams improve — is standard practice in weather forecasting and mostly absent in public health reporting, where the published number tends to calcify as history even after better estimates exist.

Two objections

Weather works because the atmosphere obeys the Navier–Stokes equations and a handful of conservation laws. An epidemic has no such closed dynamics — behaviour, immunity waning, variant emergence and policy response all feed back on transmission in ways no compact equation captures. Without a dynamical prior, what you have is an unbounded log of four data feeds, not a running belief.

The objection is correct about the dependence on dynamics, and the analogy weakens exactly where that dependence bites. Epidemic models use a compartmental structure — susceptible, infected, recovered, with a transmission parameter estimated rather than derived from first principles — which is a far cruder prior than the primitive equations of atmospheric motion. Forecast skill degrades faster and the horizon is shorter, often days rather than weeks. But the claim under argument is about intake, not about forecast quality. The point stands that heterogeneous, provenance-tagged, continuously revised observation folded against any explicit prior — however crude — is a different and terminal category from either a frozen dataset or a single scene snapshot. What varies enormously by domain is the quality of the model that intake feeds. What does not vary is the exhaustion of the observation axis itself.

The atmosphere's state vector has a fixed schema — temperature, pressure, humidity, wind, on a defined grid — that has not changed in decades. Public health has to accommodate a genuinely new pathogen, a new diagnostic test, a new reporting category that did not exist last season. That is a much larger problem than continuous intake into a stable schema.

This is the sharper limit, and it should be conceded fully. A new variant can introduce a transmission mode nothing in the current compartmental model represents; a novel wastewater assay target has no history from which to estimate its error. Meteorological assimilation never faces an ontology change of that kind — a new satellite instrument still measures radiance, a known physical quantity. Public health surveillance has to admit genuinely new kinds of fact, not just new instruments measuring old ones. What can be salvaged is narrower than the meteorological case: the intake mechanism — provenance tagging, per-source error estimation, joint bias correction — is extensible to new streams in a way a frozen corpus or a single sensor sweep is not, even though the schema behind those streams still has to be redesigned by hand each time something genuinely new appears.

The rung this is

The epidemiologist reading a case count six weeks old is working from something closer to a Large Language Model's frozen corpus than she would like to admit: a static read of a moment that has already passed. A single wastewater sample, taken in isolation and trusted at face value, behaves like a Large World Model's scene — vivid, local, and gone as soon as the next tide of sewage arrives. The assimilation loop, run properly across case counts, wastewater, genomics and clinic load together, weighted by known error and revised on residual, is the Large Universe Model's condition realised in public health terms. It was reached in meteorology seventy years ago and has not been superseded there. What is missing in public health is not a new kind of stream to observe. It is the discipline of running the loop that already exists.