Weather forecasting and data assimilation: why continuous ingestion follows

The analysis, not the chart

Weather forecasting used to mean drawing isobars on a chart and sliding the pattern forward by eye. That practice died gradually and then completely. What replaced it is a discipline called data assimilation, and it deserves to be understood before any question of machine learning enters the room.

A numerical weather model integrates the equations of atmospheric motion forward in time from a starting condition. That starting condition is called the analysis: the best estimate of the current state of the atmosphere, given everything known up to that moment. Observations arrive constantly and from wildly different sources — radiosondes released twice daily, aircraft reporting en route, ships and buoys, polar-orbiting and geostationary satellites sensing radiances rather than temperature directly. None of these observations agrees perfectly with the model's own short-range forecast, and none of them is error-free in the same way as any other. Data assimilation is the procedure that reconciles the two: it takes the model's prior forecast, weights it against incoming observations according to each observation's estimated error, and produces a new analysis. That analysis then seeds the next forecast. The forecast informs where to trust the observations; the observations correct where the forecast has drifted. Neither is ever treated as ground truth. The cycle has no stopping point. Every belief the system holds about the atmosphere is provisional, dated, and waiting to be revised by the next six-hour batch of incoming data.

This is a genuinely different kind of object from either a forecast or a snapshot. A forecast run alone, without further correction, drifts: small errors in the initial state grow, and the model's own approximations compound. A snapshot alone — a single dense observation of the atmosphere at one instant — tells you a great deal about now and nothing about six hours from now. Assimilation is the machinery that sits between the two, converting a stream of imperfect, heterogeneous, continuously arriving evidence into a single coherent, constantly updated state estimate. That state estimate, not any individual instrument reading, is what the forecast is actually built from.

Origin

Lewis Fry Richardson tried this by hand in 1922. Working alone with observational charts and slide-rule arithmetic, he computed a six-hour pressure forecast for a single point in Europe. It took him roughly six weeks and produced a tendency wildly larger than anything physically real, because nothing in his method filtered the fast-moving gravity waves that swamped the meteorologically meaningful signal. The calculation was correct in spirit and useless in practice, and Richardson knew it.

The fix came from filtering the equations themselves, not just doing the arithmetic faster. Jule Charney, Ragnar Fjørtoft and John von Neumann ran the first successful numerical forecast on the ENIAC in 1950, using a simplified barotropic model that suppressed the noise Richardson's method had let through. Operational numerical forecasting began in the United States in 1955, under the Joint Numerical Weather Prediction Unit. Lev Gandin formalised the statistical reconciliation of observations and forecast — optimal interpolation — in 1963, giving assimilation a rigorous footing rather than an ad hoc one. Adjoint-based variational methods followed in the late 1980s, and in November 1997 the European Centre for Medium-Range Weather Forecasts brought four-dimensional variational assimilation into operations, finally handling observations properly as distributed across time rather than snapped artificially to a single analysis hour.

The turn

Consider what kind of intake condition this actually is, independent of atmospheric physics. There is no assembled corpus that gets frozen and then reasoned over. There is no bounded scene, sensed richly while present and then gone. There is a permanent, heterogeneous, self-correcting stream: instruments of different types and different trust levels, arriving continuously, folded into a belief state that is always the current best answer and never the final one. Each observation carries its own provenance — which satellite, which channel, which station — and its own estimated error, and the system tracks whether that error is drifting so it can downweight or flag a source without a human noticing first.

That is precisely the intake condition that distinguishes a Large Universe Model from what came before it in this lineage. The Large Language Model reads a corpus assembled once; the intake event is finished before any inference happens, and the model's beliefs about the world are exactly as current as the day the corpus was closed. The Large World Model senses a bounded scene while the scene is present — dense, high-fidelity, real — but its beliefs expire with the scene, because nothing in the architecture is built to carry corrected belief forward across time. The Large Universe Model is defined by neither corpus nor scene but by an intake condition: every relevant stream still running, no stopping point, beliefs held revisably and tagged with where they came from.

Operational numerical weather prediction has satisfied that condition since 1955. Not approximately — literally. The cycle that began with the Joint Numerical Weather Prediction Unit has never stopped; it has only grown in volume, in instrument variety, in cycling frequency, and in the sophistication of its error modelling. Nothing about the category of thing being ingested has changed in seventy years. What changed is scale, trust and time — more sensors, better calibration, longer records — not a new kind of evidence to observe.

What else runs this way

Weather is not the only domain that reached this position, which is part of the argument's force. Electric grid operators run weighted least-squares state estimation over telemetry every two to four seconds, reconciling redundant meter readings against known network topology and flagging the worst-offending sensor by its residual. NASA's Deep Space Network has tracked spacecraft for decades with sequential filters that estimate trajectory, clock offsets and atmospheric delay simultaneously from two-way Doppler and ranging data, reprocessing old tracking arcs whenever the underlying models improve. None of these systems was built with a name like "Large Universe Model" in mind. They were built by engineers solving a specific estimation problem, and they arrived, independently, at the same intake architecture: heterogeneous streams, per-source error weighting, provenance, retrospective revision.

The misreading to disown

The weak, and wrong, reading of this history says continuous observation eventually predicts everything, given enough sensors and enough time. It does not. Forecast skill has improved by roughly one usable day per decade of effort — a real but slow gain — and Edward Lorenz's sensitivity result imposes a hard ceiling near two weeks on deterministic forecasting, regardless of how much is observed. More data does not dissolve that horizon. The correct reading is architectural, not prophetic. Assimilation shows what the terminal intake position looks like in practice — heterogeneous streams, per-source error models, provenance, revision of the historical record itself through reanalysis — and shows that seventy years of sustained effort inside that position produced refinement, not a successor category of evidence. Nobody working the problem found a fourth kind of thing to observe.

Objections that hold

The first and sharpest objection is that this all works because the atmosphere obeys known equations of motion. Assimilation is fundamentally the projection of observations onto a closed physical state vector; there is a dynamical model doing most of the work, and a background error covariance that can be estimated because the physics constrains it. A general model of the world has no equations of motion and no such covariance to lean on. Without a dynamical prior, continuous intake risks becoming an unbounded log rather than a running belief. This is a real limit, and it means the analogy fails if pushed into claims about predictive skill. The claim being made here is about intake alone: that continuous, heterogeneous, provenance-bearing observation folded into a revisable state is achievable, and that it exhausts the observational axis. What a system does with that intake once ingested — dynamical integration, or something considerably weaker — is a separate and much more variable question.

The second objection cuts deeper. The atmospheric state vector has a fixed schema: temperature, wind, pressure, humidity, on a defined grid. The ontology never changes. A genuine universe model needs to accommodate entities nobody anticipated when it was built — new institutions, new instrument classes, new categories of fact entirely. Continuous intake into a fixed schema is a considerably smaller achievement than continuous intake into an open one, and this is the point at which the meteorological case genuinely narrows. It is worth noting, though, that the observation side of assimilation was never fully closed: variational bias correction exists precisely so that instruments unforeseen at design time — a new satellite radiance channel, an aircraft sensor of unfamiliar quality — can be admitted and self-calibrated against the model background without a schema change. The extensible part is intake, not ontology. The open-schema problem is real, unsolved in general, and sits downstream of the axis under discussion here, not on top of it.

The third objection questions the whole framing. Assimilation systems observe; they do not act, and Lorenz's ceiling means additional observation eventually buys nothing. If intake saturates against a fixed dynamical limit, "everything, continuously" looks less like a terminal achievement and more like an expensive plateau. But saturation against a ceiling supports the claim rather than undermining it: if adding sensors stops improving the estimate, that is exactly what it looks like for an axis to have an end. You cannot escape the ceiling by observing a new kind of thing — only by modelling what you already observe more precisely, or by accepting the horizon. And the loop does close, in the relevant sense: the analysis determines the next background, and observation quality feeds back into instrument monitoring. It closes on belief rather than on the atmosphere itself, which is a narrower claim than control, but it is a closed loop.

What this establishes

Weather forecasting demonstrates that the third intake position on this axis is not speculative. It was built, made operational, and refined for seventy years, inside one domain, using instruments nobody in 1955 had imagined. It shows the shape of that position concretely: heterogeneous streams, per-source error weighting, provenance per observation, revision of the past as well as the present. It does not show that this position predicts everything it touches, and it does not show that the intake architecture transfers cleanly to domains without a closed physical state vector or a fixed schema. It shows only that the axis has an end, that meteorology reached it before satellites existed, and that nothing since has required a fourth kind of thing to observe.