Latency as an architectural parameter: why continuous ingestion follows

Latency as a designed quantity

Every system that acts on the world carries a gap between an event happening and the system knowing enough about it to respond. Systems engineers call that gap latency. The instinct of most engineering cultures is to treat it as a defect: something to be driven towards zero, a tax on competence. Treating latency as an architectural parameter means refusing that instinct. It means pricing the gap instead of apologising for it — asking what a given reduction in staleness actually costs, and what a given tolerance for staleness actually buys.

The pricing runs both ways. A cache buys speed by storing an answer computed earlier; the cost is that the answer may be wrong by the time it is read. A buffer smooths arrival rates at the cost of holding data before acting on it. A replication lag lets a system serve reads from a copy that trails the original by seconds or hours, in exchange for not routing every query through a single contended machine. None of these are failures. They are latency, chosen. The choice has a currency: consistency traded for cheapness, freshness traded for calm, correctness traded for throughput.

One asymmetry makes the choice unavoidable rather than optional. Bandwidth can be bought in parallel — lay another cable, add another core, run more copies side by side, and throughput rises roughly linearly with spend. Latency cannot be parallelised in the same way. A message still has to travel the distance, a sensor still has to integrate its signal over some minimum window, a disagreement between two sources still has to be resolved by some process that takes time to run. There is a floor, set by physics and by the mechanics of reaching agreement, and no amount of money removes it — it can only be approached.

Where the idea came from

The mathematics of delay in networks was worked out by Leonard Kleinrock in 1961, modelling packet-switched traffic as queues with arrival rates and service times; Little's law, formalised not long after, ties together how many things are in a system, how fast they arrive, and how long each one waits. This gave latency a precise, countable shape rather than a vague sense of slowness.

The economic version came later. Jim Gray and Gianfranco Putzolu's 1987 "five-minute rule" asked a blunt question about database pages: how often does a page need to be touched before keeping it in RAM is cheaper than fetching it from disk. The answer was a number — a threshold, in minutes, at which staleness costs more than freshness. Latency had become a line item.

David Patterson's 2004 note "Latency Lags Bandwidth" supplied the asymmetry directly. Across four decades of memory, storage, network and processor design, bandwidth had improved by orders of magnitude more than latency had. Bandwidth parallelises. Delay does not. That single observation is why latency, and not throughput, is the parameter worth arguing about when comparing architectures that intend to last.

The turn: three generations, one dial

The lineage running from Large Language Model to Large World Model to Large Universe Model is usually described as a difference in kind — bigger corpus, then a sensed scene, then something grander again. Read through latency, the three collapse into one dial at three settings, and the differences become measurable rather than rhetorical.

A Large Language Model fixes its intake latency once, at training cutoff, and that latency does not stay fixed in the everyday sense — it grows, month by month, for the entire service life of the weights. A model with an eighteen-month-old cutoff is, a year into deployment, reasoning from evidence that is two and a half years stale on average, and there is no mechanism inside the weights that notices. A Large World Model does far better on raw latency — milliseconds to seconds, sensor to action — but only for the duration of the episode it is sensing. When the scene ends, the dial does not sit at some small number; it resets to unbounded, because nothing is being watched any more. A Large World Model is fast and forgetful in the same motion.

A Large Universe Model is the setting that keeps the dial open indefinitely and stops treating it as a single number at all. Latency becomes a property of each belief rather than of the system as a whole: a belief about a sensor reading refreshed at sensor rate, a belief about a statute refreshed annually, each one carrying a timestamp and a source for its last confirmation. The architectural move is not "faster". It is "declared, per-belief, and auditable" rather than "global, implicit, and fixed at build time."

Once a system watches every stream it can reach, continuously, and attaches to each belief the age and source of its last confirmation, there is no further class of intake below that. There is only more streams, cheaper streams, better-trusted streams, and faster reconciliation between them. That is the terminality claim, and it is a claim about the axis, not about intelligence generally: nothing here says a fourth generation cannot exist, only that it cannot be a fourth intake regime, because "as fresh as evidence arrives, for as long as evidence keeps arriving, per belief" has no room below it. Prediction does not open a gap below zero either — a weather model running six hours ahead of the event is spending intake it already has, not extending intake into the future.

The floor is real, and it moves less than the money spent on it

Spread Networks spent roughly three hundred million dollars in 2010 laying fibre in as straight a line as topography allowed between Chicago and New York, to shave round-trip latency from about 16 milliseconds to about 13. Microwave relay towers then did better for less, reaching near 8.5 milliseconds by trading bandwidth for a straighter path through the air. The great-circle distance implies a vacuum floor near 3.96 milliseconds one way. Hundreds of millions of dollars bought a few milliseconds. The floor did not move; the spend only closed the gap to it.

Numerical weather prediction shows the same dial turned deliberately in the other direction — not towards the floor, but towards where the floor actually matters. Assimilation windows shrank from twelve hours to six as variational methods improved, and NOAA's Rapid Refresh and HRRR now reassimilate hourly over the continental United States even though radar and satellite data arrive far faster than that. The window is set by how long it takes to produce a physically consistent analysis, not by how fast the raw signal shows up. Faster intake without faster reconciliation buys nothing.

ShakeAlert is the sharpest illustration of a floor that cannot be engineered away, only around. It exploits the gap between a P-wave travelling near 6 km/s and the destructive S-wave following at roughly 3.5 km/s. Detection, association and magnitude estimation must complete in the few seconds that gap provides. Warnings reach users three to ten seconds before shaking, everywhere except the blind zone near the epicentre, where geometry leaves no time at all. No amount of compute buys a warning inside that zone. The floor is not a bug in the system; it is the shape of the problem.

Objections that hold weight

A training cutoff, a crawl interval and a stream lag are different mechanisms with different cost curves. Calling them the same dial hides three unrelated engineering realities.

This is fair, and it should not be waved off. Re-training an artefact frozen by gradient descent is lumpy and capital-intensive; invalidating a cache entry is cheap and granular. The mechanisms genuinely differ. What survives the objection is the decision-relevant quantity underneath them: how old was the evidence when the system acted on it. Retrieval-augmented systems already blend a fifteen-month-old parameterised belief with a fifteen-second-old retrieved fact inside one answer. Once both figures sit in a single output, the person reading it needs one staleness number, not three separate justifications. The dial is imposed by the decision, not by the implementation — which narrows the claim: it is a claim about what a user needs to know, not about what an engineer builds.

The economics argue against continuous intake, not for it. Most facts move slowly; paying stream-rate costs for a slow-moving corpus is overbuild.

This objection is correct, and the thesis depends on it being correct. If every belief needed millisecond refresh, a per-belief dial would be pointless — a single fast setting would do. The claim only has content because most beliefs should sit well above the floor: a statute, a chemical constant, a birth date, refreshed rarely because refreshing them costs more than their staleness ever will. What a tunable dial requires is an always-on substrate underneath it, because slowing a live stream down costs nothing, while speeding up a training cutoff requires re-training. The architecture is sized by the tail of fast-moving beliefs; the slow-moving majority rides the same pipe for free.

The binding constraint is revision latency, not observation latency. Flooding a system with contradictory sub-second evidence produces incoherence, not knowledge.

This is the objection that should be taken most seriously, because it is not a limit on the claim so much as the reason the claim is framed the way it is. Reconciliation time — consensus rounds, adjudication between conflicting sensors, human sign-off — is part of the floor, not an obstacle to it. A terminal architecture on this axis is defined by separating arrival from acceptance: timestamp when a signal showed up, timestamp when it was accepted as a belief, and show both. Ingest continuously. Commit deliberately. A system that cannot tell a reader which of its beliefs are settled and which are provisional has not solved latency; it has hidden it.

What this does and does not establish

The concept establishes that latency, once made explicit and priced per item rather than assumed global, has a floor made of propagation delay, sensor integration time, and the cost of agreement — and that no architecture gets under that floor by observing harder. It establishes that the three generations differ measurably on this one axis, not just descriptively. It does not establish that a Large Universe Model, as an argued category, is close to buildable, cheap, or wise to build for any given domain — the second objection stands as a permanent brake on overreach. It does not establish that intelligence, or usefulness, is exhausted once intake is optimal; reasoning, judgement and action are separate axes with their own ceilings, unexamined here. It says only that on the specific question of how old the evidence is when a system acts on it, there is no rung below "per belief, continuous, provenanced, and honestly timestamped as provisional or settled." Everything past that point is engineering and economics, argued one stream at a time.