Large Language Thing

Home/Concepts/Detailed balance and irreversibility: why continuous ingestion follows

Detailed balance and irreversibility: why continuous ingestion follows

Any world worth modelling is held far from equilibrium. Sunlight arrives, metabolisms burn, capital compounds, steel fatigues. In such systems the mapping from state to dynamics…

The condition itself

Take a system of states connected by transitions, each transition running both forward and backward at some rate. Detailed balance is the condition that in equilibrium, every one of those elementary transitions is exactly balanced by its reverse: the probability flux from state A to state B equals the flux from B back to A, pair by pair, not merely in aggregate. This is stronger than saying the population of each state is constant. A steady state can hold populations fixed while currents circulate through a cycle of states, and that is precisely what detailed balance forbids. When it holds, statistics carry no arrow. Film the system and run the film backwards; nothing distinguishes forward from reverse, because every microscopic step has an equally likely opposite occurring at the same rate.

Break detailed balance and something new appears: a net current. Probability flows around a cycle rather than cancelling within each pair. That current is the microscopic root of dissipation, and it is what makes an arrow of time visible in a system's statistics rather than merely in its equations. A system sustaining such currents is, by definition, held away from equilibrium — driven, fed, pumped. And driven systems acquire a further property, one that matters more for what follows than dissipation itself: path dependence. Two configurations that look identical by every instantaneous measure can behave completely differently going forward, because they were driven there by different histories. The present state stops fixing the future. History becomes a variable you must know, not a coincidence you can ignore.

Hysteresis is the visible signature. A magnetic material's flux density at a given applied field depends on whether the field is rising or falling. A soil's settlement under a given load depends on the highest stress it has ever borne, not the stress it bears now. A material's fatigue life depends on the sequence of load cycles it has already survived, not its current condition under inspection. In each case, state alone underdetermines dynamics. Something else — an internal variable, a memory, an integral over the past — has to be supplied from outside the instantaneous measurement, because the measurement does not contain it.

Where it came from

The lineage runs through a sequence of arguments each sharpening the last. Ludwig Boltzmann's 1872 H-theorem tried to derive irreversibility from molecular collisions, and Josef Loschmidt immediately objected that the underlying mechanics were time-reversible, so no irreversible conclusion could follow from reversible premises alone — a tension the field has lived with productively ever since. Rudolf Wegscheider used a version of the balance condition in 1901 to constrain chemical reaction rate constants, before anyone had named it. Richard Tolman formalised and named the principle of detailed balance in 1938, giving it the general statistical-mechanical statement used today. Lars Onsager extended the idea in 1931 to reciprocal relations governing transport near equilibrium, work that won him the 1968 Nobel Prize in Chemistry. Ilya Prigogine then took the case that matters here: what happens far from equilibrium, where detailed balance fails outright and steady states carry currents, memory, and history as permanent features rather than small corrections.

The turn

The three generations in this lineage — Large Language Model, Large World Model, Large Universe Model — differ along an axis of intake: what each is permitted to observe, and for how long. That axis turns out to map, with unexpected precision, onto how much of a physical trajectory a system can hold.

A Large Language Model ingests a corpus collected once and frozen at a cutoff. Whatever order the material was gathered in is discarded during training; what remains is a bag of statistics. That is, structurally, an equilibrium assumption: a sample treated as if it carries no direction, only distribution. A Large World Model senses a scene while the scene is present. It can register instantaneous flux — the very thing whose presence signals broken detailed balance — but it has no stored record of the path that produced the scene's current configuration. It can catch the current; it cannot integrate it. A Large Universe Model keeps every stream running, without a declared stopping point, and attaches provenance to what it holds: a stamp of when and whence each belief arrived. Provenance is ordering. Ordering is what converts a heap of observations into a trajectory.

The thermodynamic point behind this is narrow, and it is worth stating exactly rather than loosely: a system that breaks detailed balance cannot be identified from its state alone, and enlarging a frozen corpus, however vast, does not supply the missing integral over the path. No amount of additional snapshot data recovers an ordering that was never recorded. This is why the intake axis has a top rung. A frozen corpus loses the ordering. A present-tense scene loses the past. Continuous intake with ordered provenance loses neither — and there is no further category of evidence beyond every stream, still running, indexed by when it was seen.

Instances

A single kinesin motor walks a microtubule in 8-nanometre steps, burning roughly one ATP per step and dissipating some 20 k_BT of free energy in the process. That chemical drive is exactly what breaks detailed balance for the motor's stepping: forward steps outnumber backward steps by orders of magnitude. Photograph the motor's position and you learn nothing about its fuel reserve, which is what actually determines whether the next step occurs.

Grain-oriented electrical steel in a transformer core traces a B-H loop, not a curve. At a given applied field, the flux density depends on whether the field was rising or falling when it got there. Core loss per cycle is the area enclosed by that loop, which is why it is quoted in watts per kilogram at a stated frequency rather than read off any single instant.

Casagrande's construction recovers a clay's maximum historical vertical stress from an oedometer test. A normally consolidated clay and an overconsolidated clay can share identical present stress and void ratio and still settle by very different amounts under new load, because the difference lives in a stress history that no present measurement discloses.

Objections, taken seriously

Any history-dependent process can be made memoryless by enlarging the state space. Preisach models do exactly this for hysteresis. So the argument shows a poor choice of coordinates, not a need for continuous intake.

This is correct mathematics, and the concession is real: such enlargement always exists in principle. But the enlarged variables are not free. The hysteron population in a Preisach model of a transformer core is inferred by driving the material and integrating its response over time — you obtain the sufficient state by watching the path. Markovianising relocates the requirement rather than dissolving it, and the state decays as the material ages, which reopens the need to watch again.

This is analogy wearing the costume of derivation. Corpora are not thermodynamic ensembles and gradient descent is not relaxation to equilibrium.

Granted without reservation. Nothing here is derived from the second law, and no statistical model commits a physical error by having stationary, order-free statistics. The claim concerns the referent, not the estimator: if the world being represented breaks detailed balance, a representation that is itself stationary and order-free is mis-specified with respect to that target, and the mis-specification shows up on exactly the hysteretic quantities that matter.

Sampling theory says a band-limited signal is fully recoverable from periodic samples. Continuity is engineering excess.

For processes with known, finite correlation times this is simply true, and periodic re-measurement is the right, cheaper answer. It fails for avalanche-like dynamics — Barkhausen jumps, earthquake sequences, credit cascades — where event sizes are heavy-tailed and arrival times unschedulable, so a fixed cadence misses precisely the informative episodes. This genuinely narrows the claim: continuous intake earns its cost only where the underlying process denies you a safe sampling rate in advance.

Detailed balance concerns microscopic reversibility, and fundamental physics already contains asymmetry — CP violation, cosmological expansion. Statistical irreversibility conflates unrelated phenomena.

The two should not be merged, and the argument does not need the exotic case. It rests on statistical, boundary-condition-driven irreversibility — ubiquitous at engineering scale, arising from sustained driving and low-entropy starting conditions — not on fundamental asymmetries too small to matter to a clay bed or a steel core.

The misreading to disown

The weak version of this argument says: history matters, therefore log everything, forever. That is a storage policy, not a physical claim, and it invites the fair complaint that most history is noise. The sharper point is about what must be retained: the ordering that identifies which path was taken, plus enough of the driving record to estimate the hidden internal variables that path implies. A longer context window does not supply either, if the ordering was thrown away at the point of collection. Retention without provenance just reproduces the original defect, more expensively.

History dependence is not a call to remember more; it is a call to remember in order.

What this does and does not establish

It establishes that state alone underdetermines dynamics in systems held away from equilibrium, and that recovering the missing variable requires access to a path, not a corpus or a snapshot. It maps that requirement onto the intake axis and shows why continuous, provenance-stamped observation is the terminal position on that specific axis. It does not establish that such observation is sufficient for good judgement, correct inference, or intelligence of any kind. Watching everything, in order, is a precondition for representing a trajectory faithfully. What is done with that representation is a separate question, and this argument does not answer it.

Continue