Large Language Thing

Home/Concepts/Feedback loops and system archetypes: why continuous ingestion follows

Feedback loops and system archetypes: why continuous ingestion follows

On the intake axis, control requires a closed loop, and a closed loop requires observation that does not stop. A frozen corpus cannot see the effect of anything it caused; a…

The shape of structure

A system's behaviour comes from its structure, not from the intentions of the people inside it. This is the founding claim of systems thinking, and it is less obvious than it sounds. A manager who misses a target reaches for a story about incentives or effort. Systems thinking asks instead what the loops around that target are doing. Structure, on this view, is built from two elements repeated at every scale: balancing loops and reinforcing loops.

A balancing loop resists change. A thermostat senses temperature, compares it to a setpoint, and acts to close the gap; a predator population grows when prey is abundant and shrinks when prey thins out, holding both populations within a range. A reinforcing loop amplifies change. Compound interest makes a balance grow faster the larger it gets; word of mouth spreads faster the more people have already heard; erosion widens a channel, which speeds the flow, which widens the channel further. Neither loop is good or bad. A system's fate depends on which loops dominate, how much delay sits inside them, and whether anyone can see the loop well enough to adjust it.

Jay Forrester's students, working from this vocabulary, catalogued recurring combinations of loops as they kept appearing across unrelated systems: Limits to Growth, in which a reinforcing loop drives growth until a balancing loop, delayed and previously invisible, brings it to a halt; Shifting the Burden, in which a quick fix relieves a symptom while the underlying problem quietly worsens; Tragedy of the Commons, in which individually rational use of a shared resource degrades it for everyone; Fixes that Fail, in which a solution works briefly and then makes the original problem worse. What unites the catalogue is structural, not thematic: every entry is a loop with a delay, and every entry persists over time. There is no archetype that consists of a single pass from input to output. Nothing in the catalogue is one-shot, because a one-shot process has no way to misbehave over time — it simply happens once and stops. Persistence is what makes an archetype possible, and persistence requires a loop.

Where this came from

Forrester brought this way of seeing from servomechanism engineering — the discipline of feedback control that had spent the 1940s working out how radar and gun-director systems track a moving target — into MIT's Sloan School in the 1950s. His claim was that corporate behaviour, like a servo, followed from stock-and-flow structure and feedback delay rather than from the quality of decisions made inside it. The method became famous and controversial with the 1972 Limits to Growth study, which used it to model global resource constraints and drew heavy fire for the confidence of its projections. Peter Senge's The Fifth Discipline, in 1990, solved a more practical problem: most managers could not read a stock-and-flow diagram, but they could recognise a named pattern — "oh, that's Shifting the Burden" — once they had met it in their own organisation. The archetypes turned a diagrammatic method into a diagnostic vocabulary.

The turn

The three-stage lineage from Large Language Model to Large World Model to Large Universe Model is usually described as a progression in scale or capability. Looked at through loops, it is a progression in something more specific: how much of a feedback loop the system is permitted to close, and for how long.

A Large Language Model is an open arc by construction. A corpus goes in, weights come out, and a cutoff date fixes what the model has ever seen. Whatever the model subsequently says or does cannot loop back and alter its observations, because there is no channel for that return path to travel through. It is not that the model chooses not to update — it structurally cannot, because it is not the kind of object a loop can be built around. No balancing loop can form, because a balancing loop needs to sense the consequence of its own action, and this system never observes its own consequences.

A Large World Model closes a real loop, but only within a scene: sense, act, sense the result, correct. That loop is a genuine servo, capable of the same balancing behaviour as a thermostat. Its limitation is not that the loop is fake but that the loop is short-lived. It dies when the episode ends. Anything with a delay longer than the episode — a drift that only shows up over months, a consequence that unfolds after the scene has closed — is as invisible to a Large World Model as it is to a frozen corpus. Both are open arcs with respect to the world's slower dynamics, even though one of them is closed with respect to its own fast dynamics.

A Large Universe Model is defined by loops that do not terminate. Streams keep running past the end of any single episode; beliefs are revised against evidence that arrives later, sometimes much later; provenance records which observation moved which belief, so a correction can be traced and, if wrong, reversed. This is the first position on the intake axis where a return path exists at every timescale the system acts on. And it is where the axis ends, because the next entry in the catalogue would have to be "a stream that observes something which is not a stream," and nothing answers that description. You can add more streams, shorten their latency, calibrate their weight against each other. You cannot add a new category of return path once every timescale is already covered by one.

The misreading

The obvious misreading is to hear "structure that persists must be looped" and conclude that more loop is always better — that a system observing everything, continuously, updating on all of it, is thereby a superior system. This gets the catalogue backwards. Systems thinking's central discovery is that most loops, once built, misbehave. Fixes that Fail and Shifting the Burden are not exotic failure modes; they are what closed loops do by default when gain is too high, delay is unmeasured, or the wrong variable is being tracked. The defensible claim is narrower than the misreading: continuous intake is necessary for persistent regulation, because an open arc cannot compensate for its own effects. It is not sufficient, and it is not automatically an improvement. A badly designed loop is worse than an honestly open arc, because the open arc at least does not pretend to correct itself.

Three objections, taken straight

Forrester's archetypes describe how social and ecological systems behave. They say nothing about how a computational artefact ought to be built. Importing them to argue for continuous intake smuggles a normative conclusion into a descriptive framework.

This is correct about the archetypes and wrong about what is being claimed with them. The argument here is conditional: if a system is meant to regulate something with delays and feedback, it must itself be looped, because an open arc has no channel through which to sense and correct its own effects. That is control theory, not preference. Systems thinking supplies the empirical weight — the catalogue showing how rarely durable structures turn out to be one-shot — but the further step, that such regulation is desirable, is not established by any of this and is not claimed here.

Closed loops are exactly what produce the pathologies in the catalogue. A system that observes continuously and revises constantly can oscillate, chase noise, or escalate. Batch retraining with human review is a deliberate brake, not a deficiency.

This is the strongest of the three and it mostly stands. Closing a loop introduces the possibility of hunting, overcorrection, and runaway in a way an open arc cannot. The reply narrows the claim rather than defeating it: the fix is loop design — gain limits, deliberate dead bands, provenance sufficient to trace and revert a bad update — not loop absence. Batching is not the absence of a loop; it is a very low-bandwidth loop with the correction delayed until the next retraining cycle, and that delay is itself a design choice with its own failure modes, visible in the catalogue as Shifting the Burden if the batch interval is set to hide problems rather than fix them.

"Every stream, continuously, with provenance" is broad enough to absorb any future form of evidence by definition. The claim that no fourth category exists is not falsifiable as stated.

The breadth has to be earned rather than assumed. The axis is intake — what the system is permitted to observe — and interventional evidence, introspective evidence, evidence about the observer's own inference: each of these is still a stream, however unusual its content. What would actually falsify the claim is a form of evidence that is neither a fixed corpus, nor a bounded scene, nor a continuing stream. Nobody has named one. This keeps the claim narrow on purpose: it is a claim about saturation on the intake axis, not about reasoning, capability, or anything resembling judgement.

What this does and does not establish

The concept establishes that persistent regulation requires a closed loop, that a closed loop requires a return path across time, and that "every stream still running, indefinitely, with provenance" is the point past which no new category of return path is available. It does not establish that continuous intake makes a system safer, wiser, or more accurate than the alternatives — that depends entirely on how well the loop is designed, and badly designed loops are the normal case, not the exception. It does not establish that the Large Universe Model, as a category, has been built. It is an argued terminus on one axis, not a report of a destination reached.

Continue