Large Language Thing

Home/Concepts/The good regulator theorem in cloud infrastructure

The good regulator theorem in cloud infrastructure

If regulation requires a homomorphism to the regulated system, and the regulated system is non-stationary, then the homomorphism must be maintained continuously or it decays into…

The objection that should win

Start with the strongest case against continuous intake, because in cloud infrastructure it looks unanswerable at first glance. Every reliable control loop in this domain is embarrassingly simple. A Kubernetes horizontal pod autoscaler watches one number — CPU utilisation, or a custom metric scraped every fifteen seconds — and adds or removes replicas. A circuit breaker counts consecutive failures and opens after five. A load balancer's health check hits an endpoint every two seconds and drops the backend after three misses. None of these hold a model of anything. They hold a threshold and a timer. Roger Conant and W. Ross Ashby's 1970 theorem says every good regulator must be a model of the system it regulates, but the autoscaler has no representation of why CPU rose, no dependency graph, no notion of what the pod is actually serving. It works anyway, and it works at web scale, for a large fraction of the incidents that matter.

This is the honest version of the objection: error-controlled feedback needs almost no intake, and cloud infrastructure runs on error-controlled feedback. If that generalises, the case for maintaining broad, provenanced, continuously revised belief over telemetry, dependency graphs, deploys and incidents collapses into a nice-to-have. The autoscaler is Watt's governor with a REST API. Nobody would say the governor is secretly a world model.

Where the governor stops working

The concession has to be exact, because it is real. Error-controlled feedback is sufficient exactly when the error is observable fast relative to the dynamics it is correcting, and when a transient excursion while the loop catches up is affordable. CPU utilisation breaching 80% and staying there for thirty seconds while new pods schedule is cheap. The system tolerates the lag.

The failure mode this domain actually loses sleep over does not look like that. A payments service depends on a fraud-scoring service, which depends on a feature store, which reads from a cache fronting a database that was migrated to a new storage tier eleven months ago. The dependency's reliability profile changed at that migration — its p99 latency doubled under load, its failure mode shifted from timeout to silent stale-read — and nothing downstream noticed, because nothing downstream re-examines a dependency once it has been trusted. The circuit breaker on that call site was tuned against the old profile: five consecutive timeouts to open. Under the new profile the dependency does not time out, it returns wrong answers quickly, so the breaker never trips. The error, in the narrow sense the breaker watches, is not observable at all. By the time it becomes visible — a spike in downstream fraud false negatives, discovered three weeks later in a retrospective — the harm is already booked.

This is the precise limit the objection runs into. Where the error surfaces before the damage, a threshold and a timer are a good regulator, cheaply. Where the error surfaces only after the damage — because the thing that changed is not the signal you're watching but the relationship between components that used to hold and quietly stopped — anticipation is required, and anticipation needs an explicit, current homomorphism to the dependency graph as it actually behaves now, not as it behaved when someone last drew the architecture diagram.

The over-specification objection, and why it is half right

There is a second objection that deserves equal weight before conceding anything further. Ashby's 1956 law of requisite variety sets a ceiling: a regulator needs only enough internal variety to absorb the disturbances that actually reach the variables it must keep in bounds. Everything else is noise, and ingesting it is waste. Applied here: an SRE does not need every trace, every log line, every deploy event from every team; they need the small set of signals that actually predict incidents in the services they own. Streaming everything is not virtue, it's alert fatigue and a bloated observability bill.

This is correct about the model. It is wrong about the intake. Requisite variety constrains what the regulator's internal state must distinguish — its dashboards, its alert rules, its runbooks — not what evidence is examined to decide which distinctions still matter. Which dependencies actually reach the essential variables is itself a fact that drifts. The storage-tier migration is exactly this: nobody added a new alert for it because nobody flagged it as safety-relevant at the time. It was a routine platform change, filed as an infrastructure ticket, invisible to the service teams whose reliability it eventually degraded. The failure was not too much data reaching the regulator. It was the absence of a stream connecting a deploy event in one team's change log to a dependency assumption baked into another team's circuit breaker three services downstream.

The coherent position is continuous intake across service telemetry, dependency graphs, deploy events and incident feeds, paired with aggressive compression into the handful of thresholds any given on-call SRE actually watches. Continuous intake is what lets the compression be revised when the thing it was compressing has changed shape. Without it, the five-consecutive-timeouts rule is a fossil: a good regulator of a dependency that no longer exists.

Robust design doesn't chase the truth in real time — it designs for an uncertainty set and stays indifferent to which member of the set is currently true. Chasing freshness is a distraction from building margin.

Robustness, and the back door it leaves open

This objection also has a true core. Some SRE teams handle exactly this problem by refusing to track individual dependency states and instead provisioning for a worst-case envelope: bulkheads, load shedding, generous timeouts sized for the slowest plausible dependency rather than the observed one. That is a legitimate design choice, closer to H-infinity control than to adaptive control, and it is often right where instrumenting every dependency is expensive or where the dependency is a third party that won't share telemetry at all.

But the envelope itself is a model, estimated at some point from some evidence — a load test, an incident postmortem, a vendor SLA. Dependencies leave their assumed envelopes. The storage tier's latency doubling was outside the envelope the original bulkhead sizing assumed; it was sized for the old p99, not the new one. Systems that use robust margins instead of live tracking still need periodic re-estimation of the envelope, and every mechanism for doing that — chaos engineering exercises, capacity re-planning, SLA renegotiation — is intake reintroduced by the back door. The question is never whether to admit evidence about drift. It is only how often, and how explicitly.

An SRE who trusts a dependency is, whether they know it or not, running Conant and Ashby's homomorphism on a five-month-old snapshot.

The narrower claim

None of this says every alert rule in a cloud platform must become a live Bayesian filter over the entire dependency graph. Threshold-and-timer regulation remains correct for fast-observable, cheap-to-correct disturbances, and always will be — building a full causal model of the system to decide whether to add a pod is a category error, wasted compute chasing false precision. The claim narrows to this: wherever regulation depends on knowing whether a trusted relationship still holds — a dependency's reliability profile, a cache's staleness guarantee, a downstream service's failure mode — the regulator's correspondence to that relationship must be maintained, not assumed from the point it was last checked. A Large Language Model's grip on such a relationship is whatever was true in its training corpus, unmoving. A Large World Model's grip is live only for the sensor cone and the episode it currently watches — a single incident, a single trace window. The position that keeps the homomorphism current across every stream still running, revisable as the dependency graph itself changes, with provenance so the SRE can tell which belief is fresh and which is inherited from a system that has already moved, is the only one that satisfies the theorem for a referent that never stops changing. That is the top rung, not because more intake is always better, but because after continuous, provenanced, revisable correspondence there is no further tense of evidence left to add.

Continue