Large Language Thing

Home/Concepts/Observation is theory-laden in cloud infrastructure

Observation is theory-laden in cloud infrastructure

If observation is theory-laden, then any system whose frame is fixed at build time has a ceiling set by its builders' categories, not by its data. Widening intake at a single…

A dawn Hanson would recognise

Norwood Russell Hanson picked an odd example to break the positivists' picture of science. In Patterns of Discovery (1958), he pointed at the sky: Tycho Brahe and Kepler, standing at the same horizon, watching the same sunrise, do not see the same thing. Brahe sees the sun beginning its circuit of a stationary earth. Kepler sees the earth's horizon dropping away from a stationary sun. Same photons, same retina structure, different observation. Hanson's target was the logical positivists' dream of a neutral observation language, a layer of "protocol sentences" untouched by theory on which rival hypotheses could be adjudicated fairly. He argued no such layer exists. What counts as a datum, what counts as noise, what even counts as seen, is set by the frame the observer already holds. Duhem had said something adjacent in 1906: no experiment tests a single hypothesis, only a bundle of theoretical commitments together. Kuhn, in 1962, turned the idea into a theory of scientific revolutions themselves. Feyerabend pushed it further, into incommensurability between paradigms that could not even fully translate each other's terms.

None of this was meant as licence for relativism. Hanson and Kuhn were both explicit: theory-ladenness does not mean any frame is as good as any other. Anomalies still pile up. Instruments still fail in ways that can be traced. A successor frame still has to explain what the old one got right, plus the residue it couldn't. The claim is narrower and harder to dismiss: observation is never raw. It arrives already sorted by categories the observer brought to the encounter.

The dependency graph as protocol sentence

Move from Tycho's horizon to a service mesh, and the same structure reappears with different nouns. An SRE watching a dashboard is not viewing raw telemetry. They are viewing telemetry through a dependency graph someone built, months or years ago, encoding a theory of which services call which, at what expected latency, with what failure modes worth alerting on. The graph is the frame. It decides what counts as a signal.

Here is the characteristic failure, and it is not exotic: a downstream dependency — a managed queue, a third-party auth provider, an internal caching layer — was reliable for two years. The team's mental model, baked into runbooks, alert thresholds and the on-call rotation, treats it as a stable floor: "that service basically never goes down, don't page for it." Then its reliability profile changes. A provider migrates infrastructure. A library upgrade introduces a retry storm under load nobody tested. The dependency now fails at 2% under peak traffic instead of 0.01%. Nothing in the observation pipeline notices, because nothing in the frame is looking for a changed failure rate on a component classified as trustworthy. The telemetry keeps arriving. It gets filed under "known good," because that is the category the frame supplied at design time, and the frame is not itself under observation.

This is Le Verrier's move, run in production. Newtonian frame, applied correctly, found Neptune from Uranus's perturbations in 1846, within a degree. The identical frame, applied to Mercury's perihelion precession, produced Vulcan — a planet that does not exist. Same inferential move, one triumph, one phantom, because the frame decided in advance what an anomaly meant. The SRE's dashboard does the same thing to a degraded dependency: an anomaly under the old frame ("brief blip, ignore") is exactly the signal a revised frame would treat as the whole story.

Incident retrospectives are Kuhn's anomaly log

Anyone who has run a postmortem knows the ritual: reconstruct the timeline, find the moment the system's behaviour diverged from the model, and ask why nobody caught it sooner. The honest answer is usually that someone did catch it — a latency graph had a slow upward drift for six weeks, a retry count crept from background noise to a visible slope — and it got dismissed, because the monitoring frame classified that metric as low-priority for that service. The data existed. It wasn't a datum, in Hanson's sense, because nothing in the frame made it salient.

Kuhn's anomalies mattered historically because astronomers wrote them down for centuries before anyone had a paradigm that could use them. The equivalent discipline in cloud infrastructure is logging unexplained variance with its source, timestamp and instrument before classifying it as noise. Most incident tooling does the opposite: it samples, aggregates, and discards at ingestion, because storage and cardinality cost money, and the discard policy is itself a theoretical commitment about what won't matter later. Filtering at intake is the operational version of a frame that cannot be revised, because the evidence that would revise it was thrown away before anyone could look at it twice.

Why widening the sensor suite doesn't fix it

A tempting response is: add more telemetry. More metrics, more traces, higher-cardinality dashboards. This is the Large World Model move — a bounded, richer scene, sensed live, but sensed through an ontology of "service," "dependency," "incident" fixed by the observability platform's schema before anything was deployed. It genuinely helps: an SRE watching a live scene can correct a wrong reading within that scene, the way a live dashboard beats a static report. But the categories that decide what counts as an object worth tracking, what counts as an edge in the dependency graph, what counts as a boundary between "this service" and "that one," were set by tooling choices and org-chart conventions made in advance. Widening the sensor suite fills those categories faster. It does not expose them to revision, because nothing forces the schema itself back onto the table.

The Large Universe Model position on this axis is the claim that the frame has to become a revisable object, and that this requires two things simultaneously: streams that never stop, and provenance that survives the classification step. A dependency's reliability profile is not a fact fixed at onboarding time; it is a belief with a decay rate, and the only way to catch its drift is to keep watching it after you've stopped thinking you need to, with a record of exactly which reading, from which probe, at which time, contradicted the "known good" label.

More data under a bad ontology is just a more confident error. Continuous ingestion doesn't guarantee revision — it can just as easily entrench the wrong dependency model at higher throughput.

This objection is correct, and it names the real failure mode: confirmation at scale. An SRE with ten times the telemetry and the same triage rules will produce ten times the false confidence, not ten times the insight. Continuity is necessary and not sufficient. What makes it bite is the retention of disconfirming residue — keeping the readings the frame couldn't parse, rather than aggregating them into a mean that erases them. The architectural commitment is to record before you classify: store the anomalous latency spike with its trace ID and upstream call chain even when the alerting rule says "below threshold, discard." That log is the material a revision draws on. A pipeline that filters at ingestion has no such material, however many streams feed it.

A dependency graph is a theory of the system wearing the costume of a diagram.
Frame revision is a normative, social act — an SRE team decides to reclassify a dependency for reasons of caution, incident fatigue, or a bad quarter, not because a stream of numbers logically forced the conclusion.

Also correct, and it narrows the claim rather than breaking it. Continuous, provenance-bearing telemetry does not choose the new reliability model for a dependency. Two plausible revised models — "this provider now has a 2% failure floor under load" versus "this was a one-off migration event, treat as resolved" — can be equally consistent with three weeks of data. No stream count settles that by itself. What continuous observation does is smaller: it keeps the disagreement visible, shrinks the space of live hypotheses as more incidents either recur or don't, and forces the team to a decision point rather than letting the old classification survive by default because nobody revisited it. The verdict — downgrade this dependency's trust tier, add a circuit breaker, renegotiate the SLA — stays human and institutional. The system supplies the pressure that makes ignoring the question costlier than answering it.

Where the ladder actually ends

A Large Language Model's frame is fixed at the corpus cutoff: it cannot notice that a dependency it "knows" as reliable has since degraded, because no stream reaches it after training to say so. A Large World Model watches a live incident dashboard and can correct a misread metric within that session, but the schema of what counts as a service, an edge, a boundary, was frozen before deployment and is not itself up for revision by the scene it's watching. Only a frame kept open indefinitely, with every stream's provenance intact, can catch a dependency's reliability profile changing after the fact that it was ever trustworthy has stopped being news.

That is the terminal rung on this axis, and it is worth stating plainly what it doesn't claim. The classification schema — services, dependencies, incidents, timestamps — is itself a theoretical commitment, chosen in advance, no less fixed than a corpus cutoff. That's a real limit, but it sits one level up from the intake question. On the axis of what a system may observe — a frozen corpus, a bounded present scene, or every running stream with provenance — the third option exhausts the space. Nothing beyond it is a new kind of seeing. What's left is quantity, latency, trust and time.

Continue