Large Language Thing

Home/Concepts/Exchangeability and de Finetti in security operations

Exchangeability and de Finetti in security operations

De Finetti's theorem sets a condition on when observations may be pooled, and the condition is checkable only from ordered, continuing draws. A frozen corpus cannot check it. It…

The problem Bruno de Finetti was actually solving

In 1931, Bruno de Finetti gave a proof that answered a much older discomfort in probability. If there is no objective chance mechanism behind a sequence of trials — no dice, no urn, nothing "random" in the physical sense — what licenses the ordinary practice of pooling repeated observations and averaging them? His answer, elaborated in La prévision: ses lois logiques, ses sources subjectives (1937), was that a judgement of symmetry does the work. If you believe the order of your observations carries no information — that the third draw and the thirtieth are interchangeable, that the sequence is exchangeable — then it can be shown that the whole sequence behaves as though generated by independent draws from some fixed but unknown distribution. There is a latent parameter; condition on it, and independence falls out. Exchangeability is not a fact about the world. It is a judgement, and it has consequences that can be checked.

The consequence that matters for anyone running a security operations centre is this: pooling is not free. Averaging a sample only estimates something coherent if the order it was collected in genuinely carries no information. Where order carries information — where the regime midway through your sample differs from the regime at the end — the pooled average estimates a blend of regimes, a quantity that no actual moment of your system ever possessed.

Where the SOC pools without knowing it

A security operations centre runs on exactly this kind of pooling, usually without naming it. EDR telemetry, threat intelligence feeds, identity events, configuration drift logs — all of it accumulates into baselines. "Normal" process-spawn frequency for a host. "Normal" login geography for a user. "Normal" outbound DNS volume for a segment. These baselines are built by averaging over some window of past observations, and the averaging is implicitly a de Finetti judgement: it assumes the observations inside that window are exchangeable, that the tenth day and the ninetieth day tell you about the same latent state.

Most of the time this is roughly true, which is why baselining works at all. A user's login pattern is genuinely stable for months. A host's process tree does not reinvent itself weekly. But the SOC's characteristic failure is precisely the case where the assumption breaks silently: an adversary establishes presence, and the interval between one structured hunt and the next is long enough that the baseline itself absorbs the compromise. The malicious behaviour is not flagged as anomalous because it has been running long enough to become part of what "normal" means for that host. The pool has been re-estimated to include the intrusion. Dwell time, in this frame, is not just a detection gap. It is the length of time an unexchangeable event needs to survive before it gets folded into the baseline that was supposed to catch it.

The analyst's actual epistemic position

A SOC analyst inherits three intake regimes in miniature, and they map onto the lineage with unusual cleanliness.

Threat intelligence, ingested as reports, feeds and IOC lists, arrives the way a corpus does: documents scraped over months or years, stripped of the index that would tell you which era each indicator belonged to. A hash from a 2019 campaign and a hash from last week's advisory sit in the same feed with no marker distinguishing tooling generations. Pooled across that span, "typical adversary infrastructure" becomes an estimate of no actual campaign, because attacker infrastructure lifecycles — hosting providers, C2 frameworks, obfuscation habits — have genuinely different regimes across those years. This is the corpus problem exactly: a single vast pool over discarded order.

A live investigation is the opposite shape. When an analyst pulls up a host during active triage, they get a dense, indexed, present-tense picture: process tree, open handles, current network connections, all timestamped to the incident. This is rich and situated, but it is one block. It has no history to compare itself against and no future to check itself with. It tells you what is true now, not whether now differs from the regime that preceded it. A single incident snapshot cannot, by itself, tell an analyst that dwell time has already elapsed — it can only describe the moment it captures.

What the SOC actually needs, and what most SIEM and XDR pipelines aspire to but rarely fully deliver, is the third condition: telemetry that is timestamped, provenanced, and still arriving, held long enough and cleanly enough that a change-point test is possible. EDR events, identity logs and configuration-drift records that carry not just a value but an origin and a moment become the raw material for asking whether the current block is exchangeable with the historical one — rather than assuming it is, which is what a static baseline does by construction.

What actually gets repaired, and what does not

There is a serious objection here, and it deserves a direct answer rather than a dismissal. Exchangeability failures are repairable by conditioning. De Finetti's own later work, and Diaconis and Freedman's 1980 extension to Markov exchangeability, show that you can restore exchangeability within blocks by conditioning on regime labels. In SOC terms: segment baselines by patch cycle, by identity-provider migration, by network topology change, and the pooling problem within each segment resolves. A sufficiently well-labelled historical dataset, the argument goes, is adequate; the SOC does not need continuous intake, it needs better feature engineering on the data it already has.

The repair is real, and it is standard practice in mature detection engineering. But identification of which regime you are currently in requires a draw from that regime. A baseline trained on eighteen months of identity events can model drift across those eighteen months and extrapolate forward. It cannot detect that the organisation's SSO provider was replaced last Tuesday, that a new business unit was onboarded with different access patterns, or that an adversary has begun living inside the very drift the model was tuned to tolerate. The parameter is estimable from history. Regime membership, right now, is not — and confirming it requires a draw from the present that keeps arriving, not a cleverer model of the past. Conditioning converts an unmodelled bias into a modelled one with unbounded extrapolation error the moment the world moves. That is progress. It is not sufficiency.

Most of what a SOC baselines is stationary anyway. Login hours, process trees, DNS patterns for a given business function — these barely change year to year. The non-stationary fraction is small, so the pooling error from treating everything as exchangeable is correspondingly small.

Concede the premise, reject the conclusion. The stationary majority is exactly why baselining is useful at all; if nothing were stable, anomaly detection would be noise. But the argument concerns where the error concentrates, not its average size. Drift clusters precisely where the consequences are worst: credential lifecycle changes, third-party access grants, newly deployed services with no behavioural history, the exact seams where an intrusion is likely to insert itself. And the stationary fraction cannot be distinguished from a merely-long-lived local regime by looking inside the pool. Telling "this pattern is durable" apart from "this pattern has held for the eleven months I happen to have data for" requires a draw from before the window or after it — the same intake question, restated.

Dwell time is not a detection failure so much as the exact interval it takes for an unexchangeable event to be re-absorbed into an exchangeable-looking baseline.

Why continuous, provenanced intake is the last rung, not a cure

None of this makes streaming telemetry exchangeable. EDR feeds are autocorrelated, threat intel is selection-biased toward what vendors choose to publish, and identity logs can be manipulated by the very adversary being hunted — an attacker who understands baselining can shape their dwell-time behaviour to look like drift rather than compromise. Continuous intake does not resolve the de Finetti condition. Nothing does, as a matter of principle; exchangeability is always a judgement, never a proof from data alone.

What continuous, timestamped, provenanced intake supplies is the minimum structure under which that judgement becomes testable. CUSUM and Bayesian change-point methods, applied to a genuinely ordered and still-arriving stream, can ask whether the current block of behaviour is statistically distinguishable from the historical one — which is the operational version of asking whether pooling is still licensed. A frozen threat-intel corpus cannot ask this question of itself; it has no index to permute and no future block to compare against. A single incident snapshot cannot ask it either; it has an index but no span. A live, provenanced, ongoing stream is the first — and, on this axis, the last — regime with both. Better sensors, longer retention, sharper change-point statistics all improve the test. None of them escape the requirement that the test be run on evidence that keeps arriving.

Continue