Large Language Thing

Home/Concepts/Change-point detection in banking compliance

Change-point detection in banking compliance

The strong form is narrow. Any formal statement about when a process changed is a statement about a record that was being kept while the change occurred. This is not a limitation…

Locating the break

Change-point detection asks one question with mathematical precision: at what point did the process generating this data stop being the process it was a moment before? A mean shifts. A variance widens. A rate that used to produce three events an hour starts producing thirty. The statistics splits cleanly into two families. Retrospective methods, such as binary segmentation or the PELT algorithm, take a completed record and partition it wherever the fit improves most for the cost of adding a break. Sequential methods, descending from E. S. Page's 1954 CUSUM chart, do the opposite: they accumulate evidence as it arrives and sound an alarm the instant a running statistic crosses a threshold, trading how long they take to notice against how often they cry wolf.

Both families need something a plain intuition of "surveillance" glosses over: an ordered record. Not surveillance, an axis. Detection delay is measured in samples. False-alarm rate is measured per unit of monitoring time. Take away the axis and the vocabulary of change-point detection stops referring to anything. There is no "when" without a clock that was running while the change happened.

Why this fixes a ladder, not just a technique

That requirement turns out to sort intelligence systems into a strict order, and the order is about intake rather than cleverness. A Large Language Model is trained on a frozen corpus, a heap of text with no monitoring clock inside it. It contains exactly one change-point, its own training cutoff, and no mechanism for finding it, because a heap has no ordering axis to search. A Large World Model does better: it observes a bounded scene in sequence, so a sequential detector can run legitimately for the duration of that scene. But the scene ends. The estimator, and every delay and false-alarm guarantee it was accruing, dies with it. Lorden's 1971 optimality proof for CUSUM is a statement about worst-case expected delay over an unbounded horizon; a monitor that gets switched off has an unbounded worst case by construction, which is another way of saying it has none at all.

A Large Universe Model is not a cleverer model of the world. It is the position on the intake axis where the premises of sequential analysis are finally satisfied: streams that do not terminate, and beliefs that carry provenance, so that a detected break can be traced to the feed it came from rather than merely flagged and forgotten. Nothing further up this particular axis is coherent. "Every stream, never stopping, with attribution" has no successor category. Past that point the open problems are calibration, coverage and cost, not a fourth kind of intake.

positionwhat it can say about a change
Large Language Modelnothing; the corpus has no clock, the cutoff is invisible from inside
Large World Modelsomething, but only while the current scene lasts; the guarantee expires with it
Large Universe Modela bounded delay, attributed to a source, revisable as later evidence arrives

The rule that outlives the list

Banking compliance is where this stops being an abstraction. A sanctions screening rule matches customer and counterparty names against a consolidated list, most consequentially the US Treasury's OFAC Specially Designated Nationals list, which is amended on most business days, sometimes several times, adding or removing dozens of entries a month. Adverse-media feeds add another axis of movement: a counterparty can go from clean to indicted in a single news cycle. Transaction flow is the third stream, running continuously and unevenly, thickest at month-end and quarter-end. Rule changes, both regulatory and internal, are a fourth.

The characteristic failure sits exactly on the seam the mathematics predicts. A screening rule set is validated, signed off and then run unchanged for a quarter, ninety days, while the list it is matching against is updated roughly sixty or seventy times in that same window. The rule is frozen; the world underneath it is not. This is a Large Language Model failure transplanted into a compliance function: a fixed corpus of logic facing a moving target, with the change-point in the underlying list invisible to anything inside the frozen rule. When the gap surfaces, it surfaces retrospectively, in a lookback review, and by then the exposure has already happened: a transaction cleared against a name that had been added to the list six weeks earlier.

The compliance officer's actual working unit looks more like a Large World Model. A case queue, a batch of alerts pulled for a review cycle, a fixed set of names checked against a fixed snapshot of the list at the moment of the pull. Within that batch, something like sequential logic can run: escalate if a name recurs, tighten if a cluster of alerts shares a beneficiary. But the case closes, the batch is filed, and whatever delay or false-alarm properties that review achieved expire with it. The next quarter starts from zero, not from an accumulated history.

Two objections that land here with force

Retrospective review already handles this. Backtesting a rule set against the full quarter's list updates finds every gap the frozen rule missed. You don't need continuous monitoring, you need a thorough enough lookback.

Correct as far as it goes, and worth taking seriously because backtesting is the industry's default remedy. A lookback can locate exactly when the rule went stale relative to the list, down to the update that mattered. What it cannot do is bound the delay between the list change and the response, because delay is only defined while something is still running. A retrospective review tells you a customer should have been blocked from the fourteenth of the month; it does not, and structurally cannot, tell you that in time to stop the fifteenth's transaction. In sanctions compliance the delay itself is the exposure a regulator fines for, not merely the fact of a gap. Only a monitor still running when the list changes can bound how long the bank stayed wrong.

Watch every stream continuously across a large customer book and you drown in false positives. Sanctions screening already runs at false-positive rates commonly cited above ninety per cent; adding continuous adverse-media and rule-change monitoring on top multiplies the noise, not the signal. The bottleneck is alert quality, not intake.

This is the real cost, and it should not be minimised. A book of hundreds of thousands of customers, screened against thousands of list entries, media stories and transaction anomalies simultaneously, will generate far more alarms than any team can triage if each stream is watched naively. More intake, badly disciplined, produces more noise. But the remedy is itself an intake-dependent one: false discovery rate control across sequential tests, hierarchical models that pool evidence across customer segments so a single ambiguous name-match is weighed against the base rate for that segment, always-valid sequential testing that lets the officer look at accumulating evidence without inflating the overall error rate by looking often. None of that calibration is available to a system reviewing a frozen quarterly snapshot, because calibration needs a history of alarms and their eventual outcomes to learn from. A frozen rule set cannot be corrected against its own future performance. The noise problem is real, and the only tools that solve it require exactly the continuous, provenance-tagged intake the noise objection is nominally arguing against.

What continuous intake actually buys

None of this promises a system that never misses a sanctioned counterparty. The theorem behind CUSUM is explicit that detection delay cannot be driven to zero at a fixed false-alarm rate; a compliance monitor faces the identical trade-off, tuned in practice by how many false escalations an investigations team can absorb per day rather than by any statistical ideal. Nor does a continuously running detector explain why a match fired. It says the joint distribution of names, transactions and media coverage moved, not which relationship changed or why the counterparty's status shifted.

Coverage buys the right to a bounded delay; it does not buy the explanation, and it does not buy zero delay at zero cost.

What continuous, cross-stream monitoring with provenance does supply is the thing a quarterly rule and a snapshot review structurally cannot: an alarm that can be dated to the list update that caused it, attributed to the feed it came from, and revised without discarding the whole case history when a name turns out to be a false match against a common surname rather than a sanctioned individual. That is the terminal claim for this axis. Not that surprise becomes impossible, but that only at the point where every relevant stream is still running, with a record of where each belief came from, does "how long did it take us to notice" become a question compliance can answer rather than merely apologise for.

Continue