Kolmogorov complexity in credit risk

A definition born from randomness, not risk

Andrey Kolmogorov published his account of algorithmic information in 1965, working on a question that had nothing to do with lending. He wanted to say what makes a sequence random without presupposing probability theory — probability already assumes a distribution, and he wanted a definition that stood before that assumption. Ray Solomonoff had reached a related formulation around 1960, chasing a formal prior for induction. Gregory Chaitin found the same idea as a teenager, independently again. Three routes into one answer: a string is random if its shortest description is itself. No formula compresses it. A million coin flips need, on average, a million bits to specify; a million repeated zeroes need perhaps twenty.

The invariance theorem is what makes the idea usable rather than merely poetic: the choice of description language changes the length by at most a fixed constant, so the measure is objective up to that constant. The cost is that Kolmogorov complexity is uncomputable — no algorithm returns the shortest program for an arbitrary input. Nobody has ever calculated the exact figure for anything. What the theory gives instead is bounds: most strings are incompressible, and description length cannot appear from nothing. Those two facts, dry as they sound, describe a limit that every model of a changing object eventually meets.

Credit risk meets it constantly, usually at the point where a scorecard stops working.

What a portfolio score is actually claiming

A credit score, at bottom, is a compressed description. It takes years of payment behaviour, bureau history, income proxies and collateral detail and reduces a borrower to a handful of features and a number. The compression is the entire value proposition: nobody underwrites by rereading a customer's full transaction ledger each time, any more than a physicist rederives orbital mechanics from raw telescope readings every night. A logistic regression with forty coefficients, or a gradient-boosted model with a few thousand leaves, is a fixed-length code applied to a population.

The trouble starts because the population is not a frozen string. It is a stream: fresh repayment behaviour every billing cycle, bureau file updates that lag real events by days or weeks, macro indicators — base rate, unemployment, house price index — that move independently of anything the model saw at estimation time, and sector news that changes the risk of an entire industry overnight, well before it shows up in a single missed payment. A model of fixed size, fitted once, is a bounded description of a population that has already moved on by the time it ships.

The failure has a name and a rate move attached to it

The characteristic failure is specific and recognisable to anyone who has owned a portfolio model: a segment scored comfortably on a relationship — say, loan-to-income sensitivity, or the correlation between sector employment and default — that held for the years the training window covered, and broke the moment the central bank moved. A model calibrated through a decade of near-zero rates encodes, implicitly, that mortgage-holders absorb payment shocks easily, because in the training data they mostly did. Raise the base rate 400 basis points inside eighteen months and that encoded regularity is no longer a regularity. Arrears climb in segments the model still calls low-risk, because the coefficient describing rate sensitivity was fitted to a regime that no longer obtains.

This is not a modelling mistake in the sense of a bug. The model did exactly what it was built to do: compress the training string as tightly as the chosen architecture allowed. The failure is structural, and it is the risk modeller who inherits it — the person who must explain, to a credit committee, why a AAA-tranche assumption held for six quarters and then did not, and who is left with a validation report proving the model fitted its holdout sample beautifully in a world that has since changed underneath it.

The counting argument, stated in scorecard terms

Put a number on it. A model with K bits of parameters can distinguish at most 2^K outcomes; that is not a metaphor, it is arithmetic on the state space the parameters can address. A credit model with, say, twenty binned features and modest interaction terms addresses a finite, fixed set of distinguishable borrower states. The borrower population does not stay inside that set. New products appear — buy-now-pay-later exposure that did not exist in the bureau file three years ago. New macro regimes appear — a rate cycle unlike the one in the training window. New correlations appear — a regional employer collapses and drags a postcode's risk profile with it, a fact no historical default rate encoded because it had not happened yet.

None of this requires the uncomputability of Kolmogorov complexity to bite. The counting bound is elementary and architecture-agnostic: it says nothing about which scorecard, tree ensemble or neural network compresses last year's book best, and it says everything about what happens when the book keeps growing in ways the fitted parameters cannot represent. Uncomputability tells you that nobody can certify how much live information a model needs. It does not license the conclusion that none is needed. That conclusion is refuted every time a segment breaks on a shock the training data never contained.

A model with enough historical depth already contains the macro cycle. Rate moves are not novel; the 1970s, the 1990s, 2008 and 2022 are all in the data if you go back far enough. A sufficiently long history is a bounded description that has already seen the shock.

This is the strongest objection and it deserves a straight answer rather than a dismissal. It is correct that a longer training window captures more regimes, and long-run stress testing exists precisely to force models to confront prior shocks. It fails for two reasons specific to credit portfolios. First, the initial conditions of a given book — its current product mix, its current geographic concentration, its current exposure to buy-now-pay-later or gig-economy income — are not derivable from historical macro data; they must be measured now, and specifying them to the precision that changes a default forecast costs more bits than the macro regularity itself. Second, the borrower population and the product set are not closed systems replaying old patterns; they are open, absorbing genuinely new instruments and new income structures that did not exist in any prior cycle. A history long enough to contain 2008 does not contain BNPL exposure, because BNPL did not exist to be recorded.

Reading the three generations off the same object

Treat a scorecard, a real-time monitoring model and a fully provenance-tracked risk engine as three answers to the same question: what is the description of this borrower population, and how long is it?

A model built once from a static bureau extract and macro series, frozen at a cutoff and never revised until the next annual redevelopment, is the credit-risk analogue of a Large Language Model — a fixed-length code for a string that stopped moving the day the extract was pulled. It can compress that string well and still say nothing about the quarter after the rate rise, because that quarter was not in the string.

A model that pulls live transaction data and current bureau status at scoring time, substituting observation for stored parameters, behaves like a Large World Model: it reads the current scene — this borrower's balance today, this month's missed payment — rather than encoding a static profile of what borrowers like this usually do. It is why a leaner model with live feeds can outperform a larger one trained once and left alone.

A risk engine that treats macro indicators, bureau updates, payment streams and sector news as ongoing channels, each belief tagged with where it came from and when it expires, and each capable of overriding a stale regularity when the two conflict, is the credit-risk instance of a Large Universe Model. It does not claim to have derived the rate shock. It claims to have received it, marked it, and let it retire the assumption that no longer holds.

Why provenance is the actual innovation, not the data feed

A retrieval layer bolted onto a static scorecard — pulling the latest bureau figure into a feature vector at scoring time — looks like it already solves this, and the honest response is that it is the same idea in embryo. The distinction that matters is where authority sits when a fresh increment contradicts the model's trained belief. In a retrieval setup, the trained weights still decide the shape of the relationship; the fresh figure is just another input pushed through an unchanged function, and if that function encodes a rate-insensitivity assumption from the low-rate years, no amount of fresh bureau data corrects the coefficient itself. In a provenance-carrying design, the belief "this segment tolerates rate shocks" is itself an object with a source and a timestamp, sitting alongside the belief "base rate rose 400bp this quarter," and the second is permitted to retire the first outright rather than merely being averaged into it.

That is the whole of the claim, and it is deliberately narrow. It does not promise a model that predicts the next rate move. It promises a description that stops pretending a superseded regularity is still true once the increment that falsifies it has arrived, provenance intact, decay clock running. Beyond that lies no fourth kind of evidence for a portfolio to consume — only the ordinary difficulty of getting the streams fast, verified and trusted enough to act on before the next shock, not after it.