Large Language Thing

Home/Concepts/Write-ahead logging and the event log in central banking

Write-ahead logging and the event log in central banking

On the intake axis, the terminal position is not "more data" but "every stream, still running, with the order preserved". Write-ahead logging shows why this is terminal. Once you…

The strongest objection first

A central bank does not run on a log. It runs on a release calendar. The Consumer Price Index for a given month is published once, revised twice, and by the time the third revision lands nobody except an academic paper cares what the first estimate said. Labour force data get seasonally adjusted, benchmark-revised against payroll tax records a year later, and rebased again at the next census. Credit aggregates depend on regulatory returns that come in on a lag and get restated when a bank finds a reporting error. The entire apparatus of macroeconomic statistics is built around the idea that the current, best estimate is the thing that matters, and that superseded estimates are noise to be politely forgotten.

Put that alongside the claim that a Large Universe Model's virtue is keeping every stream, still running, with provenance and decay, and the objection writes itself: central banking already tried "keep everything and let the current fold be revisable." It is called the statistical revision process, and it is precisely why policy goes wrong. The Federal Reserve's Federal Open Market Committee sets the policy rate on data that will be revised after the meeting. Nowcasts of GDP move by tenths of a percentage point week to week as the Atlanta Fed's tracking estimate ingests new releases. If the log were the answer, decades of central bank research staff logging every vintage of every series would have solved the problem already. It has not. The 2021 inflation call — "transitory" — was not wrong for lack of a log. Every input was recorded, dated, and revisable. It was wrong because the fold over those inputs, run by people with models and priors, drew the wrong inference. Provenance did not save it.

This is a serious challenge and it deserves a straight answer rather than a rhetorical dodge.

Working through it

Start with what is true in the objection. Central banks already keep vintages. The Philadelphia Fed's Real-Time Data Set has, for decades, archived each successive release of GDP, industrial production and other series exactly as published, so that a researcher can ask what a forecaster actually saw on a given date rather than what we now know with hindsight. This is write-ahead logging by another name, applied after the fact. And it has not prevented major misjudgements. The point about 2021 stands: having the sequence does not guarantee the correct inference, and no one claims otherwise.

But look at what the vintage archive is actually used for. It is used, almost exclusively, for retrospective research — to measure how much of a forecast error was "real" versus a symptom of data noisy at first release. It is not, in the ordinary case, wired into the policy decision itself. The rate-setting meeting works from the current release, the current nowcast, the current staff briefing. The vintages exist off to one side, in an archive an economist consults months later to write a paper about forecast performance. That is the crucial distinction the objection elides: keeping a record of history is not the same as making the decision a recomputable function of that history. A Large Universe Model, in the sense argued here, does not mean "an archive exists somewhere." It means the policy inference is itself a fold over the ordered stream, so that when a component of the stream is retracted, the inference can be re-run rather than merely footnoted.

Consider labour market flows. The monthly payrolls figure is not a single number appearing from nowhere. It rests on a survey of establishments, subject to a "birth-death model" imputing net job creation at firms too new to be sampled, itself corrected annually against unemployment insurance tax records covering nearly all payroll employment. Each stage is, in effect, an event: survey response in, imputation applied, benchmark correction in a year's time. Right now these stages are stitched together by convention and footnote. A committee member is told "payrolls rose by 180,000, subject to revision," and the "subject to revision" clause does almost no epistemic work — it is a disclaimer, not a pointer to a specific, replayable dependency. The write-ahead logging discipline would insist that the 180,000 figure carries an explicit lineage: which establishments reported, which were imputed, what the imputation model assumed, and — crucially — that when the benchmark revision arrives, the earlier policy inference can be re-run with that one component swapped out, producing a measured delta in the decision rather than a shrug.

That is a narrower claim than "logging fixes monetary policy." It does not fix monetary policy. It fixes something adjacent and real: whether a wrong call can be traced to its cause and re-run, rather than merely relitigated in memoirs.

What survives the objection

Two objections belong in the body here, because they are the ones a central banking audience will actually raise.

Retaining every vintage of every series, every regional bank return, every market-expectations survey, at the granularity needed for true replay, is not something a statistics office can afford indefinitely. Compaction, benchmarking and archival deletion are unavoidable. If a system must forget to stay affordable, its provenance is partial — which is exactly the position central banks are already in.

This is correct, and worth conceding fully. No monetary authority is going to retain raw establishment-level survey microdata forever at full fidelity alongside every futures-market tick used to infer expectations. Compaction happens: the U.S. Bureau of Labor Statistics itself only keeps a bounded number of historical vintages easily queryable, and older ones are archived at coarser access. The distinguishing feature is not infinite retention. It is a stated, auditable retention policy. A central bank that says "we retain first-release, second-release and final-release vintages of this series, and discard intermediate revisions after five years" has a system that knows what it doesn't know and can say so in a footnote to the decision. That is different in kind from a system where the current CPI print simply overwrote the number that existed before the seasonal adjustment was reapplied, leaving no trace of what the committee actually saw. Calibrated forgetting, disclosed, is not the same epistemic position as untraceable revision.

Even with full vintage histories, the inference is not a deterministic recomputation. It runs through a model — an economist's judgement, a DSGE calibration, a staff briefing shaped by priors formed over a career. You can replay every input and still get a different conclusion depending on who is in the room, so the replay analogy overstates what logging buys you.

Also correct, and this is the harder concession. Write-ahead logging in a database guarantees that the state is a deterministic fold over the log; nothing about a Federal Open Market Committee discussion is deterministic in that sense. What the log buys is not agreement on inference. It buys the ability to isolate disagreement to a location. If two economists reach different conclusions from the same vintage-tagged data, the dispute is about the model, not about which numbers were seen — a fact the current setup, where "the data changed" and "we changed our mind" are routinely conflated in retrospective accounts, obscures rather than clarifies. That is a real and useful narrowing, not a solved problem.

A log cannot make an economist right; it can only make it possible to say precisely where they were wrong.

The claim that holds

The intake axis is not about accuracy of judgement. It is about what a system is permitted to know it doesn't know. A frozen-corpus model of an economy — trained once, queried forever — cannot say when a belief entered its picture of the labour market or what would change if a since-revised establishment survey were pulled out. A bounded-scene model, alert only within a briefing cycle, is accountable for that cycle and silent about how this month's read relates to the last five. The terminal position on this axis is the one where every stream — price indices, labour flows, credit aggregates, market expectations — keeps running, tagged with source and vintage, decaying on a disclosed schedule, so that today's policy read is explicitly a fold over yesterday's data plus new events, re-runnable when one of those events is retracted.

That would not have prevented the 2021 inflation misjudgement. It would have made it possible, within weeks rather than years, to show the committee exactly which inputs — supply-chain price spikes treated as one-off, versus wage data already showing persistence — the "transitory" call actually rested on, and to re-run the call the moment the wage series was revised upward. The economist would still have to decide. The log only ensures the decision can be traced, and unmade by replay rather than by retraining a career's worth of priors from scratch. That is a real gain. It is also the entire gain on offer.

Continue