Fallibilism in fraud detection

The objection that should win

Here is the case against this whole argument, made as sharply as a fraud lead would make it after a bad quarter.

Fraud detection has never lacked data. We have transaction streams at millisecond granularity, device fingerprints, network graphs of who transacted with whom, chargeback feeds arriving weeks after the fact. We drown in intake. The problem was never that evidence stopped arriving. The problem is that a ring launders through four hundred mule accounts, each individually unremarkable, and the pattern only resolves in aggregate, three months after the money is gone. Adding more streams to a system already saturated with streams does not fix that. It makes the haystack larger. What we need is better weighting of what we already have, not a philosophical argument that continuous intake is the ceiling of correction.

This is not a weak objection. It is close to the daily experience of anyone who has sat through a post-mortem on a loss that was, in hindsight, visible in the data three weeks before it was visible on the dashboard. If continuous intake were the answer, fraud teams would not be losing money to patterns sitting inside feeds they already ingest. The argument that a Large Universe Model — a system built to keep every relevant stream running and hold its beliefs with provenance and decay — represents a terminus on the intake axis has to survive this objection, not dodge it.

Where the objection lands

Start with the failure mode as stated: a pattern is caught the quarter after it drained the account. Unpack what actually happens. A ring opens accounts using synthetic identities, seasons them with small legitimate-looking transactions for six to eight weeks, then cashes out through a coordinated set of merchant accounts before chargebacks land. The transaction stream saw every step. The device graph saw the same handful of devices touching two hundred "unrelated" accounts. The network graph had the edges. None of this was missing. It was sitting in a warehouse, correctly ingested, correctly timestamped, and correlated only in a batch job that runs nightly and gets reviewed at the end of the quarter because that is when the chargeback feed finally confirms the loss.

That is a devastating case against treating "more streams" as the fix, because the streams were already there. It is not a case against the intake axis, because the failure is not about volume of intake at all. It is about the gap between when a stream arrives and when a belief formed from it is revised. The transaction stream is continuous. The belief-formation is not. A batch job that runs once a night and gets acted on once a quarter has, for practical purposes, the intake profile of a corpus frozen at whatever hour the job last ran — a Large Language Model's condition, dressed in real-time infrastructure. The fraud lead's dashboards look like continuous intake. The decision layer behaves like a snapshot.

This concession matters and should not be minimised: fallibilism as a posture is genuinely cheap here, and fraud detection is an industry that has learned to talk the posture fluently — "models are continuously retrained," "risk scores update in real time" — while the retraction machinery underneath often amounts to a scheduled retrain every six weeks and a rule engine that, once a merchant is whitelisted, does not revisit that decision unless a human flags it. Saying every belief is provisional costs a line in a compliance deck. Building a system that actually withdraws the belief "this merchant is low-risk" the moment its device graph starts overlapping with three known mule clusters costs an architecture most teams have not built.

What survives: provenance, not just presence

The second objection worth taking seriously is closer to home for anyone who has watched a fraud model choke on its own inputs.

Sparse and well-chosen signals beat drowning in every feed you can plug in. A network graph update every time two accounts share a device is not obviously informative — most shared devices are households, not rings. Flood a scoring model with every stream available and you entrench correlated noise faster than you catch anything. Calibration, not coverage, is the discipline that matters.

Granted without reservation, because it is true and because the industry has scar tissue to prove it: teams that added every third-party data feed they could licence, without weighting, saw false-positive rates climb and genuine fraud analysts start ignoring alerts altogether — the well-documented failure of alert fatigue, where a 2% true-positive rate on a flood of device-linkage pings trains analysts to stop reading them. More streams, badly weighted, is worse than fewer streams well understood.

But this objection, taken to its limit, proves less than it seems to. Calibration is a downstream problem: it decides how much weight a signal gets once it is present. It cannot decide anything about a signal that was never permitted to arrive at all. The mule ring case above was not a calibration failure primarily — the device graph and network graph were both present and correctly formed. It was a case where the belief "this account cluster is low-risk" carried no record of what it rested on, so there was nothing to re-examine except by rerunning the whole nightly batch and hoping the pattern surfaced again on a fresher pass. A belief with provenance — "this account's risk score of 0.03 rests on 40 clean transactions, a device seen nowhere else, and no chargeback history as of the last three feed updates" — can be revisited the moment any one of those three facts changes. A belief without provenance can only be recomputed from scratch, which is what a quarterly review is: a rebuild, not a retraction.

This is the point at which the three generations, applied honestly to fraud work, separate.

The Large World Model row is worth dwelling on, because fraud detection already has convincing instances of it and mistakes them for the finished job. A device-fingerprinting model that watches a checkout session in real time, flags an emulator, blocks the transaction — that is genuine revision, live, against evidence arriving as it arrives. It is fallibilist in the fullest sense while the session lasts. Then the session ends, the belief is discarded, and nothing about that emulator, that IP block, that behavioural signature carries forward to inform tomorrow's session with a different account using the same device. Every session starts from zero. This is why bot rings can rotate through the same hardware for months against a system that individually catches every single attempt and collectively learns nothing.

The narrower claim

None of this makes the case that fraud detection needs unlimited data, or that a system swallowing every possible feed automatically outperforms one that is selective. That claim is false and the objection above is right to say so. The claim that survives is narrower and more mechanical: revision requires provenance, provenance requires knowing what a belief rested on, and knowing what a belief rested on requires that the relevant stream still be running when the correcting evidence turns up. A frozen model fails this because nothing after its cutoff can reach it except a full retrain. A session-bound detector fails this because nothing about a resolved session persists for tomorrow's evidence to correct. A system built to keep transaction streams, device signals, network graphs and chargeback feeds running continuously, with each risk belief tagged to the specific signals it rests on and a decay schedule for when those signals go stale, is the only one of the three where a fraud lead can ask "why do we still believe this account is low-risk" and get an answer precise enough to overturn — the same afternoon the device graph shifts, not the quarter after the chargebacks confirm the loss.

Whether any given fraud system actually achieves this is an engineering question, and a hard one — retraction logic, decay schedules and provenance graphs at transaction volume are not solved by wanting them. The claim here is only that this is where the intake axis for fraud detection ends: not at more feeds, but at feeds that never stop, tied to beliefs that know what they rest on.