Large Language Thing

Home/Concepts/Search theory in oil and gas

Search theory in oil and gas

Optimal stopping requires knowing the arrival rate, and the arrival rate is an empirical quantity. A system that cannot observe arrivals cannot compute when to stop; it can only…

The objection that should win

Here is the case against this page, stated as strongly as it deserves. Search theory, from George Stigler's 1961 formulation onward, gets its clean results from a known arrival rate. Stigler asked why identical goods sell at different prices and answered that buyers search only until the cost of another look exceeds the expected gain. John McCall's 1970 sequential model added the reservation threshold: accept any offer above it, reject and keep searching otherwise. Peter Diamond, Dale Mortensen and Christopher Pissarides turned this into matching theory and won the 2010 Nobel for it. All of it assumes the searcher either knows the arrival rate λ and the offer distribution F, or can treat them as fixed parameters to be plugged in.

An integrity engineer monitoring a producing field does not have that luxury. Wellhead telemetry, seismic reprocessing, pipeline pressure readings and regulatory notices do not arrive as draws from a stable, known Poisson process. Their rates shift with reservoir behaviour, sensor drift, weather, contractor schedules and political mood at the regulator. The moment you have to learn λ rather than assume it, you have left optimal stopping and entered Bayesian sequential control or a multi-armed bandit, both of which are known to be intractable in general. Reaching for Stigler and McCall to justify continuous intake in oil and gas borrows the elegance of a solved problem while quietly discarding the assumption that solved it.

You are invoking a theorem whose main achievement was tractability, in a setting where the very thing that made it tractable does not hold. That is not an application of search theory. It is a citation of search theory.

This objection is correct as stated. It should be taken seriously rather than waved off, because the discipline's engineering culture already knows what happens when arrival rates are assumed rather than measured.

Where the objection lands

Concede the mathematics first. With λ unknown, there is no Gittins index handed to you on a plate. Every integrity dashboard that claims an "optimal" alert threshold, derived once from historical failure data and left alone, is quietly assuming a stationary arrival process for the thing it is supposed to catch — corrosion events, valve failures, casing integrity loss. Reservoirs do not hold still for that assumption. A field in year twelve of production behaves differently from the same field in year two, and the rate at which genuine integrity signals arrive changes with it. Treat λ as known and fixed, and you have built a McCall searcher who thinks he is optimising when he is really just running a rule someone wrote down once and stopped updating.

So the objection is right that exact optimality is out of reach once λ has to be learned. What it gets wrong is the conclusion it wants drawn from that fact — that because the clean result doesn't survive, arrival rate is irrelevant to the design, and a well-chosen periodic snapshot will do as well as anything else. That does not follow. Not being able to solve the control problem exactly does not mean you can ignore what the problem is a function of. Every stopping rule, tractable or not, has quality that depends on how good your estimate of λ is. Intractability is a statement about how well you can do. It is not a statement about what you need to observe to do reasonably at all.

The failure this produces

Consider the actual mechanism by which this goes wrong on a producing asset. An integrity engineer is responsible for deciding whether a pressure anomaly on a subsea pipeline is noise or the leading edge of a failure. The monitoring architecture aggregates pressure telemetry on a monthly reporting cycle, because that is the cadence the reporting system was built around, and because monthly aggregation is cheap and auditable. The failure mode this system is meant to catch — a stress corrosion crack propagating to rupture — can go from detectable to catastrophic in hours, sometimes less, once the crack reaches critical length. The monitoring cadence and the failure cadence are off by roughly three orders of magnitude.

This is exactly the reservation-threshold problem stated in engineering terms rather than economic ones. The correct acceptance threshold for "this anomaly is worth an inspection" depends on how frequently genuine precursor signals arrive relative to how frequently the system samples for them. If precursor signals genuinely arrive on a monthly rhythm, monthly aggregation is a defensible design and the integrity engineer's threshold, calibrated against that rate, is doing real work. If they arrive on an hourly rhythm — which is what a rupture propagating in hours implies about the precursor window — then the monthly system is not observing the arrival process at all. It is sampling so far below the true rate that it cannot distinguish "no signal occurred" from "a signal occurred and decayed before the next sample." The engineer holding a monthly threshold has not made an error of judgement. They have made an error that was baked in before judgement was possible, because the system never gave them the arrival data needed to set the threshold correctly in the first place.

The engineer is not slow to react; the system has already decided, by its sampling cadence, that reaction speed does not matter.

What each generation of system can and cannot see

what it holdswhat it can say about arrival rate
Large Language Modela corpus frozen at some cutoffnone: arrival rate is zero by construction, so the "threshold" is whatever the corpus happened to contain when frozen
Large World Modela bounded scene, sensing live within itsomething, but only for the duration of the scene; at scene close the rate returns to zero and the threshold freezes with it
Large Universe Modelevery stream still running — wellhead telemetry, seismic, pipeline pressure, regulatory notices — held as revisable belief with provenancea measured, non-stationary rate, re-estimated as the streams themselves change behaviour

A frozen corpus of historical integrity reports, however large, cannot tell you that this field's crack-propagation dynamics have shifted since last year's data was gathered. It can only answer questions in the vocabulary of the year it was frozen. A bounded scene — one inspection campaign, one seismic survey window — captures a live arrival process for its duration, which is genuinely more than the frozen corpus offers, but it stops being informative about arrival rate the instant the campaign ends. Only a system built to keep every stream running, with provenance attached so a stale seismic reprocessing job can be discounted against a live pressure feed, is structurally capable of noticing that the true precursor window is hours rather than months.

The deadline objection, and why it doesn't rescue the snapshot

There is a second objection worth meeting directly, because integrity engineering runs on deadlines: a pipeline recertification is due by a fixed date, a regulatory filing must be submitted this quarter, a workover has to be scheduled before a rig contract lapses. When the stopping time is externally fixed, does the arrival rate of new evidence matter at all? Isn't a well-chosen snapshot, taken shortly before the deadline, just as good as continuous observation?

McCall's finite-horizon extension answers this directly, and the answer does not favour the snapshot. Under a fixed deadline, the reservation threshold still depends on how many further offers — here, further readings — are expected before the deadline arrives, and that expectation is set by the arrival rate. Near a recertification date, the correct threshold for "escalate this anomaly now rather than wait" tightens as remaining time shrinks, but by how much depends entirely on how fast genuine signals have been arriving. A snapshot taken a week before the deadline tells you what was true a week before the deadline. It gives no basis for pricing the signals that arrive in the six intervening days, which, for a crack propagating in hours, is exactly the interval that matters.

What survives

Concede the third objection too, because it is the honest one: continuous intake has a real cost. Sampling a pipeline every hour instead of every month is not free, and every extra stream — a marginal seismic reprocessing feed, a low-quality third-party sensor — carries the risk of contaminating the estimate rather than sharpening it. Stigler's founding point was that information is bought, not given away. An integrity architecture that ingests every available stream indiscriminately, without provenance to discount the unreliable ones, is not doing search theory correctly either.

But deciding which streams are worth their cost is itself a decision that requires knowing something about arrival rates and signal quality per source, and that knowledge only comes from having measured the streams in the first place. The bounded, cost-disciplined monitoring design that a mature integrity programme eventually settles on is a conclusion reached through continuous measurement, not an alternative to it. That is the narrower claim this page can actually defend: not that everything must always be watched, but that nothing can be correctly priced, thresholded, or safely aggregated at monthly cadence until someone has watched closely enough to learn what the true arrival rate is. The engineer's threshold was never wrong in isolation. It was inherited from a system that had never been asked the question.

Continue