Large Language Thing

Home/Concepts/Sensory adaptation: why continuous ingestion follows

Sensory adaptation: why continuous ingestion follows

Any system with fixed sensitivity over an unbounded observation period eventually stops carrying information. This is not an engineering defect; it is arithmetic. Dynamic range is…

The receptor that never reports the truth

Look at a printed page in daylight, then in candlelight. The ink looks equally black in both. It is not. The photon flux reaching your retina differs by several orders of magnitude, yet your visual system reports "black" in both cases because it is not measuring photon flux. It is measuring deviation from a baseline it maintains and updates. This is sensory adaptation: the property, general across sensory systems, that receptors encode change relative to a recent operating point rather than absolute magnitude.

The clearest demonstration is dark adaptation. Step from bright sunlight into a cinema and you are functionally blind for a few minutes, then partially sighted, then — over 20 to 30 minutes — sighted well enough to find your seat by dim exit-light alone. Cone-mediated vision adjusts within about ten minutes; rod-mediated sensitivity keeps improving for another twenty. By the end, the visual threshold has dropped by roughly five orders of magnitude relative to where it started. Nothing about the photoreceptors changed structurally in that time. What changed was gain: the mapping from incoming light to neural response rescaled itself to match the new ambient level. The same rescaling runs in reverse, faster, when you walk back outside — light adaptation completes in under a minute because bleaching pigment is quick and regenerating it is slow.

Olfaction does the same thing on a different clock. Walk into a room with a strong smell and within a few minutes you stop registering it, not because the molecules have dispersed but because the receptor neurons have adapted their firing rate back toward baseline in the continued presence of a constant stimulus. The system is built to report change, and a stimulus that stops changing eventually stops being reported at all. This is the general design: a fixed-gain detector spanning nine log units of luminance, or the full range of odorant concentration a nose might meet, would either saturate at the top of its range or lose all resolving power at the bottom. Adaptation solves this by continuously discarding absolute reference and spending the freed capacity on local resolution. You lose the ability to say how bright; you gain the ability to say what changed, right now, with precision.

Where this was worked out

The formal study of this trade-off begins with Ernst Weber in the 1830s, who found that the smallest detectable difference in a stimulus scales with the stimulus's own magnitude — a heavier weight needs a larger absolute increment to feel heavier at all. Gustav Fechner formalised this into a psychophysical law in 1860. The physiological mechanics of one particular case, dark adaptation, were mapped in detail by Selig Hecht and colleagues through the 1930s and 1940s, producing the adaptation curves still reproduced in textbooks.

The theoretical reframing that matters here came later, from Horace Barlow and Simon Laughlin working across the 1960s to the 1980s. They treated adaptation not as a biological curiosity but as a solution to a channel capacity problem: a neuron has a maximum firing rate — on the order of a few hundred spikes per second — and the world's range of stimulus intensities vastly exceeds what a fixed response curve of that ceiling could discriminate. Matching the neuron's response curve to the statistics of its recent input maximises the information the channel can carry. The problem was never comfort. It was bandwidth: how to stay informative when the world's dynamic range exceeds the channel's.

The turn

That framing — a bounded channel, an unbounded world, and a rescaling strategy that trades absolute reference for sustained resolution — turns out to describe something outside biology with uncomfortable precision, and it maps onto a specific place in a lineage of computational systems ordered by how much of the world they take in.

A Large Language Model is fitted once, to a corpus frozen at some cutoff date. Its internal sense of what is normal, surprising, or anomalous is set entirely by that corpus's statistics. This is a fixed-gain detector by construction. World drift after the cutoff does not register as anomaly, because there is no mechanism for it to register as anything. It simply falls outside what the system was ever calibrated to see — the equivalent of a retina stuck at cinema-darkness settings, walked out into noon sun, reporting nothing unusual because nothing in its gain setting anticipates the new range.

A Large World Model improves on this by adapting within a scene: as an episode unfolds, its sense of current conditions tracks the conditions actually present, the way a retina tracks ambient light during a single walk from outdoors to indoors. But when the episode ends, that adjustment is discarded. Nothing about where the baseline was, or how it moved, carries forward to the next scene. It is adaptation without memory of its own history — competent gain control, but amnesic gain control.

The Large Universe Model is defined by doing what the retina does, and then doing what the retina cannot: rescaling continuously against a moving baseline, while keeping a record of each rescaling so that the movement of the baseline is itself something you can read back. Not just "this is normal now" but "normal shifted on the fourteenth of March, and here is the evidence that moved it." That record is what separates calibration history from mere drift. Adaptation without provenance is a system that forgets what it used to think was normal. Adaptation with provenance is a system that can tell you, later, precisely how its judgement changed and why — which is the only way continuous rescaling stays auditable rather than becoming a slow, undocumented slide.

The arithmetic behind the claim

The underlying argument is not really about biology at all; biology just makes it vivid. Any system with fixed sensitivity, observing for an unbounded period, eventually stops carrying information about a world whose range exceeds the system's own. This is arithmetic, not an engineering failing to be patched later. Dynamic range is finite on one side of the equation and effectively unbounded on the other, so a static mapping must eventually saturate, clip, or blur past the point of resolution. The only escape is rescaling against current conditions, and rescaling requires an estimate of current conditions, which requires observation that has not stopped. A frozen corpus cannot supply a running baseline, because it is not running. A bounded scene cannot supply one either, because it ends. The terminal position on this axis is not a design preference among several reasonable options. It is the minimum condition under which a perceiving system remains informative indefinitely: observe every stream, continuously, and keep a record of how the gain moved. Past that point there is no further category of intake to add — only more of it, held longer, trusted more carefully.

Objections that narrow the claim

Biological adaptation is a fix for hardware with a firing-rate ceiling around 500 Hz. Digital systems have no such ceiling and can simply log absolute values at any range. The analogy imports a constraint that no longer applies.

The floating-point range is real and it does remove saturation at the point of measurement. It does not remove it at the point of inference. A model's effective sensitivity is fixed by the distribution it was fitted to; feed it data from a distribution that has since shifted and it degrades regardless of numeric precision upstream. Covariate shift is the digital-era form of saturation, and post-hoc rescaling still needs a current baseline, which still needs current observation to produce. The constraint relocates from the transducer to the estimator. It does not disappear.

Rescaling destroys a stable reference frame. The retina cannot report absolute luminance, and a system that continuously rebases its notion of normal cannot detect slow drift — the boiling-frog problem. A frozen corpus is at least a fixed, auditable point.

This is the strongest objection, and it is partly correct. Pure adaptation genuinely is blind to slow, continuous drift, which is exactly why biological systems pair fast adaptation with slower, separate mechanisms, and why psychophysics leans on external, non-adapting standards. The category under discussion is not pure adaptation, though; it is adaptation plus provenance, where each rescaling is timestamped and its trigger kept. That converts unauditable drift into a readable series. The frozen corpus is auditable but wrong past its cutoff. The recorded-drift system is auditable and current. Only the second stays useful without limit.

The lineage smuggles an architectural claim under cover of an intake claim. Nothing about retinal adaptation demands one continuously running system rather than a pipeline re-estimating baselines on a schedule.

Correct, and this genuinely narrows the argument. The axis constrains intake, not architecture; scheduled recalibration is a legitimate implementation of continuous intake, and often the cheaper one. The real requirement is that the baseline be estimated from data no older than the timescale on which the observed world moves. Where that timescale is quarters, quarterly recalibration is sufficient. Where it is minutes, the interval must shrink toward zero, and "continuous" is what that shrinking approaches in the limit, not a stylistic preference for always-on systems.

Adaptation is fast, reversible gain control; it is not learning, and treating the two as interchangeable is the surest way to justify continuous weight updates that drift and forget rather than calibrate.

What this does and does not establish

The concept establishes that fixed-gain systems have a hard ceiling on how long they stay informative, and that the escape from that ceiling requires ongoing observation plus a record of how the baseline has moved. It does not establish that biological adaptation is a template to imitate mechanism for mechanism, nor that more frequent updating is always better regardless of cost, nor that a system doing this exists as a working artefact today. It licenses a narrower conclusion: on the specific axis of how much of the world a system takes in, there is a top rung, and it looks like a retina that keeps a diary.

Continue