There Is No Forgetting

In 2011, with Jim Smith, I published “A Bayesian approach to event trees” in Bayesian Analysis. Among its moving parts was one I was quietly pleased with: a way for a non-stationary model to track a world that shifts underneath it. The models were conjugate — the well-behaved pairs where an update is a matter of adding to a count — and to keep old observations from tyrannising the present, I multiplied the conjugate parameters by a forgetting factor $\gamma < 1$ before each update. Yesterday’s evidence decayed geometrically. The agent stayed limber. When a regime changed, the tail of stale counts thinned fast enough that the posterior could move to meet it.

This was sensible. It was standard. It has a long and respectable pedigree — power priors, discount factors, the whole apparatus of dynamic linear models. My own Research page used to say, accurately enough, that the mechanism prefigured methods now used in adaptive agent architectures. It no longer stops there. It now says that the forgetting factor is the thing a successor project of mine deletes — and that forgetting turns out to have been content smuggled in as machinery all along. That is a large claim to make in one sentence on a page nobody reads closely, and this essay is the working behind it: the successor deletes the mechanism entirely, does better without it, and does better for a reason worth the length of an essay. The pleasure here is a specific one — being shown that a thing you were proud of was a workaround for not having had a cleaner idea. So I will report the margins honestly, because the margins are what make the claim credible rather than triumphal, and one of them is a loss.

A second mechanism is a first mechanism mistrusted

Start with the flaw stated abstractly, because it is easy to miss from inside working code.

A Bayesian agent already has a rule for changing its mind: condition on evidence. That is the only rule coherence permits — Bayes’ rule is the unique update no Dutch book can exploit, which the earlier essays argue at length. A forgetting factor is a second rule. Between two observations, with no new evidence arriving, it reaches into the posterior and moves the weights. Nothing was observed; the beliefs changed anyway.

Say it plainly: there is no forgetting. Decay, likelihood tempering, sliding windows, forgetting factors — every one of them is a second learning mechanism wearing the costume of a tuning knob. And a second mechanism is a first mechanism mistrusted. It exists precisely because the designer did not believe the first rule would do the job, and it lets the implementation disagree with itself — update one way when evidence arrives, another way when the clock ticks — which is exactly the self-inconsistency coherence forbids. Two ways of updating beliefs are a standing invitation to a bookmaker.

So what do you do when the world really does drift? You do not reach for a mechanism. You notice that “the world drifts” is a claim about the world, and claims about the world belong in the hypothesis space, where the one update rule can weigh them against the data like any other claim. If the success rate wanders, then a hypothesis that says the success rate wanders at rate $\rho$ is simply a better model than a stationary one, and conditioning will find it. The drift-rate is a latent variable to be inferred, not a coefficient to be tuned. Non-stationarity is content, not machinery.

This is the whole move, and it belongs to a language called proplang, whose agents are programs and whose prior over programs is their description length — the alphabet is the prior. An agent that must adapt to a changing world is not handed an adaptation subsystem. It is handed a grammar rich enough to state that the world changes, and the single update does the rest. The forgetting I was proud of is not disabled in that language, or deprecated, or discouraged. It is unsayable.

The forgetting-factor trap

Abstract arguments about coherence are cheap. The project made this one expensive to itself: it built the forgetter, on purpose, to see whether it would win.

The construction is a Beta tracker with a forgetting factor $\gamma$ — my 2011 mechanism, near enough — and it was built in the test file and quarantined there, never admitted into the language. Then it was run head to head against the proplang agent, which contains no forgetting anything, on two worlds. The scores are in bits of log-loss, and lower is better: fewer bits means the agent was less surprised by what happened next.

On a drifting world, the agent scores 339.8 bits against the 340.9 of the best forgetter — and “best” here means oracle-tuned, its $\gamma$ set with hindsight to the value that minimises its own loss. The agent wins. It wins carrying no decay, no discount, no sliding window, against a forgetter tuned by an oracle it does not get to consult. And when you ask the agent how it won — what its highest-posterior program actually is — it is the drift sentence, with rate $\rho = 0.1$. The drift rate was not a knob someone turned. It was a number the agent read off the data, the way it reads off everything else.

On a stationary world, the agent scores 351.1 against the best forgetter’s 350.3. The agent loses. I want to dwell on this loss, because it is the most credible thing in the experiment. First, notice which forgetter won: the best $\gamma$ on the stationary world is $\gamma = 1.0$. The forgetter wins only by declining to forget. Every $\gamma < 1$ — every actual instance of the mechanism doing the thing it exists to do — is strictly worse. On a world that does not change, forgetting is pure damage, and the tuning procedure discovers this by tuning the mechanism out of existence.

Second, the 0.8-bit gap is not a defect to be engineered away. It is a premium, and I can tell you exactly what it buys. The proplang agent walks onto the stationary world still carrying a drift hypothesis in its head — it has to, because it cannot know in advance which world it is on. The complexity prior charges rent on that unused hypothesis, and 0.8 bits is the rent. It is insurance against a change that, this time, did not come. Had the agent won on the stationary world — matched or beaten a forgetter that reduces to no-forgetting — that would not have been a triumph. It would have been evidence of a bug: a drift-carrying agent has strictly more to explain and should pay for it, and an accounting in which it doesn’t is an accounting that isn’t adding up. The honest outcome is the small loss. Both results, incidentally, sit inside a 2% margin; nobody is being crushed here. The point is not the size of the wins. The point is that the agent with no mechanism is competitive with an oracle-tuned mechanism on the world the mechanism was designed for, and beats it on the world where change is real — and that it pays a legible, correct premium for the privilege.

Then the forgetting factor was deleted. Not disabled behind a flag. Deleted, out of the test file, and never present in the language to begin with.

Sayability is the whole design demand

So if the language does not handle change, and change nonetheless gets handled, where does the handling live? In the ordinary machinery of prediction and posterior, which is easiest to watch on a world built to break an agent’s confidence.

Take a Bernoulli whose success rate flips: $\theta = 0.9$ until $t = 60$, then $\theta = 0.1$. Through the interval $[30, 60)$ the agent is confident — it has watched a long run of successes and settled. Then the world turns over. Its predictions start failing, and here is the mechanism, with no monitor in it: a hypothesis that keeps predicting 0.9 keeps being wrong, its marginal likelihood drops on every miss, and the posterior mass drains out of it. The one-step predictive collapses back toward 0.5 — the agent is no longer sure what comes next — and a meta-entropy over its hypotheses, which had tightened to 3.15 bits at $t = 59$, re-disperses to a peak of 3.95 bits. The agent has become uncertain again, and it became uncertain because the arithmetic dispersed it, not because anything detected a change and fired.

What the agent does with that uncertainty is the part I find most satisfying. For several ticks after the flip, its chosen action becomes “consult.” Not because a humility routine woke up and told it to ask for help — there is no such routine. “Consult” is an ordinary action sitting in the action space beside all the others, and the world attaches a utility of 0.35 to it. While the agent was confident, acting directly beat consulting. Once the posterior dispersed, the expected utility of gathering more evidence rose above the expected utility of acting on beliefs it no longer trusts, and the same argmax that always runs picked “consult.” The consultative turn is Bayes flowing through an honest utility. There is no humility subsystem, because a confidently-acting agent that meets a changed world is made consultative by the mathematics that disperses it.

And then it recovers. By $t = 76$ the agent is confidently predicting the new regime, and its highest-posterior program is the exact change-point sentence — the world changed, at $t = 60$ — held at posterior 0.64. It did not detect the change point. It inferred it, as content, the way it infers everything.

The project does not ask you to take its restraint on faith. The same test then greps the language’s own source — excluding comments and string literals, so no cheating on technicalities — for ten words: detect, forget, window, decay, sliding, reset, trigger, temper, anneal, throttle. Every one of them is absent. There is no adaptation vocabulary in the implementation, because adaptation is not implemented. It is inferred.

This is why the design demand reduces to one word: expressiveness. Programs that carry temporal structure — “the world drifts at rate $\rho$,” “there is a change-point at an unknown time” — out-predict stationary programs exactly when change is real, and lose to them under the complexity prior when it is not. The prior does the model selection. The designer does nothing. So the grammar must be exactly rich enough that non-stationarity is a sentence you can write — not richer, because a bespoke change-point subsystem would be content smuggled in as machinery, and not poorer, because a grammar that cannot say “$\rho$ drifts” cannot adapt, and no runtime cleverness will rescue it after the fact. The sharpest way I know to put it: you were asked to make change sayable, not handled. If your language handles change, you have failed. If it can merely say it, you have succeeded, and the agent adapts.

What holds it up, and where it stops

The load-bearing terminal is a modest one called rw — a latent random walk, an emission that drifts. It is what lets a program say the success rate wanders rather than merely say it is fixed. The deletion audit that vets every terminal in the language put rw on the block too: remove it and drift becomes literally unsayable, and on a drifting world the crippled language scores 211 bits against the intact language’s 207. (That is a smaller, separate experiment from the forgetter trap above — different world, different numbers; do not read the 211/207 pair as if it were the 339.8/340.9 one.) Four bits is not a landslide. But it is the whole difference between a language that can state that the world changes and one that cannot, and it is paid by the presence or absence of a single terminal. That is what it costs to be able to say the thing.

I will not pretend the frontier is clean. The agent’s hypotheses are programs it enumerates, and enumeration has a depth. Temporal structures too deep to reach are not merely improbable to the agent; they are assigned probability zero, and an agent that is certain a possibility cannot occur will never be argued out of that certainty by evidence — a Cromwell’s-rule violation sitting at the edge of what the search can see. I would rather name that frontier as a vice than dress it as a feature. Time-varying preference inference — an agent whose utilities, not just its beliefs, drift — is barely explored here at all. But notice what these are and what they are not. They are limits on reach: on which sentences the agent can get to. They are not an argument for reaching back into the posterior between observations and moving the weights by hand. The remedy for a hypothesis you cannot yet say is to make it sayable. It is never to go back to forgetting.

Which returns me to that sentence on my Research page. The claim that the 2011 mechanism prefigured today’s adaptive agents was true and incomplete, and the amendment now standing there is the one this essay has been earning: it prefigured them the way a workaround prefigures the fix it was standing in for. The forgetting factor was a real answer to a real problem, and it was wrong in a way I could not have seen from inside it: it treated a fact about the world as a property of the machinery. The cleaner idea does not track the world better by forgetting more artfully. It refuses to forget at all, says out loud that the world changes, and lets one update rule believe it. The most durable thing you can build into a language is not a mechanism. It is the ability to state the thing the mechanism was quietly assuming — and then, as the companion essay has it, to arrange matters so the assumption cannot be smuggled back in. What you can merely say, you can get right. What you build into the machinery, you can only hope you built correctly.