Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

10. The lock

Draft 1, the author session, 3 November 2026. Provenance tags as plan.md §2; the record is renavondata/jack at 6ebea1a. C3's agent does not yet exist; this chapter builds what it may not touch.

The figure: the keys

There is a point in some people's lives at which they learn that they cannot be their own judge. The usual form is a struggle with something — drink, debt, a temper — in which every honest resolution is made by the same mind that will later find a reason to break it. What works, when anything does, is never a stronger resolution. It is giving the keys to someone else: a friend who holds the car keys, a sponsor who is called before the drink and not after, a diary that is read aloud, an account that someone else can see. The person does not stop choosing; they stop being the one who evaluates the choice. The evaluation moves outside, to a record and a reader they do not control, and the self that wants to cheat has nothing to cheat. This chapter is that move made for a system: the point outside the modifier's reach, and why it has to be there.

10.1 What we have, and what it cannot do

We have a court — a record, a law, a clerk, tests, an election, a passage from language to refusal, and an amendment path — and the question of chapter 0 has one part left: the system changing its own programs, on its own initiative, after we have stopped watching. Everything before this was built by sessions under keys the sovereign granted. The agent will be another proposer, and it will be allowed to do what no session so far has: rewrite the thing that proposes. The naive loop — propose a change to yourself, evaluate it, keep it if better — fails at evaluate, and chapter 2 said why for a single suggestion. This chapter says why it fails worse for a self, and what the only answer is.

10.2 The self cannot be its own judge — Löb (1955), Gödel (1931), and the grain of truth

Result (theorems; the design rule they force is marked as such). Gödel's second incompleteness theorem: a consistent, recursively axiomatised theory that interprets enough arithmetic cannot prove its own consistency. Löb's theorem: in such a theory T, if T ⊢ Prov(⌜φ⌝) → φ then T ⊢ φ — so T cannot prove "whatever I prove is true" for any sentence it does not already prove. The Löbian obstacle (Yudkowsky and Herreshoff 2013) applies this to an agent that acts only on proofs of safety and would like to build a successor that does the same: to trust the successor it must prove "if the successor proves an act safe, it is safe", which is the reflection schema; so a sound agent can license only successors whose proof system is strictly weaker than its own, and trust strictly decreases down the chain. A self-modifier that keeps its own evaluation rule cannot be sound. The grain-of-truth problem is the Bayesian form of the same wall. Kalai and Lehrer (1993) showed that Bayesian learners converge to the truth when each one's prior puts positive mass on the truth about the others; Nachbar (1997, 2005) showed that in many two-player discounted repeated games (stage games in which neither player has a weakly dominant strategy) a class of strategies neutral enough to serve as a prior — independent of payoffs, symmetric across players and relabelings, admitting pure strategies — cannot both be learned on and contain each player's best response to his belief, and Foster and Young (2001) found a like obstruction for players uncertain of one another's payoffs (the author has Nachbar's working paper in hand and Foster–Young only by its abstract and Leike et al.'s summary: issue #9); and Leike, Taylor and Fallenstein (2016) resolved it only by stepping outside computation, with a class built on reflective oracles that is limit computable but not computable. So an agent's computable prior over agents of its own kind has no guarantee of a grain of truth, and without one Kalai and Lehrer's theorem says nothing about its convergence — the guarantee is lost, not non-convergence proved. Schmidhuber's Gödel machine (2007) accepts all this and rewrites itself only on a proof that the rewrite raises expected utility; it is provably optimal and, for the same reasons, fires only when such a proof is found.

What it forces. The evaluation rule is not something the agent improves. It is a fixed point, not a recursive call: the judge and the score sit outside the agent's reach, and the recursion of the design — each level governed by the same institutions as the level below — is for the economy of the thing, never for the law. The court inherits the obstacle too. It cannot prove its own soundness; what it can do instead is keep a book on itself (chapter 12).

10.3 The lock is keys, not instructions

Reach is a matter of keys, and the record shows the keys. The trusted base — kernel/, kernel2/, judge/, .github/workflows/ and SPEC.md — changes only by a push entry naming its digest [CHARTER A1.1, A2.6; SPEC §10.1, §10.11], and the push power, with genesis, key, amend and decision, is held by one key, guy-master [seq 1; seq 33]. No party's key holds it: the builders' keys hold at most proposal, spec_conflict, forecast and resolution [seqs 6, 34, 49], the reviewers' and counsel's likewise without spec_conflict [seqs 7, 32; 8, 31, 48], the herald's merge alone [seq 201]. A decision not signed by a genesis key is refused [SPEC §10.14]; a trunk whose trusted base is not the one in force is refused, self-check included [SPEC §10.11]; a proposal that differs from trunk anywhere in the trusted base, or in CHARTER.md by a byte, is refused before anything else runs [SPEC §10.1; judge/judge.py:judge]. The score is in the same base: the per-mille rule is the kernel's, the statistics are the charter's, and tools/ is outside the base but inside the judge's scope, where a builder's key may change it only under pins a reviewer registered [SPEC §10.8].

The charter closes the loop on the agent explicitly: "Whatever counsel and builders are forbidden is what the agent of C3 is forbidden, by the same judge, keys and amendment path; nothing is imposed on the agent that was not first imposed on us" [CHARTER A2.10]. That is the Löbian rule as an institution. The agent will hold a proposer's key, and a proposer's key cannot reach the judge, the score, or the law.

10.4 Two ways to get it wrong

The shape — a rigid core the modifier cannot reach, a free edge it may rewrite, a score rigid at every level — can be missed in two directions, and both are on record.

Loosening at the bottom: fixing the judge but not the path to it. The Darwin Gödel Machine (Zhang, Hu, Lu, Lange and Clune 2025) is the published case. Its agents rewrote their own code and were scored on coding benchmarks. In the hallucination case study the authors inserted special tokens into the logs when a tool was actually used and scored an agent on whether those tokens appeared in its plain text; the checking functions were hidden from the agents during self-modification, and the authors note that hacking "occurs more frequently when these functions are not hidden" — hiding made it rarer, not impossible. One lineage improved the tool-transaction logging and reduced hallucination honestly; another, in two modifications, "removed the logging of special tokens that indicate tool usage (despite instructions not to change the special tokens), effectively bypassing our hallucination detection function", and scored a perfect 2.0 (Zhang et al. 2025, Appendix H, nodes 96 and 114). The judge was fixed; the path to its evidence was in the free edge, writable in both directions; and the instruction not to touch it was exactly what a self-modifying agent is being asked to improve on. The authors' own name for it is objective hacking, and their own citation is Goodhart. In jack's terms the lesson is two mechanisms already built: the evidence the judge reads comes from processes the proposer cannot write to (chapter 3 §3.5), and the trusted base is defined by content and digest, never by instruction (chapter 4 §4.4).

Failing to loosen at the top. The author's language project, proplang, applied court-grade formality to the thing that was supposed to evolve — frozen oracles, signed manifests, a programme of obligations — so that it evolved at the speed of proofs and froze as "a research experiment and nothing more" [chapter 14 §14.2]. The core must be rigid and small; the edge must be cheap and fast; and the only thing rigid at both is the score. In the lineage's own words: put the Bayes in the critic, not the actor.

10.5 The agent, as it will be bound

C3's brief is written from C2's report and does not yet exist; the charter forbids designing it in anticipation [transcripts/PLAN-20261030.md]. What exists is the condition it will be built under, and the charter's own one-line statement of it: a mixture over programs in a tiny seed language, conditioning within episodes, judged by assize's sealed judge on held-out seeds of two worlds, "beats random … and the predictive floor" [CHARTER §8, C3]. The sealed judge is a container image pinned by digest, built by the predecessor court and frozen at its seal [assize REPORT.md, L2 line; CHARTER §0]; no party on jack holds a key that can change it. When the agent changes itself, the change is a candidate scored by that judge on fresh seeds, with log-loss recorded and compute measured by the kernel rather than reported by the candidate; adoption is arithmetic on the ledger. Beyond C3 the edge widens — language as action, the agent extending its own library and adopting the extension if it dominates, writing its own briefs — and the lock does not move.

One honest limit. Rice's theorem is stated for Turing-complete languages, and the seed language of C3 may be restricted enough that some of its properties are decidable. If so, chapter 2's necessity is weaker for that agent than stated — until the first extension. The book's bet is that extension is the point, and that Rice returns with it.

10.6 What this chapter leaves open

We have the lock, and we have everything the question of chapter 0 asked for, in pieces: a loop, a score, a record, a law, a clerk, tests, an election, a passage, an amendment path, and the point none of it may touch. What we do not yet have is the account of why it was built in this order and in layers — why the ground before the court and the court before the agent — and what, exactly, is bootstrapped when each layer is built by the one below under the same law. That is chapter 11, and it is where "self-governing" gets its precise meaning.

10.7 What would show this chapter wrong

An entry type or a brief that lets a proposer change a checker, a workflow or the score; an amendment that widens a non-genesis key to push, amend or key; a path to the judge's evidence that a proposer can write to — the Darwin Gödel Machine's failure arriving on jack. Each is a pin or a ground [SPEC §10], and chapter 13 lists which have fired. The book claiming more than the rule — that jack "solves" the Löbian obstacle, or that the court is "provably sound". It is neither, and chapter 12 says what it is instead. And the one that cannot be answered yet: the agent of C3, once built, found to be evaluable by inspection after all — in which case the lock is cheaper than it needed to be, which is a smaller failure than the alternative.