All series

Credence

7 posts

Credence treated which tool to query as a decision problem under uncertainty rather than a language-generation problem. These posts follow it from the first claim to the last measurement.

They are not all in agreement with each other, and that is deliberate. Several of the early posts assert a margin — the Bayesian agent beating an LLM agent by roughly 120 points — that a fairer benchmark later reversed. Rather than quietly edit those posts, each carries a correction note pointing here. If you read only one, read the retrospective, which sets out what the benchmark was actually measuring, what survived, and what did not.

  1. Agentic AI Is Neither Intelligent Nor an Agent

    I built a Bayesian agent and set it against LangChain on a tool-use benchmark. LangChain got more answers right and still lost — by 120 points.

    8 min read

  2. How Decision Theory Cuts Your AI Agent's API Bill in Half

    Most LLM agents use tools like a tourist uses a phrasebook. A few lines of probability theory can fix it.

    11 min read

  3. Sixty-Two Percent Correct and Winning by a Hundred and Twenty Points

    A Bayesian decision-theoretic agent scores lower on accuracy than every LLM variant it competes against — and beats the best of them by 120 points. The explanation requires thinking about something that LLM benchmarks …

    8 min read

  4. The Prompting Gradient

    Each prompting technique helps. Reasoning traces, strategy guidance, cross-question history — each one improves accuracy and score. None of them closes the gap with a Bayesian agent that does not use language at all. The …

    7 min read

  5. The Agent That Invents Its Own Rules

    Most agents are given a fixed set of decision rules. Credence's second tier generates candidate rules from sensor features, scores them by complexity, and lets the posterior decide which structures are worth keeping. …

    7 min read

  6. Ninety-Six Percent Cheaper and Slightly Better

    Credence-proxy sits between an agent and its LLM providers, learns which model is good for which category, and routes accordingly. On an OpenClaw benchmark it cut cost by 96% and latency by 52% while raising quality by …

    6 min read

  7. The Ablation That Beat the Agent

    I said a Credence paper was going to arXiv. It never went. Each of its three framings was killed by a specific measurement: frontier-LLM dominance, then a one-line greedy ablation that beat the full Bayesian agent under …

    9 min read