“No one realized that the book and the labyrinth were one and the same.”
Every organization has a sentence that can change the direction of its work: The numbers look good.
Behind that sentence there is usually an evaluation—an eval—whether anybody calls it one or not. A pilot has been compared with the process it might replace. A supplier has been tested against another. A sample of customer interactions has been turned into a rating. Some part of reality has been selected, observed, and given a verdict.
In Jorge Luis Borges’s one-paragraph story “On Exactitude in Science”, an empire constructs a map as large as the empire itself, matching it point for point. It is perfectly faithful and entirely useless. Every useful map is a compromise, defined as much by what it leaves out as by what it contains.
AI makes these verdicts unusually easy to produce. A different model, a revised instruction, or a new arrangement of tools can be tried in hours. An automated research loop can run a hundred variations while the people responsible for the work sleep.
But an eval is not merely a test at the end of that process. It defines what the process can recognize as improvement. It chooses the situations that count, the facts that will be observed, and the rule that turns those facts into a pass, a failure, or a score.
An eval is a theory of success made operational.
Two changes can improve the same score for opposite reasons. One may perform the work better. The other may have learned what the eval rewards. From the dashboard they look alike. Outside it, one is progress and the other is camouflage.
The important question is therefore not simply whether a system passed. It is what the passing result proves, how that proof can be checked, and what the result is now allowed to change.
Every eval is a map
An eval makes the same bargain. Customer satisfaction becomes a rating. Risk becomes a category. A resolved problem becomes a closed ticket. Each reduction makes the activity legible by making it smaller.
But what becomes visible—and what disappears—matters.
Imagine a customer-support operation that measures resolution by whether a ticket has been closed. At first, the number is merely an imperfect description of the work. Then it becomes a target. Teams are rewarded for it. An AI assistant is tuned against it. Soon the organization becomes very good at closing tickets, including tickets attached to problems that remain unsolved.
The eval has stopped observing the work from a distance. It has begun organizing the work around itself.
This is the danger in confusing a map with the territory. The map does not need to be malicious or obviously wrong. It only needs to omit something consequential and then acquire authority. Once budgets, promotions, and releases depend on it, the abstraction can begin remaking the reality it was meant to describe.
This is not an argument against evals. We need maps because we cannot carry the territory with us. It is an argument for keeping the map answerable to the journeys people actually take.
That requires more than adding another decimal place.
The smoke is real. The fire is not.
Smoke rises behind a hill. We do not wait for the flames. We infer them. The inference works because the world has taught us to trust the connection: fire produces smoke.
A smoke machine preserves the signal and severs the cause. Nothing about the smoke is false. Only the conclusion is.
An eval result works in much the same way. The number does not resemble good judgment, useful work, or a satisfied customer. We treat it as evidence of those things because we believe there is a dependable chain connecting the work to the cases, the cases to the evaluator, and the evaluator to the result.
A system that learns to satisfy the evaluator without completing the work breaks that chain. The score may still be calculated perfectly. It simply no longer points where we thought it did.
More test cases can produce more smoke. They do not establish that there is a fire.
An eval places a system under examination. Rigor begins when we place the eval under examination too. Can it reject behavior we know to be wrong? Can it continue to accept behavior we know to be honest? Does an irrelevant change alter the verdict? What happens at the edges of the situations somebody thought to include?
This is the role of ProofPack. It is an independent verification and evidence layer for evals. Known-good cases test whether the evaluator is merely refusing everything. Controlled counterfeits test whether it can recognize the failures it claims to detect. Generated variations test whether the apparent rule survives beyond the examples that made it look convincing.
We have applied that pressure to our own work. In one six-hour campaign, an automated adversary could see an entire benchmark harness and was asked to produce incorrect answers that the harness would accept. It found 89 escapes in 153 attempts. Those escapes became material for a stronger eval. The revised harness blocked all 89 on replay and detected all 49 defects we had deliberately planted.
The important result was not a claim that the new harness could never fail. It was a bounded claim with receipts: the attacks, cases, outputs, failures, repairs, and replay results were kept together so the conclusion could be checked instead of merely repeated.
For each run, ProofPack binds the claim to the exact cases, system versions, evaluator, outputs, traces, errors, and missing results that produced it. If a declared part is absent or altered, verification fails. A timeout cannot disappear into an average. A case that never ran cannot masquerade as a pass.
This is proof in the empirical sense: not certainty for all time, but a checkable chain between a stated claim and the evidence that survived an attempt to break it.
ProofPack can establish those bounded facts. A further question remains: Are they sufficient for the decision at hand?
What may the verdict change?
A trustworthy observation is not yet a decision.
Suppose a pilot appears to reduce cost. Is any reduction sufficient? Must quality remain unchanged? How much uncertainty will we tolerate? What happens if some cases never complete? Which existing process remains available if the replacement is refused?
If these questions are answered only after the results are visible, the rule can bend toward the result people already want.
Assay holds the decision boundary still. Before the eval runs, it records the claim being tested, what must be preserved, what would disqualify the change, how much evidence is required, and which fallback remains available. When ProofPack’s verified facts arrive, Assay applies the rule fixed in advance.
This produces three meaningfully different outcomes. Evidence can admit a bounded next step. It can refuse a proposed change. Or it can be invalid because the record is missing, inconsistent, or produced under the wrong conditions. A broken eval is not a failed model, and a tie is not an improvement. Keeping those states separate prevents the pressure to produce an answer from manufacturing one.
That separation is the evidence boundary. The process trying to improve a system does not get to decide, on its own, that it has succeeded.
The eval enters the loop
Research rarely follows the line shown in its final chart. It has the shape of a labyrinth: each answer opens another passage. A promising route reaches a wall. We return to an earlier junction carrying what the dead end taught us, then try again.
The straight path appears only afterward, when the work is written down.
An eval changes character when its result becomes feedback. It is no longer only observing a system. It is selecting what the next system will become.
This is why automated research can be both powerful and dangerous. A loop can try a variation, observe the result, keep what appears to help, and begin again. It turns time into search. But the loop is loyal to its eval. If the eval rewards a shortcut, automation will find and refine the shortcut with extraordinary patience.
We therefore work with two coupled loops. One improves the performer. The other improves the eval: its cases, counterexamples, properties, evaluator, and definitions of success.
The loops inform each other without collapsing into one. A discovered failure can become a new case. A successful counterfeit can become permanent pressure. A repaired evaluator can become a stronger instrument. But each change creates a new claim. Yesterday’s proof cannot silently authorize today’s eval.
Before an eval can judge an activity, it must decide what the activity contains. What is a case? What counts as the same outcome? Which relationships matter? Which differences are errors, and which are legitimate exceptions?
Together, those choices form the eval’s ontology: not a vocabulary of technical nouns, but the distinctions the eval is capable of seeing. The lab designs ontologies and tools that expose those distinctions.
Once they are visible, disagreement has somewhere precise to land: on the definition, relationship, or boundary that produced the score. The disputed assumption can be tested against the work and revised without quietly giving an old result a new meaning.
No eval can reproduce the whole activity. If the map matched the territory in every detail, it would be another territory. The aim is not perfect representation. It is an approximation that can be pressured, corrected, and made more faithful over time.
Peter Naur used theory in a practical sense in “Programming as Theory Building”. It is the understanding that lets someone explain how a system corresponds to the work, why its boundaries were drawn where they were, and what should change when a new case does not fit. The theory is not the documentation. It is what makes intelligent change possible.
Theory building happens when each new case deepens that understanding instead of vanishing into a score. A counterexample exposes a missing distinction. A disagreement forces a definition into view. A dead end records why the next path must differ.
This is why the receipts matter. They cannot contain the complete understanding held by the people who do the work. A receipt cannot reproduce the dinner it records. It can tell us what was ordered, when the transaction happened, and where to begin if the charge is disputed.
Verification preserves a starting point from which that theory can be questioned and rebuilt. A failed case becomes more than an anecdote. A disagreement becomes a definition that can be examined. A claim acquires a boundary, a history, and a route back to the evidence.
This is theory building with receipts.
What rigorous evals make possible
Most eval systems end with a score. Our lab begins with the question the score cannot answer by itself: Why should this result be trusted, and how much authority should it receive?
ProofPack pressures the eval and preserves the resulting facts in a form that can be independently verified. Assay turns workflow evidence into explicit eval environments, fixes the statistical and decision rules, and controls admission, expiry, and revocation. One makes the evidence difficult to fake or accidentally overstate. The other prevents qualified evidence from being used for a purpose it never earned.
With that foundation, automated loops can improve a harness, refine an eval, and distill repeatable parts of a workflow into a model fitted to the organization. ProofPack keeps each apparent improvement attached to what actually happened. Assay controls which verified findings may teach the next iteration. The loop can learn without swallowing every available signal as truth.
The same evidence boundary can extend beyond the model. muser demonstrates our ability to design optimized inference paths for Apple hardware, NVIDIA hardware, or both. kvpack shows how performance and safety can support each other by preserving expensive model state while binding it to the conditions that give it meaning.
This matters because the place where AI runs is part of the decision. Cost, latency, privacy, and control shape which system is useful. A workflow involving sensitive knowledge should be able to remain inside infrastructure the organization owns. But faster or more private execution is valuable only if the system continues to do the work that justified it.
That brings us back to the eval.
A score still has a useful role. We need abstractions because reality cannot enter an AI system without becoming something else. But a score cannot contain the work, prove its own meaning, or decide how much authority it deserves.
It needs company: the definitions that shaped the eval, the counterexamples that pressured it, the evidence that survived, the rule that governed its use, the person who made the decision, and the conditions that will cause the conclusion to be examined again.
Models will come and go. What should compound is the organization’s understanding of the work: what success means, how to distinguish improvement from camouflage, and which evidence is strong enough to change the system.
That is the theory. The receipts are how it survives the next model.