Your AI agent has no tests

Every part of your codebase has tests except the one making the decisions. You changed a prompt last week and shipped it. You swapped the model provider to cut cost. You bumped an SDK. Did anything regress? On the cases that actually matter to a customer, you are guessing. That is the normal state of most agents in production: shipped, watched in the logs, and never tested. It is tempting to blame the technology — agents are non-deterministic, so testing feels impossible. That is the wrong conclusion, and it is expensive.

Non-deterministic is not untestable

A traditional unit test asserts an exact output: given this input, return exactly that. You cannot do that with a language model, because the same prompt can produce different words each time. But "no exact-match assertion" is not the same as "no assertion". You can still assert properties, and properties are where the real requirements live. Did the agent call the right tool, with arguments in range? Did the output stay in the contract you promised the caller — valid JSON, the fields your parser expects, nothing extra? Did it refuse the request it is supposed to refuse? Did a graded score land within tolerance across a representative set, rather than on one cherry-picked example? Non-determinism removes exact-match checks. It does not remove the ability to test behavior. The shift is from testing bytes to testing invariants.

The eval harness

Give the missing piece a name: an eval harness, and treat it like any other test project in the repository. It has three parts. First, a golden set — a fixed, versioned collection of representative inputs, deliberately including the ugly edge cases that break things in production, not just the happy path from the demo. Second, expected properties per case: not one blessed answer, but the invariants that must hold for each input. Third, a score you run automatically — on every prompt change, every model bump, every dependency update — inside CI, wired as a gate that fails the build when the number drops. Scoring can be exact checks where the output is structured, assertions on shape and tool use, and an LLM-as-judge for the genuinely fuzzy parts, with the honest caveat that the judge is itself a model you have to calibrate and pin. Without this, "we improved the prompt" is a feeling in a standup. With it, it is a number, and a regression becomes something you catch on a pull request instead of in a support ticket.

Two guarantees, not one

This lands on the question an enterprise asks before an agent goes anywhere near money, claims, or customer data: can you prove what it did, and can you show it was right? Those are two different guarantees, and teams collapse them at their peril. An immutable, anchored decision log proves what the agent did and when — a tamper-evident history that a third party can verify without trusting your database. An eval suite proves something else: that the behavior is correct, and that it stayed correct across a change. One is accountability, the other is quality. Confusing them is exactly how a pilot sails through a demo and then dies in an audit, because "here are our logs" answers a question nobody was worried about. For a high-risk system the eval suite doubles as evidence: not a promise that it works, but a reproducible measurement that it does, run on a set you can show.

None of this is glamorous. It is the same discipline that made ordinary software safe to change, applied to a component that happens to be non-deterministic. The teams that will operate agents calmly a year from now are the ones building the harness now, while the agent is still small enough to wrap in one.

When your model provider ships a silent update tonight, what in your agent breaks — and would anything tell you before your users do?