Every agent project reaches the same dangerous moment: the demo goes well. The agent reads the invoice, finds the matching order, drafts the email, and the room is impressed. Budgets loosen. And then the project enters the phase where most agent initiatives quietly die — the gap between working in a demo and working in production.
The gap exists because demos are rigged, usually without anyone meaning to rig them. Three comfortable conditions hold in every demo and none of them survive contact with real operations.
The inputs were chosen. You picked examples you knew would land. Production sends whatever arrives: the scanned PDF at a slight angle, the email with the reference number in the subject line instead of the body, the supplier who abbreviates their own company name three different ways.
Someone was watching. In a demo, a mistake gets caught instantly and laughed off. In production the agent runs unattended, and an unwatched error does not announce itself — it compounds quietly in a log until a customer, an auditor, or a bank statement finds it for you.
The session was short. Demos are five minutes long. Production agents run for months, accumulating state, filling context windows, and hitting every anniversary of every edge case your business generates.
Closing the gap is not a modelling problem. Most of it is the kind of unfashionable software engineering that never makes it into a keynote. Here is what it actually consists of.
The engineering that survives contact
Validate before the model sees anything. Define exactly what inputs the agent accepts and normalise them in plain code — formats, typos, case, abbreviations — before any model call. Anything outside the definition gets rejected with a clear error, not heroically improvised around. Half of "hallucinations" in production are really malformed inputs the system should never have accepted.
Fail loudly, never silently. For every step, decide in advance what happens when the agent cannot complete it: return a partial result clearly marked as partial, ask a human a specific question, or stop. The one unacceptable behaviour is producing confident output when something upstream went wrong. An agent that says "I couldn't do this, here's why" is a colleague; one that guesses is a liability.
Assume every dependency fails. APIs go down, databases slow, rate limits bite. Every external call needs a timeout, and retry logic that distinguishes transient failures (back off and try again) from permanent ones (stop and escalate). This is decades-old distributed systems practice; agents do not get an exemption from it.
Checkpoint the work. Long workflows need natural save points — what was received, what has been decided, what remains — so a crash means resuming, not restarting. An agent that loses forty minutes of multi-system work to a timeout will erode trust faster than any accuracy number can rebuild it.
Put hard limits on everything. Maximum steps per task, tokens per session, API calls per minute, spend per day. An agent in a loop without limits is a cost incident waiting for a weekend. Alert when usage approaches a threshold, not when finance notices.
Log every decision with its context. For each action: what the agent saw, what it retrieved, what it chose, and what happened. Structured, queryable, kept. This is what makes "why did the agent do that?" a five-minute lookup instead of an unanswerable question — and it is the difference between an agent an auditor will accept and one they will shut down.
Notice what this list is: input validation, error handling, retries, checkpoints, quotas, logging. Ordinary software engineering, applied without exemption to a system that happens to contain a model.
Evals: the part everyone skips
Teams handle evaluation in one of two bad ways: manual testing and vibes, or an evaluation platform so elaborate it drifts out of sync with the actual system within a month. Both produce the same result — nobody actually knows whether the agent is safe to trust further.
The workable middle is smaller than people expect:
- Build a golden dataset from real cases. A hundred or so real examples, each with the input, the correct outcome, and a note on why it is correct. For a tightly scoped first agent — which is the only kind a first agent should be — that is enough. Weight it towards the cases where being wrong is expensive.
- Score the path, not just the answer. A human resolves an exception in steps: gather, compare, decide, act. Check the agent against the steps, not only the final output, or you will ship an agent that gets right answers for wrong reasons until the day it doesn't.
- Never collapse quality into one number. If 90% of cases are easy and 10% are hard, an agent that fails every hard case still scores 90% — while delivering nothing, because the hard cases were the reason you built it. Report accuracy by case type.
- Run it on every change. Evals that run automatically when the agent's logic, prompts, or knowledge change are the only regression protection you get. Evals that run "when we remember" are decoration.
Earn autonomy in stages
No agent should go from demo to autonomous in one step. The rollout that works is graduated: first shadow mode, where the agent does the work alongside the team and its output is compared but not used; then supervised, where the agent acts and a human approves everything; then autonomy expanded one decision type at a time, as the numbers justify it.
The corrections are the point. Every approval and every fix is a labelled example, and fed back in, they move the accuracy curve steadily — in our deployments, typically from the high 80s in week one to the high 90s within a couple of months. Trust stops being a leap of faith and becomes a threshold you can read off a chart.
One more thing that separates systems that improve from systems that rot: keep knowledge out of the code. Business rules, policies, and reference data belong in stores the agent reads and operators can edit — versioned, so when behaviour shifts you can answer "what changed?" without an archaeology project. An ops person updating a rule should never require an engineer.
The checklist
Before an agent touches real work, you should be able to answer yes to all of these:
- Inputs are validated and normalised before the model sees them
- Every failure path has a defined behaviour, and none of them are silent
- External calls have timeouts and backoff, and retries distinguish transient from permanent
- State is checkpointed; hard limits cap steps, tokens, and spend
- Every decision is logged with enough context to reconstruct it later
- A golden dataset exists, scores the path as well as the answer, and runs on every change
- Escalation criteria are written down, and escalations carry the context a human needs
- Knowledge lives outside the code and is versioned
None of this is exotic, but it is a lot of scaffolding — and it explains most of the difference between agent projects that take a year and ours that put a first agent live in two to six weeks. The scaffolding is the same every time, so we build it once and reuse it; the part that is genuinely new for each client is understanding the workflow. Which is why that is where every engagement of ours starts.