How to Evaluate an AI Agent Before It Talks to Customers
AI agent testing before launch: simulations, edge cases, rule checks and a controlled rollout, so the first real customers are not your test.
Nico Silva, Lead Forward Deployed Engineer · · 4 min read
On this page
- Why a demo is not a test
- Step 1: Write what “correct” means
- Step 2: Build a set of cases, including the ugly ones
- Step 3: Run simulations before exposure
- Step 4: Test the rules, not just the answers
- Step 5: Put real traffic in, in controlled steps
- Step 6: Decide in advance what stops the rollout
- Step 7: Keep testing after launch
- What to measure
- What good looks like
AI agent testing is the work you do before a customer talks to the agent, and the work you keep doing after. A demo tells you the agent can handle the call you rehearsed. Testing tells you what it does on the calls you did not. The goal is simple: the first real customers should not be your test.
This is the approach we use for voice agents. The same logic applies to WhatsApp or any other channel.
Why a demo is not a test
A demo uses a clean sample, a calm speaker and a happy path. Production has noisy lines, impatient callers, people who interrupt, and people who ask for a human in the first five seconds.
Most AI projects fail between the demo and production, and the cause is usually the same: systems behave unpredictably with real users, and there is no structured way to iterate with real data. Testing is how you close that gap.
Step 1: Write what “correct” means
Before any test, write the rules the agent works inside. What can it answer? What can it change? When must it hand over to a person? What can it never say or promise?
If the rules are vague, the tests will be too. An agent works inside your policy and never rewrites it. Testing checks that it stays there.
Then define success per call type: resolved, routed correctly, handed over with context, or correctly refused. Each call type gets its own definition.
Step 2: Build a set of cases, including the ugly ones
Collect real examples where you can. Then add the cases that break agents:
- The caller who interrupts or changes topic.
- The caller who asks for a human immediately.
- A vague request: “it’s about my bill.”
- A request outside the agent’s scope.
- A noisy or low-quality line.
- A caller who gives a wrong number, or dictates it unclearly.
- A customer who is upset.
- A request that tests the rules: asking the agent to promise something it cannot.
Step 3: Run simulations before exposure
We run simulation testing before the agent is exposed to customers. A simulated caller plays each case and the conversation is scored against your definition of correct.
Simulations are fast and repeatable, which makes them good at catching regressions: you change one rule and re-run everything to see what else moved. They are not a substitute for real traffic, because a simulated caller is a guess about real behavior. Treat simulation as a gate, not a verdict.
Step 4: Test the rules, not just the answers
An answer can sound right and still break a rule. Check these explicitly:
- Does the agent verify who it is speaking to before sharing account details?
- Does it say it is an AI and that the line is recorded?
- Does it escalate when it should, and only then?
- Does the handover include the transcript and the reason?
- Does it stay inside the offer your rules allow?
Step 5: Put real traffic in, in controlled steps
After simulation, we do not switch on the full volume. We introduce real traffic gradually: 30–100 interactions a day, with daily review. A person reads the conversations, finds the edge cases, and the agent is fixed. Humans and AI both evaluate. Only when it is stable do we raise the volume, with stress testing under load before full production.
This is slower than a launch event and faster than a failed project. See our post on moving from pilot to production in 30 days.
Step 6: Decide in advance what stops the rollout
Write the stop conditions before you start. For example: a disclosure missed, a policy broken, a handover with no context, a rise in complaints. When one happens, you pause, fix and resume. Deciding at the moment of a problem is harder than deciding in advance.
Step 7: Keep testing after launch
An agent that passed testing can still drift. Inputs change, policies change, a model update shifts behavior. In our Deskflow monitoring we caught drift in the agent’s judgment in about 15 minutes. That is what continuous testing is for. See our post on monitoring agents in production.
What to measure
- Share of cases resolved correctly under your definition.
- Share of cases handed over correctly, with context.
- Rule violations per hundred conversations.
- Cases the simulation passed but real traffic failed. That gap tells you how good your simulations are.
What good looks like
A good test process is boring. You know what correct means, you have cases that break things, you gate on simulation, you ramp real traffic slowly, and you know what would make you stop. If a vendor cannot describe their equivalent, ask how they would know the agent is wrong.
If you are about to launch an agent and want a second opinion on the test plan, bring it and we will go through it with you.