Why we test refusal before anything else
Most conversations about getting an AI assistant ready focus on what it should be able to do — how many topics it covers, how naturally it responds, how fast it answers. Those things matter, but none of them are the first thing we check.
The first thing we check is whether the assistant can recognize the edge of what it actually knows. An assistant that answers 90% of questions well but occasionally states something false with full confidence is more dangerous in production than one that visibly struggles, because the confident version doesn't give anyone a reason to double-check it.
So before we look at anything else, we test the assistant against exactly the situations designed to expose that failure.
The four kinds of questions we throw at it
We don't only test with straightforward questions the assistant is obviously built to answer. Every pre-launch test set includes four categories, each testing something different:
- ›Common questions — the baseline. If it can't answer these cleanly and correctly, it isn't close to ready.
- ›Unclear questions — deliberately ambiguous or incomplete, to see whether it asks for clarification instead of guessing at what was meant.
- ›Sensitive questions — topics where a wrong or overconfident answer could cause real harm, to confirm it declines appropriately.
- ›Deliberately misleading questions — phrased to lead the assistant toward a plausible-sounding but incorrect answer, to see whether it can be talked into confidence it hasn't earned.
The rule that holds a launch
If the assistant produces even one confident response that isn't backed by an approved source, we hold the launch. Not a note to fix later — the launch stops.
That's a deliberately low threshold. A single unsupported confident answer in testing means the same failure is statistically likely to happen again in front of a real user, just at a moment we don't control. We'd rather find it ourselves, on our own schedule, than have a client find it first.
What happens after a failure
A held launch starts a specific process. First, we trace exactly why the assistant answered instead of declining or asking for clarification — usually that's either a gap in the approved knowledge base, or a refusal rule that wasn't specific enough to catch that phrasing.
We correct whichever one was actually at fault. Then the exact failed scenario gets added permanently to our regression test set, so it's part of every future test run for that assistant, not just this one. Only then do we run the complete test set again — from the beginning, not just the section that failed — because a fix in one place can change how the assistant handles something else.
Checking the exit, not just the answers
Correct refusals only help if there's somewhere for the conversation to go afterward. Before any assistant is released, we separately verify that escalation to a human actually works — that a user who needs a person can reliably reach one.
This is checked as its own step, deliberately separate from refusal testing. An assistant can get every refusal decision right and still fail in production if the handoff behind it is broken.
Key takeaways
- ›A confident, unsupported answer is a more dangerous failure than a visible "I don't know" — that's what our pre-launch test catches first.
- ›The test set includes common, unclear, sensitive, and deliberately misleading questions, not just ones the assistant was obviously designed for.
- ›One confident, unsupported response is enough to hold a launch — the threshold is intentionally strict.
- ›Every failure becomes a permanent regression test, and a fix means rerunning the entire test set, not just the failed part.
- ›Escalation to a human is verified as its own separate check, never assumed.