Zealoop

AI Chatbot Testing: A Practical Test Plan for SaaS Support Agents

A repeatable test plan for checking whether a SaaS support chatbot answers from approved documentation, handles customer records safely, and escalates reliably.

ai chatbot testingsaas supportchatbot qaconversational aisupport automation

A support agent can pass a vendor demo and still fail on a company’s refund rules, identity checks, or billing records. That gap is why AI chatbot testing for SaaS support needs to check both what the agent says and what it does to customer accounts.

AI chatbot testing checks whether a chatbot gives correct, consistent, usable answers and behaves safely across normal, ambiguous, and adversarial conversations. For a SaaS support agent, the plan should test documentation-grounded answers, intent and entity handling, customer-specific lookups, permissioned actions, and escalation. Each case needs an expected result, a pass/fail rule, and a repeatable way to run it after changes.

What AI chatbot testing should prove

Testing should establish four things: correctness, consistency, usability, and control. Correctness means that a response matches approved product information or verified account data. Consistency means equivalent requests produce compatible answers and decisions. Usability means a customer can understand the answer and continue the task. Control means the system stays within its permissions, even when a request is confusing or malicious.

That combination matters because generative answers do not behave like fixed decision trees. A scripted FAQ may be tested by checking whether one input triggers one expected text. A generative agent can express the same correct answer in many ways, so exact string matching alone produces false failures. But semantic flexibility is not a reason to accept unsupported claims: the test must still identify the facts and actions that are required or prohibited.

A Reddit discussion about enterprise support-agent rollouts raises the practical deployment question: who owns acceptance testing, and how does a team verify both the conversation and the resulting CRM, billing, or account state? The answer should be shared across customer support or CX operations, engineering or implementation, and the owner of the systems and permissions. One person can coordinate the suite, but no single team can validate every layer.

For a small SaaS team, define acceptance criteria before connecting the agent to live systems. For example: “The agent may explain the documented refund policy; it must not promise a refund unless the customer’s record and policy permit one; it must route exceptions to a human.” This is more testable than “the bot should be helpful.” Teams planning an agent’s capabilities can also use Zealoop’s guide to chatbots, grounded agents, and action-taking agents to distinguish the behaviors they need to evaluate.

Build a representative chatbot test set

A test suite is only useful if it represents the conversations and customer states the agent will meet. Start with resolved support tickets, help-center searches, known escalation reasons, and documented workflows. Remove personal data or replace it with synthetic examples before using those conversations in a test environment. Then label each case by intent, required facts, relevant entities, expected response, and risk.

For a SaaS product, intents might include canceling a subscription, changing a workspace member, understanding an invoice, or reporting an outage. Entities could include a plan name, invoice identifier, renewal date, workspace, or account email. A test should state whether the entity is supplied, missing, ambiguous, invalid, or deliberately mismatched with the authenticated customer.

Build variations around each important case instead of writing only one ideal prompt. Include paraphrases (“I need to stop renewal” and “turn off next month”), shorthand, spelling mistakes, negative cases, and irrelevant input. Add multi-turn examples where a customer supplies a missing detail later, changes their mind, or contradicts information given earlier. If the bot can use customer context, test the same question with different synthetic users and account states.

A compact test-case record can include:

Balance common cases with high-consequence edge cases. A team might begin with its top recurring ticket types, then add at least one test for each documented action, permission boundary, and escalation route. This is a starting method, not a universal sample-size rule: the needed number of cases depends on product complexity, risk, and how many independent policies or flows exist.

Test documentation-grounded answers and hallucinations

For every answerable product question, record the approved source and the fact the agent is expected to communicate. Test both direct questions and realistic paraphrases. A case such as “Can I invite another teammate?” should check the documented limit or procedure, not merely whether the answer sounds plausible. Include conflicting or outdated documentation deliberately if the agent’s retrieval process might surface it, and confirm the expected source wins or the uncertainty is surfaced.

Also test questions the documentation cannot answer. Ask about a feature that is not available, an unpublished roadmap date, or a policy that does not exist in the approved material. The passing result is not a confident guess. It is a clear statement that the answer cannot be confirmed, followed by an appropriate next step, such as a human handoff where the issue warrants one.

A practical scoring rule separates factual support from conversational quality. Mark a response as failed if it introduces a material policy, limit, price, or product capability that is absent from the approved source; contradicts the source; or omits a required qualification. A response may pass with different wording if it preserves the relevant facts and scope. Keep the reference passage or policy version with the case so reviewers can explain exactly why a claim is unsupported.

Check citations or traceability when the system provides them. A citation should point to material that actually supports the claim, rather than merely mentioning the same topic. For example, a page about changing a plan does not substantiate a promise that a billing credit will be issued. Zealoop says its agent answers from company documentation with citations and a trace of each turn; its site also describes a cross-encoder reranker, a relevance threshold before material reaches the model, and a second model to validate grounding. Those are product-specific design details, not substitutes for testing the team’s own sources and expected behavior.

For each set of answer tests, track two separate outcomes: whether the response was supported and whether it was useful. A factually grounded answer can still fail if it leaves out a necessary next step. Conversely, a fluent, useful-sounding response fails if it invents a policy. This distinction makes review actionable: content owners can fix missing or contradictory documentation, while engineers can investigate retrieval, prompting, or validation behavior.

Test intent, entities, and conversation flow

Intent identification tests whether the agent recognizes what the customer is trying to do, even when the phrasing changes. For each high-volume intent, write multiple utterances and include near-neighbor intents that could be confused. “I can’t get into my account” might mean a login problem, an expired invitation, or a locked account. A passing agent asks a targeted clarifying question when the distinction changes the answer or available action.

Entity extraction tests whether the agent captures the right value and associates it with the right field. Try a valid invoice number, a mistyped identifier, two invoice numbers in one message, and a number that belongs to another account. The test should inspect the selected record or event—not just the bot’s statement that it found the right one. If the entity is ambiguous, the safe behavior is to clarify rather than silently choose.

Multi-turn consistency matters when earlier turns establish identity, scope, or a decision. Test that the agent remembers a provided invoice number, does not ask for the same detail repeatedly, and does not reinterpret an approved action after a user asks a follow-up. Also test a correction: if the customer says “I meant invoice 104, not 140,” the agent should update the target and confirm consequential steps against the corrected value.

Include small talk and irrelevant input, but keep these checks proportionate to support risk. A greeting should not trigger an account change, and an unrelated request should not be mistaken for authorization. A useful checklist for each flow is: expected intent, required entities, ambiguity threshold, clarification wording, state retained across turns, and safe exit when the user cannot provide the information.

Rasa’s documentation describes end-to-end tests that can validate a full assistant interaction, including whether a slot is set, a flow starts, an action is triggered, or a response is grounded. That approach is useful when the behavior depends on a sequence of turns: validating only the final sentence can miss an earlier mistaken intent or a wrong entity assignment.

Test customer-data lookups and guarded actions

Account lookups need tests for identity, authorization, and record selection. Create synthetic customers with similar names, multiple workspaces, different plan states, and records that should not be visible to one another. Ask a question that requires a lookup, then compare the response with the fixture data. The case fails if the agent reveals another customer’s information, selects the wrong record, or implies it checked data when the lookup failed.

Separate “can retrieve” from “may disclose.” A system may successfully query a field that should not be shown to a particular user. Test the authorization rule with authenticated, unauthenticated, and insufficiently privileged contexts where relevant. Include missing data, a stale record, and a service response that contains an error. In each case, the expected result should state whether the agent can answer, should clarify, or must hand off.

For actions, assert the final system state as well as the conversation. A test for changing a subscription should inspect the subscription fixture after the attempted action. Verify the allowed case, the denied case, a missing-approval case, a duplicate request, and a retry after a timeout. If an action can be repeated, include an idempotency check so a repeated message does not create duplicate changes or tickets.

Write a small permission matrix before implementing action tests. For each action, list the user or role allowed to request it, the conditions that must be true, whether the customer must confirm, and the fallback when the action is unavailable. For instance, if a billing credit requires staff approval, a test should prove that the agent cannot bypass that requirement by phrasing the request as urgent or by claiming an employee already approved it.

Treat customer approval and system authorization as distinct controls. A customer may confirm a request, but that does not grant the agent permission to perform an action the application disallows. Zealoop’s demo says write actions wait for customer approval rather than the model; teams should still test the deployed workflow’s identity, policy checks, and resulting record state. The broader AI chatbot security risks guide covers why access boundaries need to be considered alongside answer quality.

Test escalation, errors, and adversarial inputs

A handoff is a successful outcome when the agent has reached its limits. Test the conditions that should trigger escalation: unresolved uncertainty, sensitive account issues, a policy exception, an explicit request for a person, or a failed integration. A pass requires more than a polite sentence. The handoff should reach the intended destination and carry useful context, such as the issue summary, relevant account identifier, steps already tried, and reason for escalation—without exposing data the recipient should not receive.

Exercise failures that occur outside the model. Simulate a lookup timeout, a declined action, a missing field, a rate limit, and a handoff service error. The agent should not report success before it receives confirmation from the connected system. It should explain the problem at an appropriate level, avoid repeating a destructive action blindly, and offer a safe retry or human route. Record the actual API or workflow result alongside the transcript.

Prompt injection testing checks whether hostile instructions can override the intended behavior. Try direct requests to ignore prior rules, requests embedded in quoted text or pasted documents, and attempts to extract system instructions or private account information. Include a message that asks the agent to take a normally allowed action but bypass a required confirmation. The fail conditions are concrete: disclosure of protected information, unauthorized tool use, a prohibited action, or a response that claims a control was bypassed.

OWASP’s AI Testing Guide describes prompt-injection testing in terms of attempts to override intended instructions, expose sensitive details, or cause unauthorized actions. It also notes that conversation history can affect later outputs. So run attack cases in multi-turn sequences, not only as isolated single prompts: first establish a benign context, then introduce the conflicting instruction, and inspect both the answer and any tool events.

Repeat variable generative tests rather than treating one successful refusal as proof. Keep the input, conversation history, model or prompt version, and system state fixed when comparing runs; then report whether any run crossed a defined safety boundary. The team can use human review for nuanced responses, but data disclosure and forbidden action checks should be based on observable outputs and system state wherever possible.

Test across channels, languages, and user experiences

The same support flow can behave differently in an embedded widget, mobile interface, email handoff, or another channel. Test each channel the team actually deploys, including message ordering, buttons or forms, links, session continuity, and handoff behavior. A channel test should confirm that an answer remains readable and that a customer can complete the next step—not just that the model returns text.

Localization testing should reflect the languages the business supports. Translate or author equivalent cases in each relevant language and check policy meaning, product terminology, dates, currencies, and escalation instructions. Include mixed-language messages if customers use them. Do not assume that a fluent translation preserves a legal or billing qualification; compare the answer with an approved localized source or a designated reviewer.

Review clarity, length, and accessibility as part of task success. Ask whether the user can identify the answer, the action taken or not taken, and what to do next. Test a narrow mobile viewport, keyboard use, readable links, and the handoff interface if those are part of the customer experience. The exact accessibility checks depend on the interface and supported channels, so they should be adapted to the product’s own accessibility requirements rather than inferred from a language-model score.

Botium’s documentation describes tests for conversation flows and utterance variations, with assertions that can cover response text, interface elements, links, custom payloads, and business logic through API or data-store queries. That range helps illustrate why channel coverage should include interface and downstream behavior as well as language. A test on a web widget does not establish that a handoff link or quick reply works in a mobile client.

Automate chatbot tests and choose tools

Automated chatbot testing is most effective when deterministic checks and human evaluation have separate jobs. Automate stable expectations: correct intent or route, required entity, citation presence, API call, permission decision, record state, successful handoff, and forbidden action absence. Use human review or a calibrated evaluator for qualities such as whether a variable answer is sufficiently clear, relevant, and empathetic. Do not turn a subjective score into the only release gate for a high-risk action.

Botium is a fit to consider when a team wants scripted conversation and utterance tests, including checks that reach beyond the text response to interface elements or business logic. Its documentation describes “convo” files for expected flows and utterance lists for alternative user phrases. Those features map well to repeatable regression cases for intent recognition and channel behavior.

Rasa’s testing documentation is relevant to teams building with Rasa: it covers end-to-end conversation tests, quantitative and qualitative evaluation, and assertions about events such as slot setting, flow starts, actions, and grounding. Its CLI supports running end-to-end tests with rasa test e2e. A Rasa-specific suite is not a universal requirement; a small team can also start with its application’s own test framework and mocked APIs if that gives reliable assertions over the behavior it owns.

Choose tools against the tests that matter, not the size of a feature list. A useful selection exercise is to implement three sample cases before committing: a grounded answer with a citation check, an account lookup with a fixture-state assertion, and a denied action followed by a successful handoff. Compare how each candidate tool handles setup, test data, repeat runs, readable failures, and integration with the team’s deployment process. Vendor capability and licensing details vary; the test plan should not depend on an unconfirmed tool feature.

Run regression tests whenever a material dependency changes: support documentation, prompts, retrieval settings, model, policy rules, tools, or channel code. Keep a version label for each of those inputs so that a failure can be reproduced. A changed answer is not automatically a regression, but a previously passing policy boundary, source-support check, or action assertion should fail the build until reviewed.

Use a chatbot testing checklist and track results

A compact release checklist can keep acceptance testing practical without reducing it to a demo. Before a rollout, the team should be able to answer yes to each item:

Metrics become useful when each is tied to a test definition. Grounded-answer pass rate can count cases where every material claim is supported by the assigned source; use it to catch retrieval or content regressions. Unsupported-answer rate should count cases where the agent asserted a material fact without support, especially on intentionally unanswerable questions. Keep its numerator and denominator clear rather than folding it into a broad satisfaction score.

Task completion rate should come from end-to-end scenarios with a defined outcome, such as locating the correct invoice or opening the intended support case. Escalation success rate should measure whether required handoffs arrive with the expected context, not merely whether the bot displayed a handoff message. Regression failures should report newly failing cases by category and release, so a rise in account-selection failures is distinguishable from a tone-review issue.

Do not treat a test dashboard as proof of production outcomes. For example, Zealoop labels its homepage demo metrics as illustrative and says customer dashboards report metrics from their own conversations. The practical lesson is to separate synthetic acceptance results from observed customer conversations, and to document the population and test conditions behind each reported value.

Start with a release gate focused on high-consequence failures: no unauthorized data disclosure, no unapproved write, no false claim of completed action, and no broken required escalation. Other quality goals—such as shorter answers or better small talk—can be prioritized using actual support needs. A team should be able to explain which failures block release, who reviews exceptions, and what evidence supports the decision.

FAQ

How do you check whether a chatbot is working correctly?

Run realistic test conversations against expected answers and system outcomes. For a support agent, check an answer against its approved documentation, verify that a lookup selects the correct synthetic account, inspect whether an action changed the intended record, and confirm required escalations arrive. A successful demo alone does not establish those results.

What should you test in an AI chatbot?

Test answer accuracy and grounding, intent identification, entity extraction, multi-turn consistency, unclear or missing information, customer-data permissions, guarded actions, errors, and escalation. Add prompt-injection attempts and the channels and languages customers actually use. The balance should reflect risk: a billing action needs stronger authorization and state checks than a greeting response.

How can you automate chatbot testing?

Represent repeatable conversations as scripted cases, provide synthetic account fixtures, and assert both conversation events and backend outcomes. Run the cases after relevant changes to prompts, documentation, model configuration, or integrations. Tools such as Botium or Rasa’s end-to-end testing can help with conversation suites; human review remains useful for variable qualities such as clarity and tone.

When should a support team move from testing to a limited rollout?

A team can consider a limited rollout after high-risk tests pass in a sandbox and owners have reviewed unresolved failures. Keep the initial scope narrow—for example, documentation answers before account changes—and monitor handoffs and customer feedback. Expand only when the team can detect a regression, stop an unsafe flow, and route exceptions to a responsible human.

Who should own acceptance testing for a support agent?

A named release owner should coordinate it, while CX or support operations defines expected customer outcomes, engineering checks integrations and state changes, and the system owner confirms permissions and data boundaries. For a refund flow, for example, support can approve the policy expectation, engineering can assert the billing result, and the permissions owner can verify the approval boundary. No team should sign off only on the transcript.