How is testing an AI application different from testing a normal web app?

The interface is the same; the output is not. A normal app returns the same answer twice, so you can assert on it. A model returns something different every time, so assertions move to the shape and behaviour around the answer — that it arrives, streams, stops, fails visibly and costs what it should.

How do you write an assertion when the output changes every time?

Assert on the contract, not the prose. That a response arrived within budget, that it parsed as the schema you asked for, that it cited a document the user is allowed to see, that a refusal renders as a refusal. Judging the wording is an eval, run separately from the regression suite.

What is prompt injection, and how do you test for it?

Instructions hidden in content your system reads — a document, a web page, a ticket — which the model then obeys. You test it by planting instructions in every channel that reaches the model and asserting the effect never lands: no tool called, no data returned, no system prompt repeated.

What is the most common security problem in AI products?

Unglamorous ones. A provider key reachable from the browser, a model endpoint with no rate limit, and retrieval scoped by an instruction in the prompt rather than a filter on the query. All three are cheaper to find than a clever jailbreak and more likely to be there.

Do you check whether the model gives correct answers?

No. Correctness needs your ground truth and belongs in an evaluation harness that you own and version alongside your prompts. Regression testing answers a different question: did the thing that worked yesterday still work after today’s deploy.

AI applications

Testing AI applications: what breaks, and how to test it.

An LLM app fails in ways a normal web app does not. The model returns something different every time, so you cannot assert on the answer — you assert on everything around it: that a response arrives, streams, stops, fails visibly, respects a limit, and never returns data the person is not allowed to see.

Last updated 24 August 2026

How is testing an AI application different?

The interface is the same. The contract is not. Traditional test automation rests on an assertion that holds twice — click this, see that. A model breaks that assumption on the first run, which is why teams often conclude their AI features cannot be regression-tested at all and then ship them untested.

They can. The trick is to move the assertion off the prose and onto the parts that are deterministic: the response arrived inside a budget, it parsed as the schema you asked for, it cited a document this user may read, a refusal rendered as a refusal, a retry did not charge twice. All of that is ordinary automation. Judging whether the answer is good is an evaluation problem, it needs your own ground truth, and it belongs in a harness beside your prompts rather than in a regression suite.

What actually breaks in an LLM app

These are behaviours, not opinions about output quality, which is what makes them testable. Each one reaches users as a bug in the product rather than as a bad answer.

Failure	How it reaches a user	The test
The answer streams, then stops halfway	A dropped connection mid-stream leaves a half-written reply on screen with no error and no retry. Users read it as the answer.	Cut the response mid-stream and assert the UI shows a failure state, not a truncated answer.
The model is slow, and the UI has no answer for it	A p95 that sits at forty seconds while the interface offers no progress, no cancel and no timeout. People click again, and now two requests are billing.	Hold the response open past the timeout and assert there is a visible state, a cancel path, and no duplicate request on a second click.
The provider returns 429 or 503	Rate limits and provider outages are routine, not exceptional. Apps that treat them as impossible show a raw error or an endless spinner.	Return 429 and 503 from the provider and assert the app degrades to a message a user can act on.
Empty, refused and safety-filtered replies	The model declines, or the safety filter fires, and the interface renders an empty bubble because it only handles the happy path.	Force a refusal and an empty completion; assert neither produces a blank message or a broken layout.
The conversation loses its context	History is trimmed to fit the window and the assistant silently forgets what it was told three messages ago. It reads as the product being stupid.	Drive a conversation past the context limit and assert the app says what it dropped rather than dropping it quietly.
Tool calls fire more than once	An agent retries a step whose tool has already run. Whatever that tool did — an email, a charge, a write — happens twice.	Make a tool call time out after it succeeded and assert the retry does not repeat the side effect.
Retrieval returns another tenant’s document	The vector store is filtered in the prompt rather than in the query, so a retrieval that ignores the instruction returns data the user may not read.	Query as one tenant for a document owned by another and assert it is absent from the retrieved set, not merely unmentioned in the answer.
Cost runs away without a ceiling	No per-user cap, so one loop or one scripted account turns into a bill nobody notices until the invoice.	Drive the same endpoint repeatedly and assert a limit exists — per user, per key, per minute.

Security

LLM security testing: the surface worth checking

An AI feature usually arrives wired to two things a normal feature is not: a paid endpoint, and a component that follows instructions found in whatever it reads. Both widen the attack surface in ways a standard application security review was not written for.

Ordered below by how often it is actually found — which is not the order these get written about.

Provider keys in the client bundle

A model provider key shipped in front-end JavaScript, or reachable through a proxy route with no auth in front of it.

Anyone with developer tools can spend your quota. This is the cheapest thing on this list to check and the most expensive to miss.

Prompt injection through content

Instructions hidden in something the model reads — an uploaded document, a scraped page, a support ticket, an email body — that the model then follows.

The attacker never touches your input box. Anything your system ingests is an input, whether you designed it as one or not.

Injection that reaches a tool

Injected instructions that cause an agent to call a tool: send a message, write a record, make a request to an internal address.

This is where injection stops being an embarrassment and becomes an incident. The blast radius is whatever the agent is allowed to do.

System prompt and secret disclosure

Getting the model to repeat its own instructions, its tool definitions, or fragments of configuration it was given.

The system prompt often contains business rules, internal endpoints and occasionally credentials nobody meant to put there.

Cross-tenant retrieval

Retrieval that is scoped by asking the model nicely rather than by filtering the query.

A prompt is not an access control. If the filter lives in the instructions, one ignored instruction is a data breach.

Output rendered as markup or code

Model output placed into the page as HTML, or into a shell, a query or an eval.

Classic injection with a new source. The model is now an untrusted input, and it is one an attacker can influence.

Unbounded spend as a denial of service

An endpoint that calls a paid model with no rate limit, no per-account cap and no size limit on what it will accept.

The attack is not to take you down. It is to run up a bill until you take yourself down.

What the logs keep

Prompts and completions written to logs and analytics — including whatever a user pasted into them.

People paste production credentials and customer records into chat boxes. Your log retention is now a data-protection question.

What QA Spider automates, and what it does not

Stated narrowly enough that you can hold us to it.

We automate this

Drives your product in a real browser, so the streaming, timeout, retry, empty-state and error-state cases above are ordinary UI tests — they run on a schedule and on every deploy.
Checks the API beneath the interface, including the routes your front end calls to reach a model.
Files what fails as a ticket in Jira, Azure DevOps, ClickUp, GitHub or Linear, with the steps that reproduce it and a screenshot.
Runs a surface security pass on the public site: transport and security headers, cookie flags, mixed content, form exposure, DNS and email spoofing protection.

We do not

We do not currently ship an automated prompt-injection or jailbreak suite. If someone tells you their crawler does this out of the box, ask them to show you the assertions.
We do not evaluate whether an answer is correct. That is an eval problem, it needs your ground truth, and it is a different discipline from regression testing.

Questions people ask about testing AI apps

How is testing an AI application different from testing a normal web app?: The interface is the same; the output is not. A normal app returns the same answer twice, so you can assert on it. A model returns something different every time, so assertions move to the shape and behaviour around the answer — that it arrives, streams, stops, fails visibly and costs what it should.
How do you write an assertion when the output changes every time?: Assert on the contract, not the prose. That a response arrived within budget, that it parsed as the schema you asked for, that it cited a document the user is allowed to see, that a refusal renders as a refusal. Judging the wording is an eval, run separately from the regression suite.
What is prompt injection, and how do you test for it?: Instructions hidden in content your system reads — a document, a web page, a ticket — which the model then obeys. You test it by planting instructions in every channel that reaches the model and asserting the effect never lands: no tool called, no data returned, no system prompt repeated.
What is the most common security problem in AI products?: Unglamorous ones. A provider key reachable from the browser, a model endpoint with no rate limit, and retrieval scoped by an instruction in the prompt rather than a filter on the query. All three are cheaper to find than a clever jailbreak and more likely to be there.
Can QA Spider test my AI product?: The behaviour around the model, yes: streaming, timeouts, retries, refusals, error states, rate limits and the API beneath the interface, tested in a real browser on every deploy. Automated prompt-injection suites are not something we ship today, and we would rather say so than let you find out later.
Do you check whether the model gives correct answers?: No. Correctness needs your ground truth and belongs in an evaluation harness that you own and version alongside your prompts. Regression testing answers a different question: did the thing that worked yesterday still work after today’s deploy.

Point it at your AI product.

The behaviour around the model — streaming, timeouts, retries, refusals, rate limits and the API beneath the interface — tested in a real browser, on every deploy.

Start free →