Use case 3 · Zero-failure test bounds

What passing tests actually proves

The pattern

A chatbot that passes 50 attempts to make it commit to something the business does not offer has not been shown safe. Passing tests bound the failure rate; they do not zero it.

Question: How many tests before a chatbot can be trusted with customers?
Lane: Customer-facing AI
Basis: Standard practice

What METRIS does

Run a fixed bank of attempts to make the bot break each business rule
Report the 95% upper bound on the failure rate, not just pass or fail
Re-run the bank after every vendor or model update

The most the failure rate could be

Exact calculation · assumed parameters

After 50 passing attempts, the failure rate could still be as high as 5.8% (95% confidence); 1,000 passes bring it to 0.3%. The rule of three tracks the exact bound closely.

Method

after n passes and 0 failures: upper bound = 1 − 0.05^(1/n) ≈ 3/n P(find ≥ 1 failure in k attempts) = 1 − (1 − p)^k

Confidence: 95%
Passing runs (n): 10 to 1,000
Weakness rates (p): 0.5%, 1%, 2%, 5%

'It passed our tests' is a statement about the test bank. The bound turns it into a statement about how often attempts like these could succeed.

Chance a test bank finds a weakness

Exact calculation · assumed parameters

To find a weakness that succeeds 1% of the time with 95% probability, a test bank needs 299 attempts; at 0.5%, 598.

Reproducibility

Closed-form binomial bounds; no random draws. Python with NumPy 2.5.3 and SciPy.

Sources

This pattern is not based on any single record. Its figures come from the model stated above.

Study design and analysis: Suneeta Modekurty, SANJEEVANI AI

A statistical study of a pattern, not of any event. It identifies no organization or person. Simulated figures use synthetic data; published figures are industry-wide.