Use case 4 · Sampling error analysis

Why measuring every file beats reading a sample

The pattern

Examiners estimate a violation rate from a sample of files. The estimate moves a lot from sample to sample. A plan that checks every file has no sampling error.

Question: How much can a sample of files tell an examiner?
Lane: Human review
Basis: Standard examination practice

What METRIS does

Check every determination against each required step, every month
Report rates with intervals by delegate, service and reviewer team
Show the plan its own numbers before an examiner samples them

Same true rate, different samples

Exact calculation · assumed parameters

The true rate is 20% in both cases. Reading 60 files gives estimates from 10% to 30% (95% of samples); 300 files narrows it to 15.7% to 24.7%.

Method

p̂ = k / n, k ~ Binomial(n, p) 95% Wilson interval: (p̂ + z²/2n ± z·√(p̂(1−p̂)/n + z²/4n²)) / (1 + z²/n)

True violation rate: 20%
Samples compared: 60 and 300 files
Interval: Wilson score, 95%

A sampled rate is a range, not a number. Measuring the full population turns an examiner's estimate into the plan's own known figure.

Uncertainty shrinks with every file checked

Exact calculation · assumed parameters

The 95% margin falls from ±22.7 points at 10 files to ±2.5 at 1,000. Checking every file removes sampling error, provided the automated check itself is accurate.

Reproducibility

Exact binomial distributions, shown as probability per percentage point so both sample sizes share one scale; no random draws. Python with NumPy 2.5.3 and SciPy.

Sources

This pattern is not based on any single record. Its figures come from the model stated above.

Study design and analysis: Suneeta Modekurty, SANJEEVANI AI

A statistical study of a pattern, not of any event. It identifies no organization or person. Simulated figures use synthetic data; published figures are industry-wide.