# Audit sample size calculator

> How many AI conversations, meetings or bookings to label to estimate a share within a margin you choose, and how wide the interval is for a sample you already have.

Published 2026-11-12 · Reviewed 2026-11-12. Canonical: https://attify.app/tools/audit-sample-size

To estimate a share (resolved, held, matched) within ±5 points at 95% confidence, you need to label at most 385 records, and fewer when the share is far from 50% or the population is small. With the defaults below (synthetic): an expected share of 50% and a ±5-point margin need 385 labels, or 282 if only 1,050 claims are left to audit.

## Questions

### Why does Attify's judge read 200 per company?

200 gives a 95% interval of about ±7 points in the worst case (a share near 50%), and narrower nearer 0% or 100%. It balances the cost of judging against the precision of the rate.

### What do 50 human labels show about the judge?

Attify validates the judge for a standard version once at least 50 human labels agree with it 85% of the time or more on each judged decision, and each published score has its own human audit of 50 judged records (all of them when fewer). 50 labels measure agreement only roughly: 44 agreeing of 50 (88.0%) has a 95% interval of 76.2% to 94.4%, so passing the bar does not show that the judge's true agreement is 85% or more.

## Sources

- Journal of the American Statistical Association: [Probable inference, the law of succession, and statistical inference (Wilson, 1927)](https://doi.org/10.1080/01621459.1927.10502953), read 2026-11-08
