Confidence interval (95%)
A 95% confidence interval is the range of values for a rate that are consistent with what was measured, given how many records it rests on. If the same measurement were repeated many times on new samples, about 95% of the intervals built this way would contain the true rate. Attify prints one next to every rate, gap and agreement figure.
Which intervals#
- A single rate (an audited rate, a share of claims): the Wilson score interval (Wilson, 1927), which behaves well near 0% and 100% and for small samples.
- A difference (the gap between claimed and audited, the AI against the customer's own staff): MOVER intervals, which combine the intervals of the two rates (Newcombe, 1998).
- A pooled rate (a vendor's score across its customers, a buyer's benchmark, the Attify Index): an interval that treats each company as a cluster, because outcomes at one company move together. It is never narrower than the interval on the pooled counts (methodology).
Worked example#
Synthetic example: a judge confirms 160 of 200 sampled claims, 80.0%. The Wilson 95% interval is 73.9% to 85.0%. With 800 of 1,000 it is 77.4% to 82.4%. Five times the sample makes the interval a little under half as wide.
The formula, with p the observed share, n the sample and z = 1.96:
centre = (p + z²/2n) ÷ (1 + z²/n)
half-width = z × √( p(1−p)/n + z²/4n² ) ÷ (1 + z²/n)
What an interval does not cover#
It covers sampling error. It does not cover a wrong rule, missing records, or a judge's systematic error beyond what the human audit measured. Those are stated separately: the standard version, the provenance, the evidence tier, and the judge's agreement with people.
Questions
If two vendors' intervals overlap, are they the same?
- Not necessarily. Overlap means the difference may be chance; the interval for the difference itself is the test, and two intervals can overlap a little while it excludes zero. Attify's dashboards say 'no clear change' when a month-on-month move stays inside overlapping intervals.
Why does a small vendor's interval look so wide?
- Fewer records mean more uncertainty. Width shrinks roughly with the square root of the sample: four times the records halves it.