Attify Index methodology

The Attify Index reports, for each job type and month, the share of outcomes AI agents claimed that passed the audit under Attify's published standard, pooled across companies that agreed to contribute their anonymous data. An edition publishes only when at least 5 companies contribute and no company holds more than half of the month's claims, and it names no company, vendor or agent.

What one edition says#

For one job type and one month:

FieldWhat it is
Audited share of claimsAudited outcomes divided by claimed outcomes, pooled across contributing companies, with a 95% interval
Companies and claimsHow many companies contributed, and how many claims they made
Standard and tierThe standard version applied, and the evidence tier (see below)
Provenance mixThe share of claims from connected data and from reconciled witnessed data
Why claims failedThe share of claims at each check they failed first, in the standard's own words
Customer-confirmedWhere measured: the share confirmed among claims that had an explicit answer, and how many had one
TrendUp to the last 6 months that passed the same rules

It is also published as JSON (GET /api/v1/index, schema attify.index/1) and through the MCP tool get_index.

What counts as a claim, and as audited#

Each job type's own standard decides, applied on the rules tier: every check that is a rule, on every claim, without the judge's sample.

Job typeA claimAudited on the rules tier when
Customer-support resolutionA conversation the AI agent counted as resolved (its own flag, or the reconstructed claim)No repeat contact within 7 days and no handoff by the keyword screen or a person's reply
Sales meetingsA meeting the AI counted as bookedChecks (a) to (e): the AI booked it, a new prospect, no person stepped in, not cancelled, attended
BookingsAn appointment the AI counted as bookedChecks (a) to (e): the AI booked it, a new booking, staff did not fix it, not cancelled, the customer came

The judge's check (resolution confirmed, a real sales meeting, the right booking) is not in the Index: it runs on samples per company, and the Index pools companies. An Index rate is therefore an upper bound on the judge-confirmed rate for the same claims, and it is labelled with its tier wherever it appears.

Records are counted once their follow-up is complete (7 days for support; the standard's settle time for meetings and appointments), so an edition is made after its month ends.

Whose data#

  • Contributing buyers only. A company that uses AI agents and gave explicit consent, recorded with who agreed, when and the consent version, for its outcomes to be pooled without its name. Consent can be withdrawn at any time; data from a company that withdrew leaves every later edition.
  • Connected or reconciled data only. Data read from the company's own systems (connected), or streamed by the vendor's agent and reconciled by the company that month (witnessed). Uploaded files never count.
  • Not vendor programs. Customers in a vendor's verification program agreed to that program's scope, which does not cover pooling. Their data is never in the Index.

Anonymity rules#

An edition for a job type and month publishes only when both hold:

  1. At least 5 contributing companies with claims that month. The minimum is a setting that can rise and can never be set below 5.
  2. No company above half of the claims. Otherwise one company's result could be read off the pool.

An edition prints no per-company number, and no breakdown by vendor, agent, sector or size that would fall below the same rules.

The interval#

Each edition's 95% interval treats every contributing company as a cluster. Claims from one company are more alike than claims from different companies, so the interval is built from how far the companies' own rates spread around the pooled one: the spread gives an effective sample size, as in the approach of Korn and Graubard (1998), and the interval is Wilson's at that size (Wilson, 1927) with Student's t for one fewer degrees of freedom than there are companies in place of the normal value. It is never narrower than the interval on the pooled counts, and it is wider when companies differ. Read it as the uncertainty about AI agents at companies like these, not only about the claims counted. The method is the same for a vendor's score across its customers and for the buyer benchmark (methodology).

Schedule#

Editions are made each month with the monthly verdict (on the 1st to 3rd, for the month before), once for each job type that passes. The trend shows only months that passed under the same rules.

Empty state#

Until a job type passes, the Index shows how many companies contribute and how many it needs, and no number. It never shows an invented, partial or estimated rate.

What the Index is not#

  • Not a ranking. It ranks no vendor and compares no job types with each other.
  • Not a market estimate. It describes the contributing companies, which chose to contribute and are not a random sample of all companies using AI agents.
  • Not a vendor's audited score. A vendor's score is measured on its own independent customers under the program rules, published with a signed attestation. See the registry.

Changes#

Any change to these rules is dated here and applies from the next edition. Earlier editions keep the rules they were made under.

DateChange
2026-11-01First rules: contributing buyers, connected or reconciled witnessed data, at least 5 companies, none above half

Questions

Where does the Attify Index get its data?

Only from companies that use AI agents and explicitly agreed to contribute their anonymous outcomes. The data is read from their own systems, or streamed by their vendor and reconciled by them. Uploaded files and vendor programs are never pooled.

Does the Index name vendors?

No. An edition names no company, vendor, agent or customer and prints no per-company number. Named results exist only as published vendor verifications in the registry.

Why is the Index empty?

An edition publishes only when at least 5 companies contribute for that job type and no company holds more than half of the month's claims. Until then the page says how many companies contribute and how many it needs.

Can I compare the Index with my own agent?

Contributing buyers see the benchmark next to their own agents in their dashboard, under the same standard and tier. Comparing it with a vendor's self-reported rate mixes two different counting rules.

Can a company stop contributing?

Yes, at any time. The withdrawal is recorded, and the company's data leaves every edition made after it.

Sources

  1. Attify. Attify Index. Read 8 Nov 2026.
  2. Statistics Canada, Survey Methodology. Confidence intervals for proportions with small expected number of positive counts estimated from survey data (Korn and Graubard, 1998). Published 15 Dec 1998. Read 9 Nov 2026.
  3. Journal of the American Statistical Association. Probable inference, the law of succession, and statistical inference (Wilson, 1927). Read 8 Nov 2026.