Attify Index methodology
The Attify Index reports, for each job type and month, the share of outcomes AI agents claimed that passed the audit under Attify's published standard, pooled across companies that agreed to contribute their anonymous data. An edition publishes only when at least 5 companies contribute and no company holds more than half of the month's claims, and it names no company, vendor or agent.
What one edition says#
For one job type and one month:
It is also published as JSON (GET /api/v1/index, schema attify.index/1) and through the MCP tool get_index.
What counts as a claim, and as audited#
Each job type's own standard decides, applied on the rules tier: every check that is a rule, on every claim, without the judge's sample.
The judge's check (resolution confirmed, a real sales meeting, the right booking) is not in the Index: it runs on samples per company, and the Index pools companies. An Index rate is therefore an upper bound on the judge-confirmed rate for the same claims, and it is labelled with its tier wherever it appears.
Records are counted once their follow-up is complete (7 days for support; the standard's settle time for meetings and appointments), so an edition is made after its month ends.
Whose data#
- Contributing buyers only. A company that uses AI agents and gave explicit consent, recorded with who agreed, when and the consent version, for its outcomes to be pooled without its name. Consent can be withdrawn at any time; data from a company that withdrew leaves every later edition.
- Connected or reconciled data only. Data read from the company's own systems (connected), or streamed by the vendor's agent and reconciled by the company that month (witnessed). Uploaded files never count.
- Not vendor programs. Customers in a vendor's verification program agreed to that program's scope, which does not cover pooling. Their data is never in the Index.
Anonymity rules#
An edition for a job type and month publishes only when both hold:
- At least 5 contributing companies with claims that month. The minimum is a setting that can rise and can never be set below 5.
- No company above half of the claims. Otherwise one company's result could be read off the pool.
An edition prints no per-company number, and no breakdown by vendor, agent, sector or size that would fall below the same rules.
The interval#
Each edition's 95% interval treats every contributing company as a cluster. Claims from one company are more alike than claims from different companies, so the interval is built from how far the companies' own rates spread around the pooled one: the spread gives an effective sample size, as in the approach of Korn and Graubard (1998), and the interval is Wilson's at that size (Wilson, 1927) with Student's t for one fewer degrees of freedom than there are companies in place of the normal value. It is never narrower than the interval on the pooled counts, and it is wider when companies differ. Read it as the uncertainty about AI agents at companies like these, not only about the claims counted. The method is the same for a vendor's score across its customers and for the buyer benchmark (methodology).
Schedule#
Editions are made each month with the monthly verdict (on the 1st to 3rd, for the month before), once for each job type that passes. The trend shows only months that passed under the same rules.
Empty state#
Until a job type passes, the Index shows how many companies contribute and how many it needs, and no number. It never shows an invented, partial or estimated rate.
What the Index is not#
- Not a ranking. It ranks no vendor and compares no job types with each other.
- Not a market estimate. It describes the contributing companies, which chose to contribute and are not a random sample of all companies using AI agents.
- Not a vendor's audited score. A vendor's score is measured on its own independent customers under the program rules, published with a signed attestation. See the registry.
Changes#
Any change to these rules is dated here and applies from the next edition. Earlier editions keep the rules they were made under.
Questions
Where does the Attify Index get its data?
- Only from companies that use AI agents and explicitly agreed to contribute their anonymous outcomes. The data is read from their own systems, or streamed by their vendor and reconciled by them. Uploaded files and vendor programs are never pooled.
Does the Index name vendors?
- No. An edition names no company, vendor, agent or customer and prints no per-company number. Named results exist only as published vendor verifications in the registry.
Why is the Index empty?
- An edition publishes only when at least 5 companies contribute for that job type and no company holds more than half of the month's claims. Until then the page says how many companies contribute and how many it needs.
Can I compare the Index with my own agent?
- Contributing buyers see the benchmark next to their own agents in their dashboard, under the same standard and tier. Comparing it with a vendor's self-reported rate mixes two different counting rules.
Can a company stop contributing?
- Yes, at any time. The withdrawal is recorded, and the company's data leaves every edition made after it.