# Attify Index methodology

> How the Attify Index pools audited AI agent outcomes by job type each month, from contributing buyers only, and the anonymity rules every edition passes.

Published 2026-11-12 · Reviewed 2026-11-12. Canonical: https://attify.app/index/methodology

The Attify Index reports, for each job type and month, the share of outcomes AI agents claimed that passed the audit under Attify's published standard, pooled across companies that agreed to contribute their anonymous data. An edition publishes only when at least 5 companies contribute and no company holds more than half of the month's claims, and it names no company, vendor or agent.

## What one edition says

For one job type and one month:

| Field | What it is |
|---|---|
| Audited share of claims | Audited outcomes divided by claimed outcomes, pooled across contributing companies, with a 95% interval |
| Companies and claims | How many companies contributed, and how many claims they made |
| Standard and tier | The standard version applied, and the evidence tier (see below) |
| Provenance mix | The share of claims from connected data and from reconciled witnessed data |
| Why claims failed | The share of claims at each check they failed first, in the standard's own words |
| Customer-confirmed | Where measured: the share confirmed among claims that had an explicit answer, and how many had one |
| Trend | Up to the last 6 months that passed the same rules |

It is also published as JSON (`GET /api/v1/index`, schema `attify.index/1`) and through the MCP tool `get_index`.

## What counts as a claim, and as audited

Each job type's own standard decides, applied on the rules tier: every check that is a rule, on every claim, without the judge's sample.

| Job type | A claim | Audited on the rules tier when |
|---|---|---|
| [Customer-support resolution](https://attify.app/standards/support-resolution) | A conversation the AI agent counted as resolved (its own flag, or the reconstructed claim) | No repeat contact within 7 days and no handoff by the keyword screen or a person's reply |
| [Sales meetings](https://attify.app/standards/sales-meetings) | A meeting the AI counted as booked | Checks (a) to (e): the AI booked it, a new prospect, no person stepped in, not cancelled, attended |
| [Bookings](https://attify.app/standards/bookings) | An appointment the AI counted as booked | Checks (a) to (e): the AI booked it, a new booking, staff did not fix it, not cancelled, the customer came |

The judge's check (resolution confirmed, a real sales meeting, the right booking) is not in the Index: it runs on samples per company, and the Index pools companies. An Index rate is therefore an upper bound on the judge-confirmed rate for the same claims, and it is labelled with its tier wherever it appears.

Records are counted once their follow-up is complete (7 days for support; the standard's settle time for meetings and appointments), so an edition is made after its month ends.

## Whose data

- **Contributing buyers only.** A company that uses AI agents and gave explicit consent, recorded with who agreed, when and the consent version, for its outcomes to be pooled without its name. Consent can be withdrawn at any time; data from a company that withdrew leaves every later edition.
- **Connected or reconciled data only.** Data read from the company's own systems (connected), or streamed by the vendor's agent and reconciled by the company that month (witnessed). Uploaded files never count.
- **Not vendor programs.** Customers in a vendor's verification program agreed to that program's scope, which does not cover pooling. Their data is never in the Index.

## Anonymity rules

An edition for a job type and month publishes only when both hold:

1. **At least 5 contributing companies** with claims that month. The minimum is a setting that can rise and can never be set below 5.
2. **No company above half of the claims.** Otherwise one company's result could be read off the pool.

An edition prints no per-company number, and no breakdown by vendor, agent, sector or size that would fall below the same rules.

## The interval

Each edition's 95% interval treats every contributing company as a cluster. Claims from one company are more alike than claims from different companies, so the interval is built from how far the companies' own rates spread around the pooled one: the spread gives an effective sample size, as in the approach of [Korn and Graubard (1998)](https://www150.statcan.gc.ca/n1/en/catalogue/12-001-X19980024356), and the interval is Wilson's at that size ([Wilson, 1927](https://doi.org/10.1080/01621459.1927.10502953)) with Student's t for one fewer degrees of freedom than there are companies in place of the normal value. It is never narrower than the interval on the pooled counts, and it is wider when companies differ. Read it as the uncertainty about AI agents at companies like these, not only about the claims counted. The method is the same for a vendor's score across its customers and for the buyer benchmark ([methodology](https://attify.app/methodology)).

## Schedule

Editions are made each month with the monthly verdict (on the 1st to 3rd, for the month before), once for each job type that passes. The trend shows only months that passed under the same rules.

## Empty state

Until a job type passes, the Index shows how many companies contribute and how many it needs, and no number. It never shows an invented, partial or estimated rate.

## What the Index is not

- **Not a ranking.** It ranks no vendor and compares no job types with each other.
- **Not a market estimate.** It describes the contributing companies, which chose to contribute and are not a random sample of all companies using AI agents.
- **Not a vendor's audited score.** A vendor's score is measured on its own independent customers under the program rules, published with a signed attestation. See the [registry](https://attify.app/registry).

## Changes

Any change to these rules is dated here and applies from the next edition. Earlier editions keep the rules they were made under.

| Date | Change |
|---|---|
| 2026-11-01 | First rules: contributing buyers, connected or reconciled witnessed data, at least 5 companies, none above half |

## Questions

### Where does the Attify Index get its data?

Only from companies that use AI agents and explicitly agreed to contribute their anonymous outcomes. The data is read from their own systems, or streamed by their vendor and reconciled by them. Uploaded files and vendor programs are never pooled.

### Does the Index name vendors?

No. An edition names no company, vendor, agent or customer and prints no per-company number. Named results exist only as published vendor verifications in the registry.

### Why is the Index empty?

An edition publishes only when at least 5 companies contribute for that job type and no company holds more than half of the month's claims. Until then the page says how many companies contribute and how many it needs.

### Can I compare the Index with my own agent?

Contributing buyers see the benchmark next to their own agents in their dashboard, under the same standard and tier. Comparing it with a vendor's self-reported rate mixes two different counting rules.

### Can a company stop contributing?

Yes, at any time. The withdrawal is recorded, and the company's data leaves every edition made after it.

## Sources

- Attify: [Attify Index](https://attify.app/index), read 2026-11-08
- Statistics Canada, Survey Methodology: [Confidence intervals for proportions with small expected number of positive counts estimated from survey data (Korn and Graubard, 1998)](https://www150.statcan.gc.ca/n1/en/catalogue/12-001-X19980024356), read 2026-11-09
- Journal of the American Statistical Association: [Probable inference, the law of succession, and statistical inference (Wilson, 1927)](https://doi.org/10.1080/01621459.1927.10502953), read 2026-11-08
