AI agent verification
AI agent verification is checking the outcomes an AI agent claims (a resolved support conversation, a booked sales meeting, a booked appointment) against what the customer's own records show happened afterwards. A result states the audited rate next to the claimed rate, the rule both were counted under, and how certain the number is.
What it involves#
- A published definition of the outcome for the job, so the claim and the check use the same words. Attify publishes one standard per job type: support resolution, sales meetings, bookings.
- The claim as the vendor makes it, from the vendor's own flag where one exists.
- Evidence checked outside the vendor: the customer's help desk, CRM, calendar or booking system, or the vendor's own stream once the customer has reconciled it against them. See data provenance.
- Checks that ask what happened next: did the customer come back, did a person have to step in, did the meeting or appointment take place.
- Stated uncertainty: a 95% confidence interval and the evidence tier of every number.
What it is not#
Each answers a different question. The comparisons hub sets them side by side, from each method's own documentation.
Questions
How is AI agent verification different from an AI benchmark?
- A benchmark tests a model or agent on a fixed set of test tasks, before deployment. Verification checks what a deployed agent actually did for a real customer, in that customer's own records.
How is it different from an AI audit or certification?
- Certifications such as AIUC-1, SOC 2 or ISO/IEC 42001 check controls: security, safety, how AI is managed. Verification checks outcomes: whether the issues the agent claimed to resolve, or the meetings and appointments it claimed to book, held up.
Who can verify an AI agent?
- The buyer can check its own deployment, and a vendor can open a verification program that audits its agent across its customers. In both cases the data should come from the customer's systems, not the vendor's dashboard.