Analyst reports and review sites vs. audited outcomes

  • Comparison
  • Published
  • Reviewed
  • By

Analyst reports score vendors on criteria the analyst sets, from questionnaires, briefings, demos and customer references; review sites collect moderated opinions from users. Both help build a shortlist. Neither is a count of one agent's outcomes with a denominator, a period and an interval.

What they say about their methods#

  • Forrester Wave: the analyst drafts criteria and invites vendors; participating vendors answer a questionnaire, give briefings and demos, and name reference customers; the analyst scores against defined scales and weights; vendors see their scorecards and get a courtesy preview about five days before publication; the process runs about 18 weeks; Forrester reserves the right to evaluate vendors that do not participate (Forrester).
  • G2 reviews: filtered automatically and checked manually; reviewer tags include validated reviewer, current user and incentivized; incentives are capped at $100 a review and labelled; employees of the product's company and of direct competitors may not review; G2 says it never edits a review's content (G2).

Side by side#

Analyst reportReview siteAudited outcomes (Attify)
UnitThe vendor and productA reviewer's experienceOne agent's claimed outcomes, per customer
EvidenceVendor-supplied answers, demos, chosen referencesOpinionEvery record of the period in the customer's systems
NumberA position or score on the analyst's scaleA star ratingAn audited rate with a 95% interval
Who chooses the customersThe vendor names referencesReviewers choose to writeThe vendor invites them; then every invited customer counts, or Attify draws the ones that count with a logged seed

Using all three#

Reports and reviews for the market map and shortlist; a pilot for fit; verification for what the deployed agent achieves.

Questions

Do G2 reviews say how well an AI agent performs?

They say what reviewers experienced and think. They have no denominator, so they cannot give a rate with an interval.

Is a vendor ranked well in an analyst report a safe choice?

It is a well-evaluated vendor on the analyst's criteria. Whether its agent's outcomes hold up with your customers is a separate, measurable question.

Sources

  1. Forrester. Forrester Wave methodology. Read 8 Nov 2026.
  2. G2. Community Guidelines. Read 8 Nov 2026.