Do AI answers repeat vendors' comparison pages?
I asked ChatGPT, Claude, Perplexity and Google AI Mode 80 times which of two SaaS tools is better. Vendors' own comparison pages shaped many answers.
Key findings
When an AI answer cited a vendor's own comparison page, it repeated that vendor's arguments 84% of the time, compared with 28% when it didn't. No answer named a flat winner, and comparison pages that say when the competitor is the better pick showed up in more than twice as many answers as pages that don't.
- No AI tool picked a flat winner. All 80 answers gave a "pick X if, pick Y if" verdict, and 17 of them named a default pick while still giving cases for the other tool.
- A third of the sources came from the two vendors. Of the 417 sources the answers cited, 21% were the vendors' own sites and 12% were their comparison pages. Other software companies were the biggest group at 36%.
- Citing a vendor's comparison page went with repeating it. Answers that cited a vendor's comparison page repeated that vendor's own arguments in 36 of 43 cases (84%), compared with 28 of 101 (28%) when they didn't.
- Pages that admit when the competitor wins were cited more. Comparison pages that name buyers the competitor suits better appeared in 42% of answers about their matchup, compared with 18% for pages that don't. With 18 vendors, treat this as a pattern, not proof.
- One answer in five had a wrong price. 75% of the 240 prices quoted matched the vendor's current price, but 17 of 80 answers included at least one outdated, wrong-plan or wrong price, some of them taken from a vendor's own stale comparison page.
- Claude was the only tool that regularly warned about bias. 19 of 20 Claude answers said comparisons are written by interested parties, compared with 5 of 20 for Perplexity and none for ChatGPT or Google AI Mode.
Results
80 answers were scored, 20 from each AI tool.
| ChatGPT | Claude | Perplexity | Google AI Mode | All 80 | |
|---|---|---|---|---|---|
| Named one flat winner | 0/20 | 0/20 | 0/20 | 0/20 | 0/80 |
| Gave a default pick, with cases for the other | 6/20 | 1/20 | 10/20 | 0/20 | 17/80 |
| Cited a vendor's comparison page for the pair | 7/20 | 7/20 | 8/20 | 12/20 | 34/80 (42%) |
| Repeated at least one vendor's own arguments | 7/20 | 15/20 | 10/20 | 18/20 | 50/80 (62%) |
| Warned that comparisons are written by interested parties | 0/20 | 19/20 | 5/20 | 0/20 | 24/80 (30%) |
| Stated at least one price | 15/20 | 19/20 | 9/20 | 10/20 | 53/80 |
| Had an outdated, wrong-plan or wrong price | 2/20 | 8/20 | 1/20 | 6/20 | 17/80 (21%) |
| Tailored the answer to Canada | 2/20 | 3/20 | 4/20 | 0/20 | 9/80 |
The two runs of each question gave the same type of verdict 37 times out of 40.
What the answers cited
The 80 answers cited 417 sources. Other software companies were the biggest group (36%), ahead of the two vendors' own sites (21%) and their comparison pages (12%). "Other software companies" means tools outside the pair writing their own comparisons, like attribution tools reviewing Northbeam vs Triple Whale, plus some unrelated companies' content marketing posts.
| Share of each tool's sources | ChatGPT | Claude | Perplexity | Google AI Mode |
|---|---|---|---|---|
| The two vendors' own sites (pricing, docs, blog) | 68% | 6% | 33% | 9% |
| The two vendors' comparison pages | 9% | 8% | 20% | 14% |
| Other software companies | 8% | 55% | 18% | 33% |
| YouTube, Reddit and forums | 0% | 0% | 0% | 23% |
| Sources cited in 20 answers | 75 | 164 | 45 | 133 |
- ChatGPT mostly cited the vendors' own pricing and docs pages.
- Claude leaned on other software companies' comparison posts, and was also the tool most likely to warn that they're biased.
- Google AI Mode was the only tool citing YouTube and Reddit, and it cited a vendor's comparison page in the most answers (12 of 20).
- Perplexity shows few inline citations (45 across 20 answers), so its shares are less comparable.
Vendor comparison pages showed up most in smaller categories. Omnisend's Klaviyo page was cited in 7 of 8 answers, and Rippling's Gusto page in 5 of 8. In big categories they barely registered: HubSpot's and Salesforce's pages were cited in 0 of 8 answers each, and Ahrefs' and Semrush's in 1 of 8 each.
Vendor framing
62% of answers (50 of 80) repeated at least one vendor's own arguments, and whether the answer cited that vendor's comparison page made a big difference. Taking each answer and each vendor with a comparison page (144 combinations), answers that cited the vendor's page repeated its arguments in 36 of 43 cases (84%). Answers that didn't cite it repeated them in 28 of 101 (28%).
Some examples:
- Google AI Mode opened its AirOps vs Profound answer with AirOps' own decision rule, almost word for word: "your choice depends on whether your bottleneck is tracking data or executing content." AirOps' bottleneck line turned up in 7 of the 8 AirOps vs Profound answers.
- Google AI Mode stated Okendo's claim that Yotpo gates CSS styling behind expensive contracts, and Yotpo's in-email reviews pitch, as plain fact. Both vendors' pages were its top two sources.
- ChatGPT's Omnisend vs Klaviyo price table matched Omnisend's own comparison table row for row. The numbers were accurate, but Omnisend's "cheaper at every tier" framing became the answer's price comparison.
- Google AI Mode repeated Omnisend's headline that it "saves brands up to 35%" as fact. Perplexity, answering the same question, warned that "a blanket 'X% cheaper' claim can therefore be misleading".
Claude was the exception on attribution. It warned about vendor bias in 19 of 20 answers, sometimes naming the page ("that includes Triple Whale's own comparison page"). It still repeated vendor arguments in 15 of 20 answers, usually labelled as the vendor's claim.
Pages that say when the competitor is the better pick
Comparison pages that name buyers or uses where the competitor is the better pick were cited in more than twice as many answers as pages that don't, and their arguments were repeated more often too.
| Vendor's comparison page | Vendors | Cited in | Arguments repeated in |
|---|---|---|---|
| Names buyers or uses where the competitor is the better pick | 9 | 30 of 72 (42%) | 40 of 72 (56%) |
| Doesn't, or only gives token praise | 9 | 13 of 72 (18%) | 24 of 72 (33%) |
The nine that concede: Omnisend, PostHog, AirOps, Profound, ZoomInfo, Triple Whale, Okendo, Rippling and Gusto. The nine that don't: Gorgias, Zendesk, Amplitude, HubSpot, Ahrefs, Semrush, Salesforce, Yotpo and Apollo.
Omnisend's page, for example, says Klaviyo suits more advanced marketing, and ChatGPT's answer opened with the same split.
Treat this as a pattern, not proof:
- It's 18 vendors, and the big categories (HubSpot vs Salesforce, Ahrefs vs Semrush) all fall in the "doesn't" group. Big categories have far more third-party content competing for citations.
- Within the three pairs where one vendor concedes and the other doesn't, the result is mixed. PostHog (concedes) beat Amplitude 4 citations to 1, but Apollo (doesn't) beat ZoomInfo 2 to 0, and Yotpo edged Okendo 3 to 2.
- Counting Okendo's, Rippling's and Gusto's narrow concessions as "doesn't", the gap shrinks but holds: 42% vs 24% cited.
The safest reading: being fair to the competitor didn't cost these vendors citations, and in this sample it went with more of them.
Prices and currency
Three in four prices were correct (180 of 240), and one answer in five had at least one outdated, wrong-plan or wrong price (17 of 80).
| Price claims | ChatGPT | Claude | Perplexity | Google AI Mode | All |
|---|---|---|---|---|---|
| Correct (matches the vendor's current price) | 75 | 60 | 29 | 16 | 180 (75%) |
| Third-party estimate (vendor publishes no price) | 1 | 17 | 1 | 4 | 23 (10%) |
| Outdated (a price the vendor used to charge) | 2 | 12 | 1 | 7 | 22 (9%) |
| Can't be checked | 3 | 6 | 0 | 1 | 10 (4%) |
| Wrong plan | 1 | 2 | 0 | 0 | 3 (1%) |
| Matches nothing | 0 | 0 | 0 | 2 | 2 (1%) |
| Total | 82 | 97 | 31 | 30 | 240 |
Where the outdated prices came from:
- Vendors' own stale pages. Perplexity quoted Semrush at $139.95, taken from Semrush's own comparison page ("Data as of May 2025"). Semrush's public plans have since changed.
- A competitor's comparison page. Google AI Mode gave Profound's old $99 and $399 plans, which Profound removed in mid-September 2026 but AirOps' comparison page still lists.
- Old agency and review posts. Claude gave Salesforce Enterprise at $165 and HubSpot Starter at $15, both from earlier price lists.
Google AI Mode also made the gap between two tools look bigger than it is. It put Klaviyo at about $240/month for 10,000 contacts next to Omnisend's $130, while Klaviyo's own calculator says $150.
Currency: every price was in US dollars, written either as a bare dollar sign (163) or as "USD" (77). None were in Canadian dollars, even though HubSpot shows Canadian prices to visitors from Canada. Claude noted once that Omnisend "bills only in USD, which matters if you're paying in Canadian dollars."
Other findings
- Location: 9 of 80 answers were tailored to Canada without being asked, including two Perplexity answers that said "Since you're in Victoria" and leaned toward Rippling for Canadian payroll. Even ChatGPT's unpersonalized temporary chats and Claude's incognito chats raised Canada on Gusto vs Rippling.
- No ads: none of the 20 Google AI Mode answers showed a Sponsored block.
What this means
- If you publish a comparison page, expect AI tools to read it and repeat parts of it, especially in a smaller category with less third-party content competing for the same citations.
- Saying who should pick the competitor didn't cost these vendors anything. In this sample, pages that did it were cited in more answers and repeated more often.
- Every answer framed the choice as "pick X if, pick Y if". A comparison page written the same way gives AI tools something they can quote.
- Stale numbers on a comparison page get passed on, including numbers about the other vendor. Date your figures and update the page when either side changes its pricing.
Limitations
- Small sample. 80 answers, 10 matchups and one day, so treat the numbers as directional.
- One location and one set of accounts. The study ran from Victoria, BC, and 9 answers were tailored to Canada. Google AI Mode was signed in.
- Patterns, not cause and effect. Big categories have more third-party content, which may explain some of the gap between pages that concede and pages that don't.
- Citation counts aren't like for like. Perplexity and ChatGPT show fewer inline citations than Claude and Google AI Mode, so per-tool source shares should be compared with care.
- What counts as a comparison page. Counting any vendor page that names the competitor, 34 of 80 answers cited one. Counting only the saved comparison page (or an alias of it), it's 33.
- Rules added during scoring. A few rules were added while scoring, each recorded with its effect. Two move the numbers: Gorgias' prices with the default "Add AI Agent" toggle switched off were counted as correct (without that ruling, 70% of prices would be correct instead of 75%), and Semrush's retired Pro and Guru prices were counted as outdated, even though Semrush still shows them to logged-in users.
Study data
Download the scored answers (CSV), with one row per answer: the AI tool, matchup, run, verdict, vendor framing, bias warning, comparison pages cited, price counts and notes. Download the price claims (CSV), with one row per price an answer stated and its verdict.
Vendor comparison pages saved as ground truth:
- Triple Whale vs Northbeam
- Omnisend vs Klaviyo
- Gorgias vs Zendesk and Zendesk vs Gorgias
- Okendo vs Yotpo and Yotpo vs Okendo
- PostHog vs Amplitude and Amplitude vs PostHog
- AirOps vs Profound and Profound vs AirOps
- Ahrefs vs Semrush and Semrush vs Ahrefs
- HubSpot vs Salesforce and Salesforce vs HubSpot
- Apollo vs ZoomInfo and ZoomInfo vs Apollo
- Gusto vs Rippling and Rippling vs Gusto
The study archive (protocols, rubric, scoring rulings, ground-truth captures, raw answers with checksums, screenshots and analysis scripts) is available on request. Email aliya@conrad.studio.
Methodology
Conrad Studio original research. All answers were collected on October 29, 2026.
Matchups
10 B2B SaaS matchups, some of them ecommerce software. Each pair was asked in the order people search it most often (Semrush US data, October 29, 2026). 18 of the 20 vendors publish an indexable comparison page against the other one; only Northbeam and Klaviyo don't.
| Matchup | US monthly searches | Vendors with an indexable comparison page |
|---|---|---|
| Northbeam vs Triple Whale | 180 | Triple Whale only |
| Omnisend vs Klaviyo | 2,080 | Omnisend only |
| Gorgias vs Zendesk | 320 | Both |
| Okendo vs Yotpo | 180 | Both |
| PostHog vs Amplitude | 130 | Both |
| AirOps vs Profound | 120 | Both |
| Ahrefs vs Semrush | 22,000 | Both |
| HubSpot vs Salesforce | 6,400 | Both |
| Apollo vs ZoomInfo | 870 | Both |
| Gusto vs Rippling | 1,600 | Both |
Question and tools
Each AI tool was asked the same question, "[A] vs [B]: which is better?", twice per matchup in a fresh chat each time. That's 10 matchups, 4 tools and 2 runs, or 80 answers. The first run of every matchup was asked before any second run.
- ChatGPT: logged-in Team account, temporary chat set to "Unpersonalized", default model (reported as gpt-6, "Instant"). It searched the web on every answer.
- Claude: incognito chats with every connector switched off, default model (Opus 5.5) with web search.
- Perplexity: Pro account, default Search mode, memory switched off. A first set of 20 Perplexity answers was set aside because three of them referred to earlier searches on the account, and all 20 questions were asked again with memory off.
- Google AI Mode: signed-in Google account.
All four tools were used from the same Chrome browser in Victoria, BC, without a VPN. Every setting changed for the study was switched back afterwards.
Ground truth
Before scoring, each vendor's pricing page (Canadian and US views) and its comparison page against the other vendor were saved. Every answer was saved word for word with a checksum, and a screenshot was taken at the time it was collected.
Scoring
- Verdict: "pick X if, pick Y if" (a case for each tool), default pick (names an overall winner but says when the other is better), or single winner.
- Vendor framing: an answer repeats a vendor when it uses a distinctive claim or close wording from that vendor's comparison page, not a fact both sides state. "Zendesk is built for larger teams" doesn't count, but "per-seat pricing protects you from soaring volume costs" does. Attributed repeats ("Profound says...") count too.
- Bias warning: the answer says comparisons, or a specific comparison page, are written by an interested party.
- Sources: every URL an answer cited was classified by type, such as the vendors' own sites, their comparison pages, other software companies, review sites, publishers, agencies, YouTube and Reddit.
- Prices: every price was checked against the saved pricing pages and labelled correct, outdated (a price the vendor used to charge), wrong plan, third-party estimate (the vendor publishes no price), can't be checked, or matches nothing.
Four scorers worked from the same written rubric. A second scorer re-scored 16 answers (4 per tool) without seeing the first scores, and an independent checker recomputed every number in this report from the raw files and spot-checked 12 more answers.
- Verdicts, citations and bias warnings: the two scorers agreed on all 16 answers.
- Vendor framing: 82% agreement (Cohen's kappa 0.65). Every disagreement was the first scorer saying "no repeat", so the 62% repeat rate is more likely an undercount than an overcount.
- Prices: 74% agreement on matched claims, mostly because the scorers split some claims differently (a base price and a per-user price as one claim or two).
Research by Aliya Conrad
