ChatGPT ads are overpriced and considerably less effective than paid search, according to an early-stage research paper.
Analysis of the performance of ChatGPT ads in the US showed that each click brought in an average revenue of $0.89, compared with $2.45 for a Google Search ad click.
Comparing purchases from within the same shops showed that visitors from ChatGPT ads buy less often, too, converting at 0.37%, compared with 1.17% for Google Search ads.
In fact, ChatGPT advertising was the only channel that underperformed direct traffic in all six of the metrics tracked by the researchers. The six metrics were: conversion rate, order value, revenue per session, bounce rate, pageviews per session, and duration per session. The other channels measured were: organic search, paid search, and organic ChatGPT.
According to the researchers’ calculations, a ChatGPT ad should cost brands approximately $0.26 per click in order to deliver the same kind of returns as a Google Search ad. But in reality, the largest publicly known account paid $1.72 per click.
On the plus side, the introduction of ads did not show signs of significantly diminishing the performance of ChatGPT as a product. A broader dataset, looking at countries beyond the US, showed that organic referrals from ChatGPT declined only 6%-8% after the introduction of ads. There was no observed change in the conversion rates among organic referrals, either, which the researchers interpret as a sign that users did not trust ChatGPT’s recommendations any less as a result of the introduction of ads.
Authors Maximilian Kaiser (Vienna University of Economics and Business) and Christian Schulze (Frankfurt School of Finance and Management) essentially used last-click attribution to track the effects of the ads, so all the usual caveats with regards to that method apply. They also write that their findings concerning ChatGPT ad performance should be treated as descriptive, rather than causal, because the analysed brands self-selected by choosing to advertise on the platform, rather than being picked randomly.
Only 102 of the 387 brands in the dataset had complete financial tracking, too, meaning that the insights into the effectiveness of ChatGPT ads are based on a relatively small sample. And only 18 of those 102 brands recorded any sales from ChatGPT ads, with one advertiser making up 66% of all the ad-driven purchases attributed to the platform.
The researchers hypothesise that the poor performance of ChatGPT’s ads compared with those within Google Search stems from the former’s design, with sponsored listings separated from the answer provided by the bot.
Can Advertising Fund Free AI? Evidence from ChatGPT’s International Advertising Rollout is still awaiting peer review. The data used by the study was provided by Grips Intelligence, for which one of the authors provides consulting work.
