The domain of AI has quickly gone from seeking answers from a chatbot to getting multi-step web-based work done for you, like booking restaurant tables, adding stuff to your Amazon cart, and performing deep back-and-forth research. But it looks like the answers you get, especially when it comes to online shopping, depend on the AI chatbot you are using.
“Platforms disagree on brand recommendations for 61.9% of queries,” says an analysis conducted by BrightEdge. The company assessed tens of thousands of shopping-related questions that were provided to ChatGPT, Google’s AI Overviews, and Google’s AI Mode in Search.
Do I trust one, or the other?
The brand recommendations offered by each AI product were wildly different. The disparity runs so deep that in only 17% of cases do all three AI platforms agree on the same brand and recommend it to the user. Below are some of the key findings from the research:
- 61.9% of queries return conflicting brand recommendations depending on the platform.
- Brands from all three platforms show up in just 33.5% of queries.
- In 4.6% of cases, none of the platforms recommend any brands at all.
- Google AI Overviews lead with brand mentions in 36.8% of queries.
Then there’s the conflict in the recommendations. According to BrightEdge’s testing, these three AI platforms agree on brand picks for only 17% of the queries. In a nutshell, for the same questions, you will likely see ChatGPT recommending a different company compared to Google AI Overviews and the AI mode.
If you’re seeking healthcare-related advice, there’s a stunning 68.5% disagreement in the brands recommended by the three AI tools, followed by 62.1% disparity for education-related queries, and 57.9% variation for questions falling within the finance industry’s domain.
This is not good news for shoppers
In a nutshell, what we are seeing is that chatbots take a different approach when it comes to shopping-related queries. That means you will have to jump between at least two AI products to get a proper idea of all the brands worth considering if you’re planning to buy, for example, a shirt or jeans.
ChatGPT, as per the research, will suggest fewer brands, while Google’s AI will name-drop twice as many brands for you to pick from. That essentially means instead of going with a “one AI shopper fits all” approach, you will have to cross-check with two different AI assistants to get a realistic idea.
Some would prefer ChatGPT, as it throws 2.5x fewer brand names in your face compared to Google’s AI, which means you are less spoiled for choice. On the flip side, one can argue that the more knowledge you get, the higher the chances of making the right pick. Take a look at the research stats:
- On average, ChatGPT recommends 2.37 brands per query.
- Google AI Overview serves up 6.02 brands per query—over twice as many as ChatGPT.
- Google AI Mode lags behind with just 1.59 brands per query.
- ChatGPT doesn’t mention any brands in 43.4% of queries.
- Google AI Overviews skip brand mentions in only 9.1% of queries.
Google’s AI tools are more transparent
Unfortunately, things are not as straightforward as they seem. Online shopping entails a lot of research, especially through reviews from buyers or websites where experts have tested a product and given their verdict. And that’s where the tables turn on their head.
ChatGPT gives out more brand mentions, but far fewer citations. On average, it mentioned 2.37 brands per query, while giving out 0.73 citations. Google AI Mode has 2.4x more citations compared to the number of brands it recommends, averaging at more than 14 citations for six brand names in the answer it provides.
It would seem Google’s AI mode performs the best if you are looking for more transparency into how brands or a certain product are being recommended to you. It gives six times as many citations as it mentions the name of brands. But as a shopper, it would make sense to cross-check across ChatGPT and Google’s AI tools in order to make an informed decision. It’s not the most convenient route, but at least it’s less prone to errors.
