Amazon’s decision to clean out product reviews written by people who got stuff free in exchange for writing those reviews sounds like a good move to help consumers know what real buyers think. Amazon has been busy deleting incentivized review for a couple of months now. It turns out, however, according to a recent study by ReviewMeta, that product rating changes aren’t changing much, if at all, as reported by Tech Crunch.
Incentivized reviews involved vendors either giving away or heavily discounting products in return for a promise to write an “unbiased review.” Incentivized reviews may have created more of a perception problem than actual rating rigging, but earlier ReviewMeta research did show a difference in the reviews that resulted. The study found that incentivized reviews rated products on average 4.74 stars (out of five) compared to 4.36-star average ratings for reviews that were not incentivized.
More: Amazon has finally banned those pesky ‘incentivized’ reviews
Would a 4.74-star rating differ all that much from a 4.36-star review in your decision to buy a product? Maybe or maybe not, but the greater issue was that all those reviews, and some products, especially new ones, often had a high proportion of identified incentivized reviews. And so the appearance in those cases was that the reviews and ratings were essentially bought and paid for and therefore couldn’t be trusted. This in turn gave the overall impression that Amazon’s ratings themselves weren’t maybe all that trustworthy.
So Amazon cracked down. ReviewMeta analyzed roughly 65 million reviews of a bit more than 32,000 product categories. The researchers found that more than 500,000 reviews were deleted in the crackdown, of which 71 percent were incentivized. The average star rating for the deleted reviews was 4.75, higher than average. Just before the ban and mass deletion program began in early October the average daily product review was 4.73 stars, and on November 1, the overall average daily review dropped to 4.65 stars,
You’d think then that the product ratings would have changed and ended up significantly lower because of the dumped ‘seems-like-paid’ reviews and lower review star averages. But that didn’t happen. Why not?
It turns out Amazon wasn’t counting incentivized reviews in calculating overall product ratings, and instead utilized reviews and ratings from “Verified Purchasers” (except with brand new product where there weren’t yet many reviews that were not incentivized). So the overall ratings didn’t change much if at all. Shoppers could pretty much depend on the overall star ratings coming from people who actually paid for the products they reviewed.
But that still didn’t take care of the perception problem. Sweeping out the incentivized reviews now means that shoppers won’t see them at all, and won’t question either the product rating or ratings and reviews overall.
The sweep, which is ongoing, has also had the effect of putting a halt to companies that started incentivized review businesses. According to Tommy Noonan, ReviewMeta CTO, “It’s obviously not 100 percent perfect. It seems [Amazon has] removed a majority of the incentivized reviews and [has] pretty much put an end to more being created. They effectively killed this industry,”
