Gaming the system: How reliable are app store ratings?

Can you trust the user ratings in any of the major app stores? In a word, no. App ratings are far from reliable and buyers would do well to exercise some caution because many developers are manipulating the system to boost their wares.

Popularity contest

Since their inception, all of the major smartphone app stores have suffered from the same set of problems. They offer an ever-growing inventory of apps and games, many of them extremely similar, many of them extremely poor. How do they allow users to browse what’s on offer? How do they highlight the best-quality apps? Disappointingly, they have all opted for exactly the same format: They simply promote lists of the best apps and games in various categories.

The more popular an app is the higher up the list it will appear. The higher up the list it appears, the more people will download it and the more popular it will become. This is a positive feedback loop. For developers, it is a complete nightmare, because breaking into that front page list is not easy. Thousands of perfectly good apps and games slip into obscurity every year. Getting your app into a top 25 list is the equivalent of a huge marketing spend – thousands of your target customers will see your product.

You may think that a setup that promotes whatever is popular with consumers and allows them to write honest reviews and give ratings provides a level playing field for developers. That’s the theory, but in practice a relatively small group of publishers dominates the top spots. The financial success of a new smartphone game could very well depend on breaking into the top 25, and so it should come as no surprise that many developers are looking for ways to manipulate the system. Sadly the integrity of those user ratings is being eroded.

Rigging the system

The most pernicious practice is using bot farms to boost app ratings. Back in February, Apple was forced to publicly condemn developers who agreed to pay cash to marketing firms in return for higher placement in the App Store. This was largely a reaction to the much publicised post at Touch Arcade in which a developer discussed details of a marketing firm’s offer to secure a top 25 place for their app in return for $5,000.

Interestingly, app downloads have declined in recent months, which could be due to a clamp down on bot farms; we simply don’t know. The trouble is that some of these marketing companies use real people to boost app ratings. They pay small fees to users for downloading specific apps. How can Apple staff really identify the legitimate downloads in this scenario? The simple answer is that they can’t.

This problem is not confined to Apple’s App Store; exactly the same approach will work in the Windows Phone Marketplace, BlackBerry App World or Google Play. Developers and publishers with deep pockets can buy their way into the top spots in the charts and then reap the rewards of that positive feedback loop. This makes it extremely tough for any new developers to break in, particularly if they have a limited budget.

Directly manipulating the charts by downloading copies of your own product is the most obvious way to cheat the system, and it only makes sense financially because of the peculiarities of the app store charts. There are a number of cheaper ways to game the system, and you’ll find a lot of apps and games are now adopting them.

Freemium games have really taken off in recent months and they’ve been especially dominant on the Android platform. Typically, you get full access to the core game for free but the gameplay progresses at an agonisingly slow pace. Players have the option of buying game currency in order to unlock new content and advance.

Many of these games adopted a clever technique to boost their ratings – they offer in-game currency or rewards in return for a five-star review from the player. Sometimes they’ll even make your five-star review one of your game tasks. The same approach is used to gain traction on Facebook and other social media. If you “Like” the game page on Facebook, you’ll earn an in-game reward.

This isn’t, strictly speaking, cheating. I’ve yet to come across a game that doesn’t allow you to ignore the request for a five star review, but it is obviously “gaming” the system.

It’s also common practice for games or apps to prompt you repeatedly to review them from within the app. If you want that annoying pop-up to quit appearing every time you open the app, then you’ll go ahead and review it. The developer will generally include a persuasive statement about how five star reviews will allow them to improve the app in future.

You may also have seen the cross-promotion techniques that publishers often use to help them dominate the charts. They’ll include ads for their other apps or games in the app that you are using, usually a splash screen when you load it up or shut it down. Some of them take it further and offer rewards or money off incentives if you download their other titles.

Who can you trust?

You can’t blame developers and publishers for trying to secure good chart positions. Unfortunately for many, integrity can cost you money.

The real responsibility here should rest with the platform operators but what is their incentive to spend time and money on stricter controls? Apple already applies some quality control, while Google is at the opposite end of the spectrum because developers are allowed to release pretty much anything they like on Android. Would they stand to gain much by policing app ratings further? Probably not, so they’ll only take it so far.

The bottom line is that we, the users, have to take responsibility for what we download. Don’t trust app ratings alone. Do your research, read reviews on the growing number of app related websites and choose carefully.

Check out our guides to the best Android apps and best iPhone apps.

Popularity contest

Rigging the system

Who can you trust?

Editors' Recommendations