Android beats iPhone 4 in web test (that may be totally wrong)

A study by web optimization company Blaze shows that websites load 52 percent faster when using phones with Google’s Android 2.3 operating system than Apple’s iPhone 4 loaded with iOS 4.3. Both of the latest updates for Android and iOS focused on improvements to their JavaScript engines and faster browsing.

Good news for Android, right? Not necessarily. It’s possible that the Blaze’s entire test is entirely flawed.

First, the test: Blaze conducted what it calls the “largest ever research study of smart phone browser performance,” carried out in order to “measure the true mobile browsing experience.” That included 45,000 page loads of websites of Fortune 1000 companies using a Nexus S and an iPhone 4. Blaze analyzed the results to discover that Android loaded websites a “whopping” 52 percent faster on average than the iPhone 4.

According to the study, Android 2.3 had a median load time of 2.144 seconds, compared to iOS 4.3’s median load time of 3.254 seconds. For websites optimized for mobile devices, Android was only 3 percent faster. But for non-mobile websites, Android clocked in at 59 percent faster than the iPhone.

“”We were very surprised by the results,” said Blaze CTO and co-founder Guy Podjarny, “We assumed that it would be closer race and that the latest JavaScript speed improvements would have a more material impact on performance. The fact that Android beat iPhone by such a large margin was not expected.”

The problem with Blaze’s test, says The Loop‘s Jim Dalrymple, is that “the study is flawed.”

In the Blaze study, the speed measurement “was done using the custom apps which use the platform’s embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone.”

Unfortunately for Android, the UIWebView didn’t actually receive any of the performance-enhancing updates in the 4.3 upgrade that the Safari browser did receive. That means the browser Blaze used in the test didn’t have any of the JavaScript speed improvements.

“Obviously someone is looking to make a mountain out of a molehill,” analyst Michael Gartenberg tells The Loop. “It’s not an apples to apples test.”

So is Android 2.3 really faster than iPhone 4.3? Maybe — but this test shouldn’t be the final word on the matter.

Update: Apple spokesperson Trudy Muller has issued a statement to Ars Technica about the test, which essentially echos The Loop‘s Dalrymple and analyst Gartenberg. It reads: “Their testing is flawed because they didn’t actually test the Safari web browser on the iPhone. Instead they only tested their own proprietary app which uses an embedded web viewer that doesn’t take advantage of Safari’s web performance optimizations. Despite this fundamental testing flaw, they still only found an average of a second difference in loading web pages.”

Editors' Recommendations