When objectivity gets befuddling: Now Consumer Reports loves the ‘hot’ new iPad

When-objectivity-gets-befuddling-Now-Consumer-Reports-loves-the-hot-new-iPad

Consumer Reports has given Apple’s latest iPad the top seat in its ranking of tablet devices, citing the screen as “the best we’ve seen” and indicating it will be revising its standards for excellence in displays based on the iPad’s retina display.

That’s quite a change from two weeks ago, when the organization triggered “heatgate” by reporting the new iPad can get up to 116 degrees Fahrenheit in places when running games — that’s just a smidge below the 120 degrees it usually takes to burn bare skin. Now, Consumer Reports says says the temperatures aren’t a problem and didn’t impact their overall rating of the device.

This isn’t the first time Consumer Reports has seemingly mis-stepped with Apple products or other consumer electronics devices. What’s behind these seemingly contradictory stances, and should everyday technology users start considering Consumer Reports in the same way they consider breathless advertisements from gear-makers — with a grain of salt?

About Consumer Reports

Consumer Reports 1960 television testing

Consumer Reports isn’t some tarted-up effort from a major publisher, nor is it a thinly-veiled marketing effort from product manufacturers hiding behind some allegedly-impartial trade group. Consumer Reports is published by the Consumers Union, a non-profit organization that conducts product research and engages in advocacy on issues it believes are in consumers’ general interest. And it’s not a fly-by-night outfit: The Consumers Union (and Consumer Reports) have been operating since 1936.

The magazine’s ad-free subscription model has always been a selling point. While typical publications that offer product reviews and assessments (including Digital Trends) often earn some portion of their revenue from advertising, Consumer Reports does not accept any advertising in any form. It also doesn’t permit commercial use of its reviews for selling products: If Consumer Reports says something nice about a product, the manufacturer can’t slap that all over its packaging or ads. When Consumer Reports reviews a product, its testers go out and buy it at retail like any other customer. They don’t accept loaners or free items from manufacturers, nor do they use any form of pricing discounts. This is all so Consumer Reports can claim to offer unbiased product reviews.

And Consumer Reports doesn’t just hand products to a reviewer and ask for their impression. Where possible, the magazine designs and conducts laboratory tests in an effort to objectively measure products’ qualities and meaningfully compare them to one another. This task is tougher with some things than others: It’s probably easier to quantify the quality differences between laundry detergents than, say, high-end audio gear. But Consumer Reports puts its money where it’s mouth is, reportedly spending over $20 million a year conducting its tests. A good portion of that money goes into its annual new car issue, published every April.

So about the new iPad…

ipad_thermal_images

Two weeks ago, Consumer Reports figuratively lit a fire under the new iPad, noting that in its tests the latest version of Apple’s tablet could reach temperatures as high as 116 degrees Fahrenheit in places when plugged in and running intensive games — their testers used Infinity Blade II. Consumer Reports also indicated the new iPad didn’t appear to be charging its battery when plugged in and running the game. Consumer Reports noted that the tablet felt “very warm but not especially uncomfortable” if held briefly. Of course, few gamers hold an iPad “briefly,” and sites were quick to note that 120 degrees is typically enough to cause a burn on bare skin. Consumer Reports wasn’t the first to look at the new iPad’s heat emissions — that seems to have been the Dutch site Tweakers. Apple’s previous iPad 2 apparently maxed out at about 111 degrees.

Of course, the technology media are generally comfortable dealing with Apple rumors, so a potentially scandalous story about an over-hot iPad from a source as reputable as Consumer Reports took off quickly. Apple even had to issue a terse statement that the new iPad was “operating well within our thermal specifications.”

However, now the new iPad is at the top of Consumer Reports latest survey of tablets, including rival devices from the likes of Samsung, Sony, Acer, Toshiba, and others. Other recommended tablets were the Pantech Element, Sony Tablet P, Toshiba Excite 10LE, and the Samsung Galaxy Tab 7.7. Consumer Reports concluded that the new iPad’s higher temperatures weren’t, in fact, a major issue.

“Responding to consumer comments on the new device, and to coverage from other reviewers, we also carried out further tests that confirmed the new iPad is warmer in its hottest spots than the iPad 2,” Consumer Reports wrote. “But we didn’t find those temperatures to be cause for concern.”

Consumer Reports further indicated the battery charging issue it previously reported occurred only when playing a “demanding” game with the screen at full brightness.

The iPhone 4s, iPad 2, and iPhone 4

iphone-4-death-grip

This isn’t the first time Consumer Reports has created a stir with its review of an Apple product. Despite giving the iPhone 4 the highest possible score in its smartphone category, Consumer Reports refused to list the iPhone 4 as “recommended” due to signal attenuation issues — the so-called “antennagate” that revealed signal strength to the iPhone can be significantly reduced depending how the device is held. Apple was forced to do significant spin control on the issue, holding a special event demonstrating that a variety of phones from competitors exhibited the same attenuation problems when held certain ways, and even offering free bumper cases for iPhone 4 owners who were experiencing problems. Ultimately, the issue that caused Consumer Reports to withhold its “recommended” rating from the iPhone 4 was an issue that was neither unique to the iPhone 4, nor a significant problem for most users — the iPhone 4 continues to be a top-selling device, and Consumer Reports continues to cite attenuation problems as by far the most significant issue with the device.

Consumer Reports does give its “recommended” rating to the iPhone 4S, claiming the device doesn’t suffer from attenuation issues like the iPhone 4. The iPhone 4S does include antenna-switching capability that enables the device to swap its transmit and receive antennas in the event of signal attenuation — for some folks, that may make the device somewhat more reliable than the iPhone 4.

Consumer Reports did give its “recommended” rating to the iPad 2. However, Consumer Reports also listed the Motorola Xoom, the original 7-inch Samsung Galaxy Tab, and the original iPad as “recommended.”

What does Consumer Reports want?

A tech-savvy reader probably looks at these recommendations and wonders if Consumer Reports is out of its mind. Is the Pantech Element (with Android Honeycomb, a 1,024 by 768-pixel display, and a sub-par camera) really in the same class as the new iPad? When Consumer Reports recommended the iPhone 4S, the organization went out of its way to point out they rated several Android phones more highly, including the Samsung Galaxy S II series, the Motorola Droid Bionic, and even the LG Thrill. Why? The primary criteria for ranking those Android devices higher was that they offered larger displays and (in the case of the Droid Bionic) “excellent keypad readability.”

These sorts of evaluations tend to fly in the face of the rigorous testing on which Consumer Reports prides itself. It’s hard to imagine the organization spends significant amounts of time designing and conducting quantifiable tests and comparing the results only to boils down to these phones are better because they have bigger screens, or the new iPad’s display is so great we lowered the scores for every other tablet’s display. The mixture of allegedly diligent and quantifiable testing and blatantly subjective qualities creates a disconnect. Consumer Reports seems to be stating an opinion, and promoting it to the world as a conclusion derived by scientific methods.

Bottom line

Suzuki Samurai failure test img

It’s important to remember that Consumer Reports is not written for experts. The publication’s primary audience is everyday consumers who are generally uninformed about a variety of high-cost products and services. Consumer Reports looks at everything from airlines and credit cards to healthcare, home appliances, cars and — yes — electronics. The magazine attempts to sum up a huge variety of products and product categories in neat snippets that can be quickly read and understood by users without much knowledge of the field. It’s not an easy task.

The tech savvy have only to look at Consumer Reports’ coverage of automobiles to understand the nature of the disconnect. Consumer Reports has been responsible for some recalls and redesigns of vehicles over the years — the most famous case probably remains Consumer Reports’ 1988 assessment of rollover problems in the Suzuki Samurai, but Consumer Reports auto testing has led to design changes in vehicles from BMW, Chrysler, Nissan, Toyota (via its Lexus brand), and others.

But car enthusiasts — you know, the “petrol heads” that might get featured on a segment of Top Gear — have taken issue with Consumer Reports’ automotive recommendations for years. Each case has its own set of details and points, but the overall disconnect will feel familiar to technology enthusiasts: Many car fans feel that Consumer Reports tests are unfair to particular models or products, or make recommendations based on subjective criteria rather than qualitative facts. Consumer Reports might look at a ton of details, and conclude one model of sedan is better than another because it can seat three adults in the rear seat — ignoring other features  (like a backup camera or availability of a six-speed manual transmission) that other people believe are more important.

We wouldn’t want to suggest buying a tablet is like buying a car, but the point is valid: It’s tempting to think of tablets (or phones, or cars) as one-size-fits-all solutions, but every individual’s situation is unique. Some folks want a 4.3-inch display on their smartphone, even if it means it no longer fits in their pocket; some folks can’t justify the expense of an iPad’s retina display on a device they just want to keep the kids entertained during their commute.

Consumer Reports has no idea what your particular situation might be. It’s testers are trying their best to address what research shows are key issues for many consumers. And, just like the crowd-sourcing of online product review, the conclusions may — or may not — apply to you.

[Consumer Reports 1960 television testing image © 1997 Consumers Union of U.S.]

The views expressed here are solely those of the author and do not reflect the beliefs of Digital Trends.

Product Review

The new iPad Mini isn’t a beauty, but it performs like a beast

Apple’s new iPad Mini has beastly performance, fluid iOS 12 software, and good battery life. It also looks like it came straight out of 2015, because the design hasn’t been changed. Here's our review of Apple’s 7.9-inch tablet.
Deals

Need a new tablet? Here are the best iPad deals for March 2019

In the wide world of tablets, Apple is still the king. If you're on team Apple and just can't live without iOS, we've curated an up-to-date list of all of the best iPad deals currently available for March 2018.
Mobile

Here are the best iPad Pro keyboard cases to pick up with your new tablet

The iPad Pro range can double as laptops, but they do need proper keyboards to fill in effectively. Thankfully, there are loads to choose from and we rounded up the best iPad Pro keyboard cases right here.
Mobile

The 2018 iPad is often the best iPad for most people — and now it’s only $250

Apple may have recently taken the wraps off of a new iPad Air and iPad Mini, but it's still the standard iPad that is best for most people. Now, the standard iPad has gotten a pretty significant discount -- it's down from $330 to $250.
Movies & TV

Apple’s next big event is minutes away: Here’s what you can expect

Apple's next big event takes place on March 25 in Cupertino, California. The company is expected to make several announcements related to its services, including Apple TV, so follow our guide to get ready for the big event.
Deals

The excellent Apple iPad gets even deeper price cuts on Amazon

The humble iPad from 2018 is still one of the best tablets around -- and a solid choice for most people. Amazon has seen some great price drops for these tablets recently, and now you can own an iPad for even less than before.
Computing

Apple’s 4K 21.5-inch iMac is now $200 off if you pre-order it

Apple's new iMacs are now available and if you pre-order one from B&H you can get the midrange version for $200. That's a near 20-percent saving on one of the most competitive configurations.
Apple

Apple March 2019 Event Coverage

Apple’s next event will take place March 25 at the Steve Jobs Theater in Cupertino, California at 10 a.m. PT. We’ve got a handy guide on how to watch, but don’t expect to see any new iPads, iMacs, or AirPods at the show, all of…
Mobile

Apple Card is a credit card you can sign up for and start using with your iPhone

Apple is getting into the credit card business. Apple Card is a credit card you can sign up for directly on your iPhone, and it doesn't have fees. There's a lower interest rate and you can even get Daily Cash from all purchases.
Apple

Apple’s Netflix competitor Apple TV Plus headlines a barrage of new services

Apple's It's Show Time event in Cupertino, California, is officially over, with the tech giant announcing a new video-streaming service to compete with Netflix and Amazon, a new way to subscribe to paid news outlets, and more.
Mobile

Check out 22 of the best iPhone 7 cases and covers for your shiny new phone

The iPhone 7 might be attractive, but it’s not rugged. To keep your device in pristine condition, you really need to think about proper protection. That's why we've rounded up some of the best iPhone 7 cases and covers available.
Movies & TV

Apple TV Plus streaming service recruits Hollywood A-listers to take on Netflix

Apple has an ambitious slate of original programming it has been developing, and now we know where we'll be able to see it. Apple confirmed its new subscription service, Apple TV Plus, and revealed some of the projects it will air.
Mobile

Apple's iOS 12.2 brings support for Apple News Plus and new AirPlay 2 features

After months of betas, the final version of iOS 12 is here to download. The latest OS comes along with tons of new capabilities, from grouped notifications to Siri Shortcuts. Here are all the features you'll find in iOS 12.
Computing

Get a new 2018 Apple MacBook Air for $1,000 with Amazon’s latest sale

Online retailer Amazon is currently running a discount on select models of the MacBook Air 2018. You can bring one home starting at $1,000, a full $200 off the usual selling price.