When objectivity gets befuddling: Now Consumer Reports loves the ‘hot’ new iPad

When-objectivity-gets-befuddling-Now-Consumer-Reports-loves-the-hot-new-iPad

Consumer Reports has given Apple’s latest iPad the top seat in its ranking of tablet devices, citing the screen as “the best we’ve seen” and indicating it will be revising its standards for excellence in displays based on the iPad’s retina display.

That’s quite a change from two weeks ago, when the organization triggered “heatgate” by reporting the new iPad can get up to 116 degrees Fahrenheit in places when running games — that’s just a smidge below the 120 degrees it usually takes to burn bare skin. Now, Consumer Reports says says the temperatures aren’t a problem and didn’t impact their overall rating of the device.

This isn’t the first time Consumer Reports has seemingly mis-stepped with Apple products or other consumer electronics devices. What’s behind these seemingly contradictory stances, and should everyday technology users start considering Consumer Reports in the same way they consider breathless advertisements from gear-makers — with a grain of salt?

About Consumer Reports

Consumer Reports 1960 television testing

Consumer Reports isn’t some tarted-up effort from a major publisher, nor is it a thinly-veiled marketing effort from product manufacturers hiding behind some allegedly-impartial trade group. Consumer Reports is published by the Consumers Union, a non-profit organization that conducts product research and engages in advocacy on issues it believes are in consumers’ general interest. And it’s not a fly-by-night outfit: The Consumers Union (and Consumer Reports) have been operating since 1936.

The magazine’s ad-free subscription model has always been a selling point. While typical publications that offer product reviews and assessments (including Digital Trends) often earn some portion of their revenue from advertising, Consumer Reports does not accept any advertising in any form. It also doesn’t permit commercial use of its reviews for selling products: If Consumer Reports says something nice about a product, the manufacturer can’t slap that all over its packaging or ads. When Consumer Reports reviews a product, its testers go out and buy it at retail like any other customer. They don’t accept loaners or free items from manufacturers, nor do they use any form of pricing discounts. This is all so Consumer Reports can claim to offer unbiased product reviews.

And Consumer Reports doesn’t just hand products to a reviewer and ask for their impression. Where possible, the magazine designs and conducts laboratory tests in an effort to objectively measure products’ qualities and meaningfully compare them to one another. This task is tougher with some things than others: It’s probably easier to quantify the quality differences between laundry detergents than, say, high-end audio gear. But Consumer Reports puts its money where it’s mouth is, reportedly spending over $20 million a year conducting its tests. A good portion of that money goes into its annual new car issue, published every April.

So about the new iPad…

ipad_thermal_images

Two weeks ago, Consumer Reports figuratively lit a fire under the new iPad, noting that in its tests the latest version of Apple’s tablet could reach temperatures as high as 116 degrees Fahrenheit in places when plugged in and running intensive games — their testers used Infinity Blade II. Consumer Reports also indicated the new iPad didn’t appear to be charging its battery when plugged in and running the game. Consumer Reports noted that the tablet felt “very warm but not especially uncomfortable” if held briefly. Of course, few gamers hold an iPad “briefly,” and sites were quick to note that 120 degrees is typically enough to cause a burn on bare skin. Consumer Reports wasn’t the first to look at the new iPad’s heat emissions — that seems to have been the Dutch site Tweakers. Apple’s previous iPad 2 apparently maxed out at about 111 degrees.

Of course, the technology media are generally comfortable dealing with Apple rumors, so a potentially scandalous story about an over-hot iPad from a source as reputable as Consumer Reports took off quickly. Apple even had to issue a terse statement that the new iPad was “operating well within our thermal specifications.”

However, now the new iPad is at the top of Consumer Reports latest survey of tablets, including rival devices from the likes of Samsung, Sony, Acer, Toshiba, and others. Other recommended tablets were the Pantech Element, Sony Tablet P, Toshiba Excite 10LE, and the Samsung Galaxy Tab 7.7. Consumer Reports concluded that the new iPad’s higher temperatures weren’t, in fact, a major issue.

“Responding to consumer comments on the new device, and to coverage from other reviewers, we also carried out further tests that confirmed the new iPad is warmer in its hottest spots than the iPad 2,” Consumer Reports wrote. “But we didn’t find those temperatures to be cause for concern.”

Consumer Reports further indicated the battery charging issue it previously reported occurred only when playing a “demanding” game with the screen at full brightness.

The iPhone 4s, iPad 2, and iPhone 4

iphone-4-death-grip

This isn’t the first time Consumer Reports has created a stir with its review of an Apple product. Despite giving the iPhone 4 the highest possible score in its smartphone category, Consumer Reports refused to list the iPhone 4 as “recommended” due to signal attenuation issues — the so-called “antennagate” that revealed signal strength to the iPhone can be significantly reduced depending how the device is held. Apple was forced to do significant spin control on the issue, holding a special event demonstrating that a variety of phones from competitors exhibited the same attenuation problems when held certain ways, and even offering free bumper cases for iPhone 4 owners who were experiencing problems. Ultimately, the issue that caused Consumer Reports to withhold its “recommended” rating from the iPhone 4 was an issue that was neither unique to the iPhone 4, nor a significant problem for most users — the iPhone 4 continues to be a top-selling device, and Consumer Reports continues to cite attenuation problems as by far the most significant issue with the device.

Consumer Reports does give its “recommended” rating to the iPhone 4S, claiming the device doesn’t suffer from attenuation issues like the iPhone 4. The iPhone 4S does include antenna-switching capability that enables the device to swap its transmit and receive antennas in the event of signal attenuation — for some folks, that may make the device somewhat more reliable than the iPhone 4.

Consumer Reports did give its “recommended” rating to the iPad 2. However, Consumer Reports also listed the Motorola Xoom, the original 7-inch Samsung Galaxy Tab, and the original iPad as “recommended.”

What does Consumer Reports want?

A tech-savvy reader probably looks at these recommendations and wonders if Consumer Reports is out of its mind. Is the Pantech Element (with Android Honeycomb, a 1,024 by 768-pixel display, and a sub-par camera) really in the same class as the new iPad? When Consumer Reports recommended the iPhone 4S, the organization went out of its way to point out they rated several Android phones more highly, including the Samsung Galaxy S II series, the Motorola Droid Bionic, and even the LG Thrill. Why? The primary criteria for ranking those Android devices higher was that they offered larger displays and (in the case of the Droid Bionic) “excellent keypad readability.”

These sorts of evaluations tend to fly in the face of the rigorous testing on which Consumer Reports prides itself. It’s hard to imagine the organization spends significant amounts of time designing and conducting quantifiable tests and comparing the results only to boils down to these phones are better because they have bigger screens, or the new iPad’s display is so great we lowered the scores for every other tablet’s display. The mixture of allegedly diligent and quantifiable testing and blatantly subjective qualities creates a disconnect. Consumer Reports seems to be stating an opinion, and promoting it to the world as a conclusion derived by scientific methods.

Bottom line

Suzuki Samurai failure test img

It’s important to remember that Consumer Reports is not written for experts. The publication’s primary audience is everyday consumers who are generally uninformed about a variety of high-cost products and services. Consumer Reports looks at everything from airlines and credit cards to healthcare, home appliances, cars and — yes — electronics. The magazine attempts to sum up a huge variety of products and product categories in neat snippets that can be quickly read and understood by users without much knowledge of the field. It’s not an easy task.

The tech savvy have only to look at Consumer Reports’ coverage of automobiles to understand the nature of the disconnect. Consumer Reports has been responsible for some recalls and redesigns of vehicles over the years — the most famous case probably remains Consumer Reports’ 1988 assessment of rollover problems in the Suzuki Samurai, but Consumer Reports auto testing has led to design changes in vehicles from BMW, Chrysler, Nissan, Toyota (via its Lexus brand), and others.

But car enthusiasts — you know, the “petrol heads” that might get featured on a segment of Top Gear — have taken issue with Consumer Reports’ automotive recommendations for years. Each case has its own set of details and points, but the overall disconnect will feel familiar to technology enthusiasts: Many car fans feel that Consumer Reports tests are unfair to particular models or products, or make recommendations based on subjective criteria rather than qualitative facts. Consumer Reports might look at a ton of details, and conclude one model of sedan is better than another because it can seat three adults in the rear seat — ignoring other features  (like a backup camera or availability of a six-speed manual transmission) that other people believe are more important.

We wouldn’t want to suggest buying a tablet is like buying a car, but the point is valid: It’s tempting to think of tablets (or phones, or cars) as one-size-fits-all solutions, but every individual’s situation is unique. Some folks want a 4.3-inch display on their smartphone, even if it means it no longer fits in their pocket; some folks can’t justify the expense of an iPad’s retina display on a device they just want to keep the kids entertained during their commute.

Consumer Reports has no idea what your particular situation might be. It’s testers are trying their best to address what research shows are key issues for many consumers. And, just like the crowd-sourcing of online product review, the conclusions may — or may not — apply to you.

[Consumer Reports 1960 television testing image © 1997 Consumers Union of U.S.]

The views expressed here are solely those of the author and do not reflect the beliefs of Digital Trends.

Mobile

The 2020 iPhones may only use OLED displays and Intel 5G modems

While some reports hinted that Apple was looking to move away from using Intel tech, a new report suggests the exact opposite. Reportedly, Apple has chosen Intel to supply 5G modems for Apple's first 5G-enabled iPhone, due in 2020.
Mobile

How to choose an iPad in 2019: A practical guide to Apple’s tablets

Selecting an iPad from Apple's lineup can be intimidating, but it doesn't have to be. Our comprehensive guide should put the numbers and specs in practical, easy-to-understand terms. Find your ideal iPad with the help of our guide.
Deals

Need a new tablet? The best iPad deals for January 2019

In the wide world of tablets, Apple is still the king. If you're on team Apple and just can't live without iOS, we've curated an up-to-date list of all of the best iPad deals currently available for December 2018.
Mobile

Here’s how to take a screenshot on an iPad, step by step

The ability to capture screenshots may not be the iPad's most glamorous feature, but it's one of its most useful. Here's a step-by-step guide on how to take a screenshot on an iPad, whether it's an iPad Pro from 2018 or an older iPad model.
Mobile

Apple resurrects the iPhone SE with brand-new units starting at $249

Apple quietly started selling the iPhone SE again, at even lower prices than when it was discontinued four months ago. Brand new units of the 32GB version are on sale for $249, while the 128GB version is going for $299.
Mobile

Text up a storm with the best messaging apps for iOS and Android

These days, most people tend to favor digital messages over phone calls. We have the best messaging apps that allow you to share photos and documents, send text messages, and more with end-to-end encryption.
Product Review

Controversy has dogged the MacBook Pro lately. Is it still a good purchase?

The MacBook Pro is a controversial laptop these days -- and that's unfortunate. Due to some divisive changes Apple made to the functionality of the MacBook Pro, fans are more split. Does the 8th-gen refresh change that?
Apple

Want a MacBook that will last all day on a single charge? Check these models out

Battery life is one of the most important factors in buying any laptop, especially MacBooks. Their battery life is typically average, but there are some standouts. Knowing which MacBook has the best battery life can be rather useful.
Mobile

The best iPhone 6 cases for style and protection

No one likes a bruised Apple. Scratches, scuffs, and cracks on a new iPhone 6 are enough to ruin anyone’s day. Check out the best iPhone 6 cases and get some protection on that shiny new smartphone.
Mobile

Is 5G as fast as they’re saying? We break down the speeds

We take a look at the kinds of speeds you can expect from 5G when the networks roll out. Find out 5G compares to the last generation of network technology, what the minimum and maximum speeds will be, and what it means for us.
Mobile

Tizen 4 arriving on Samsung’s Gear S3 and Gear Sport smartwatches

Samsung is updating the Gear S3 Classic, Gear S3 Frontier, and the Gear Sport to the newest version of Tizen 4. Along with of some little tweaks to usability and quality-of-life, Samsung has added some new features.
Mobile

Mobvoi’s TicWatch E2 and S2 are the most affordable Wear OS smartwatches yet

Quality smartwatches don't have to be expensive, and Mobvoi's TicWatch is the proud paladin of that philosophy. Mobvoi's new TicWatch S2 and are both available for low prices from Amazon and Mobvoi's website.
Deals

Save up to $950 with the best smartphone deals for January 2019

Need a better phone but don't want to spend a fortune? It's never a bad time to score a new smartphone and save some cash. We rounded up the best smartphone deals available that can save you as much as $950.
Mobile

Apple Pay coming to more top U.S. stores, including Target and Taco Bell

Apple Pay is the best way to pay on your iPhone and Apple Watch. Apple Pay support is being rolled out across the U.S. for a variety of top retailers, including Taco Bell, Target, Hy-Vee, and more.