Just how accurate is your wearable? Not very, new study finds

“All worked pretty well at rest,” said Dr. Marc Gillinov, the Cleveland Clinic cardiac surgeon who led the experiment. “But as people exercised, the accuracy diminished.”

Ultimately, Gillinov found that none of the fitness wearables were as accurate as a chest strap monitor, though the Apple Watch and Mio Fuse came closest. The now-defunct Basis Peak, however, overestimated heart rate, while the Fitbit Charge HR underestimated heart rate during vigorous exercise.

But Fitbit remained adamant that its devices are trustworthy, noting, “Fitbit devices were tested against properly calibrated industry devices like an EKG chest strap across the most popular activities performed worldwide, including walking, running, biking, elliptical and more.” This isn’t the first time Fitbit has faced criticism over inaccuracies in their measurements (and the company itself admits that its wearables are not meant to be medical devices).

Study co-author Gordon Blackburn, a cardiac researcher at the Heart and Vascular Institute at Cleveland Clinic, told the Verge that accuracy became particularly contentious when heart rates reached 100 beats per minute, and became “very inconsistent above 130 and 140 beats per minute.” This is problematic as these ranges are quite common for healthy individuals to aim for while exercising. “Patients come in with data and they say, ‘I’m exercising and my heart rate is going sky high,’ or ‘I can’t get into the zone’ and they’re getting frustrated,” he said. “The people developing these trackers say it’s not a medical technology, but that message isn’t being absorbed by users. And we’re caught in the middle when they come into the office presenting data and we have to convince them there is significant error.”

Of course, this doesn’t mean that you should just be chucking your wearables out the window.

Dr. Mitesh Patel, an assistant professor of medicine at the University of Pennsylvania’s Perelman School of Medicine in Philadelphia, was not involved in the new study, but specializes in the impact of technology-based interventions on behavior. “For the general consumer, wearable devices may still be able to give them a general sense of their heart rate trends,” Patel told CBS News. “However, further study is needed to determine which devices are more reliable for use in clinical care.”