Even the most accurate device was off by an average of 27%. And the least accurate was off by 93%, Stanford says.

Fitness wearables are very good at accurately tracking a user’s heart rate but lacking in counting calories burned, says new research from the Stanford University School of Medicine.

An evaluation of seven devices in a diverse group of 60 volunteers showed that six of the devices measured heart rate with an error rate of less than 5%. Stanford researchers evaluated the Apple Watch, Basis Peak, Fitbit Surge, Microsoft Band, Mio Alpha 2, PulseOn and the Samsung Gear S2 fitness trackers. Some devices were more accurate than others, and factors such as skin color and body mass index affected the measurements, Stanford says.

In contrast, none of the seven devices measured energy expenditure accurately, the study found. Even the most accurate device was off by an average of 27%. And the least accurate was off by 93%, although Stanford didn’t break out metrics for each device.

“People are basing life decisions on the data provided by these devices,” says Stanford professor of cardiovascular medicine, genetics and biomedical data science Euan Ashley. “But consumer devices aren’t held to the same standards as medical-grade devices, and it’s hard for doctors to know what to make of heart rate data and other data from a patient’s wearable device.”

Manufacturers may test the accuracy of activity devices extensively, Ashley says, but it’s hard for consumers to know how accurate such information is or the process that the manufacturers used in testing the devices.

advertisement

To make some comparisons, 60 volunteers, including 31 women and 29 men, wore the seven devices while walking or running on treadmills or using stationary bicycles. Each volunteer’s heart was measured with a medical-grade electrocardiograph. Metabolic rate was estimated with an instrument for measuring the oxygen and carbon dioxide in breath—a good proxy for metabolism and energy expenditure, Stanford says. Results from the wearable devices were then compared to the measurements from the two instruments with the highest accuracy.

“The heart rate measurements performed far better than we expected,” Ashley says.“But the energy expenditure measures were way off the mark. The magnitude of just how bad they were surprised me.”

The take-home message for consumers is that a user can pretty much rely on a fitness tracker’s heart rate measurements, but basing the number of doughnuts a consumer eats on how many calories a device says the user burned is not a good idea, Ashley says.

Stanford researchers could not conclude why energy-expenditure measures were so far off. “My take on this is that it’s very hard to train an algorithm that would be accurate across a wide variety of people because energy expenditure is variable based on someone’s fitness level, height and weight, Ashley says.” Heart rate is measured directly, whereas energy expenditure must be measured indirectly through proxy calculations.”

advertisement
Favorite