A top level analysis of dozens of power meters has revealed how inaccurate some of them can be.

The researchers say cyclists should get theirs calibrated if they want to rely on the information about their own performance.

>> Save up to 31% with a magazine subscription. Enjoy the luxury of home delivery and never miss an issue <<

Fifty-four units were put to the test in the laboratory, where special equipment was used to make the measurements, away from the variables of the road and track that can affect results.

>>> Training with power: how do you do it properly, and what mistakes should you avoid?

The 54 power meters included 23 different models from nine manufacturers, including SRM, PowerTap, Quarq, Stages, Verve, power2max, Garmin, Polar and Rotor.

They found that individual power meters deviated a lot, even when units came from the same manufacturer. The scientists are concerned that six units deviated by more than five per cent, including products from Stages, Quarq and power2max.

This means riders can’t always be sure whether they are improving, maintaining fitness or even possibly getting worse.

Many cyclists, particularly those at an elite level, are happy when they record an improvement of just one per cent.

It’s clear from the study that the inaccuracies from a poorly performing power meter can easily mask or exaggerate any change in the rider’s ability.

A rider using the worst meter would have to change their own performance by at least 2.8 per cent to have reasonable confidence that the measurements from the unit accurately reflected the difference.

The scientists don’t know whether the worst meters are not well calibrated before they leave the factory or if their set up deteriorates during use.

“Power meters used by elite and recreational cyclists vary considerably in their trueness,” say the scientists.

“Precision is generally high but differs between manufacturers.” Models from long-established makers SRM and PowerTap were among the best.

The study has been done by five researchers in the elite sport section of the Swiss Federal Institute of Sport, which works closely with the national team. It’s Switzerland’s equivalent to British Cycling’s famously clandestine secret squirrel project.

Watch: Vo2 max – does it matter?

The tests involved placing bicycles with power meters on a downward sloping treadmill in the laboratory because riding on the road or track can make it difficult to get reliable measurements.

A restraining force stopped the bicycles rolling forwards and it was measured as riders pedalled at a range of known speeds. The scientists calculated the true power output from their own measured data and compared it with the figures from the power meters.

Accuracy of cycling power meters against a mathematical model of treadmill cycling by Thomas Maier, Lucas Schmid, Beat Müller, Thomas Steiner and Jon Peter Wehrlin, published in the International Journal of Sports Medicine.

Max Glaskin is an award-winning freelance journalist who tweets about cycling and science as @CyclingScience1 and is the author of Cycling Science.