Chris Hannemann, a 32-year-old product engineer in San Diego, California, was diagnosed with Type 1 diabetes when he was 8. For the past 24 years, multiple times a day, every day, he’s pricked his finger and used a blood glucose meter to measure the amount of sugar in his blood and decide whether to administer either insulin or a snack.The meter Hannemann uses regularly sometimes gives him readings that suggest his blood sugar levels are normal, even when he feels woozy or loses fine motor control (early effects of low blood sugar levels). “As someone who’s been comatose multiple times [due to other diabetic issues],” he told us, “it’s not fun.”

During a doctor’s visit, Hannemann noticed that his glucose levels in lab tests seemed different than the measurements he would take himself. He suspected that his blood glucose meter was giving him inaccurate readings. To prove his theory, he ran a series of tests on 10 different meters.

Hannemann found that readings from different meters varied from each other by as much as 60 percent, even though they were analyzing the same drop of blood, and varied 30 percent on average from each other. He published his findings in a Medium post.

This discovery frustrated him because there’s so little information on glucose meter accuracy. “As a patient, you have no knowledge of this,” he said. Now, if he is using the inaccurate meter, he mentally calculates the difference. “If I check my glucose and it reads 90, I have to remind myself, ‘Oh, you actually need to eat something before you go drive or run or something.’”

Accuracy matters to people like Hannemann and the many patients like him. Twenty-one million Americans have been diagnosed with diabetes, and another eight million have diabetes but don’t know it. Combined, that’s more than 9 percent of the population. With so many people in need, you’d think that a blood glucose meter—a tool used to decide how much insulin to take—would be extremely accurate. Unfortunately, many are not.

Wirecutter hoped to research and test to find the best blood glucose meter for most people, but we found that though many doctors, governmental regulators, and patients say the inaccuracy of meters and test strips is the biggest issue with this product, there is limited data to compare meter accuracy head-to-head.

“Most meters are good to excellent,” said Dr. David Marrero, an endocrinologist who runs an independent lab that does testing for glucose meter manufacturers. “But some are unreliable, and they could give you data which you will act upon which is in fact incorrect.”

What blood glucose meters do

People with diabetes rely on blood glucose meters to monitor their blood glucose levels throughout the day and night. Those who inject insulin (which lowers blood glucose) use them to determine how much insulin to take.

A system consists of a meter, which tends to run between $10 and $50, and test strips—a long-term cost that can really add up. Insurance companies often cover these items, but many people need to cover the expenses of an extra meter or strips out of pocket.

A person with an inaccurate blood glucose meter could overdose on insulin, which could excessively lower their blood sugar and bring on hypoglycemia. Hypoglycemia can cause you to feel dizzy, become unconscious, or even go into a coma. If people underdose, they’re at risk of hyperglycemia. Hyperglycemia can cause a host of long-term health effects, including kidney damage, nerve damage, and blindness.

Blood glucose meters can be lifesaving devices when used correctly. “Blood glucose monitors have transformed the ability of people with diabetes to take optimal care of themselves,” said Dr. Marrero. “It was the dark ages before that.”

A “crisis” of accuracy issues

In a 2014 statement, the American Association of Clinical Endocrinologists told the FDA, “There is currently a crisis of post-market accuracy issues for blood glucose monitoring systems ...” Margins for accuracy are wide and reported in different ways, and the FDA does not test meters for accuracy either before or after they hit the market unless there’s a widely reported problem.

If you test four times a day, you could get a completely useless reading at least once a week, and your meter would still meet the FDA’s accuracy standards.

The FDA states that blood glucose meters’ readings must be accurate within 20 percent, 95 percent of the time for glucose levels above 75 mg/dl (non-hypoglycemic). For hypoglycemic levels (under 75 mg/dl), accuracy is less stringent: It must be within 15 points of the actual measurement (also 95 percent of the time).

This means if your actual blood sugar is 140 mg/dl, your blood glucose meter might tell you it measures 112 mg/dl, or 168 mg/dl. And up to 5 percent of the time, the reading can be completely wacky. If you test four times a day, you could get a completely useless reading at least once a week, and your meter would still meet the FDA’s accuracy standards.

Beyond that, the margin for error gets wider: A 2015 literature review concluded that a quarter of the FDA-cleared blood glucose meters represented in those studies “do not perform at the level for which they were cleared ...”

Even a device that earns FDA approval could later fail those same standards with normal use. According to Dr. David Klonoff, an endocrinologist and the founder of the Diabetes Technology Society, test strips are a big source of accuracy issues: Manufacturing error, storage factors like temperature and humidity, hematocrit levels or substances in the blood like Tylenol, or even not washing one’s hands can all contribute to inaccurate readings.

The FDA relies on data submitted by manufacturers to evaluate meters’ accuracy before they go on the market, and it doesn’t conduct any accuracy testing afterward either. One of the ways the FDA learns about issues with devices or test strips currently is through medical device reporting. Manufacturers must share data with the FDA if they have evidence of “suspected device-associated deaths, serious injuries and malfunctions.” But sometimes manufacturers don’t report issues in a timely manner; one manufacturer didn’t report malfunctions that occurred in 2010 until 2013. Many manufacturers are diligent about reporting suspected accuracy problems to the FDA and issuing recalls: Nova Diabetes Care, Abbott Diabetes Care and LifeScan recalled meters or lots of test strips between 2013 and 2015. (Routine recalls are a good thing because they prove the manufacturer is on top of its product).

Patients and doctors can report issues directly to the FDA through what are called adverse event reports. A spokesperson for the FDA says the agency receives about 36,000 adverse event reports and medical device reports related to blood glucose meters each year. Over 90 percent of the reports are classed as “malfunctions,” which means, among other things, the device’s failure could “contribute to a death or serious injury, or other significant adverse device experiences.”

In 2014, the FDA released a draft proposal that would require all readings be accurate within 15 percent of the actual measurement, 95 percent of the time, and within 20 percent, 99 percent of the time. This wouldn’t solve the normal-use issues of accuracy, but would create an overall smaller window for readings to fluctuate. The FDA’s proposal also calls for clear labeling on blood glucose meter packaging stating the meter’s accuracy performance. Unfortunately, the FDA hasn’t released any information on the current status of the proposal, or when it’s likely to become approved.

Dr. Marrero notes that there’s a trade-off: “Clearly a more accurate meter is valuable. The problem is that as you push accuracy to tighter and tighter levels, you increase costs,” making them even less affordable.

We need more independent testing

In its 2016 Consumer Guide, The American Diabetes Association’s Diabetes Forecast lists over 90 meters made by 27 different companies, with features like Bluetooth and cloud connectivity, but the guide doesn’t include any comparison of the meters’ accuracy. The ADA said in 2012 that it would like to provide accuracy rankings for readers, but that it would be “beyond our capabilities.”

There is currently only one third-party group that tests blood glucose meters for accuracy and reliability after they are on the market: Consumer Reports. Each year, it rates several dozen blood glucose meters after testing them for accuracy, reliability, and other features.

A researcher familiar with Consumer Reports’s testing protocol told us that the publication tested several device samples of each blood glucose meter using multiple test-strip lots and different blood sugar levels. Dozens of volunteers also used each meter in the lab to compare the meter’s readings against their actual glucose levels, and used the meters at home over a period of weeks. Although Consumer Reports’s testing is valuable, the company tested only 27 glucose meters, and it publishes results just once a year.

In the absence of more data from multiple sources, we’re wary of recommending specific glucose meters without a full commitment to complete and thorough testing.

The future of testing and accuracy

Last year, the Diabetes Technology Society, a nonprofit organization that promotes the development of technologies to help fight diabetes, released its own testing protocol for ongoing accuracy monitoring of glucose meters and test strips. DTS developed the protocol over two years, in partnership with governmental agencies, including the CDC, NIH, and FDA, and over a dozen medical research organizations and universities. (It was funded, in part, by a grant from Abbott Diabetes Care, which manufactures the FreeStyle line of blood glucose monitors). Dr. Klonoff said the protocol is “as good as it gets.”

The protocol would use blood samples from over 100 participants and across a range of glycemic levels (hypoglycemic, normal, and hyperglycemic) to evaluate the accuracy of test strips. The protocol also requires obtaining the test strips “in a manner similar to how a person with diabetes would obtain test strips,” (i.e., directly from a store or pharmacy). In June, DTS announced it had begun testing meters using the protocol, but didn’t provide details about how many meters would be tested, when testing would be complete, or if the data would be made public. DTS has not yet responded to Wirecutter’s request for comment.

Some promising new systems could make it easier—and less painful—for some people with diabetes to monitor their blood glucose levels. Continuous glucose monitors use sensors under the skin to monitor blood glucose levels every five minutes. The FDA has not yet approved continuous glucose meters as replacements for traditional finger-prick glucose meters, in part because their readings are even less accurate, and they still require calibration with a traditional meter twice a day.

The Abbott FreeStyle Libre system uses a sensor with a very fine filament inserted under the skin. You scan the sensor with a reader to obtain an immediate glucose reading. The FDA hasn’t approved the Libre yet, but it is available in Europe, Australia, and Brazil.

We still hope to review blood glucose meters in the future, as soon as more organizations can do the kind of comprehensive independent accuracy testing needed. We’re not ready for such a huge undertaking, but it’s something we hope to revisit in the future. If you’d like to talk to us about it, or you have questions or concerns about blood glucose meters, we’d love to hear from you.

If you’re worried about your meter’s accuracy

Don’t stop using your meter the way your doctor told you to; something is better than nothing when it comes to monitoring your glucose levels. If you’re concerned about the accuracy of your meter, talk with your doctor to see what your options are. You can also learn more from a diabetes educator or by contacting your meter’s manufacturer, or you can purchase control solution from the manufacturer to verify that your meter and test strips are working as they should.

“The literature is clear. We know if you test more, you do better than if you test less or don’t test at all,” Dr. Marrero said.