Is the ‘best we can get’ good enough?

Even observational trials are hard to do well. Most major health setbacks are pretty rare. It’s hard to see big differences in death, cancer and heart attacks in even large groups of people, unless you follow them over long periods. But quantifying what people are eating over long periods is challenging, too, because often people don’t remember.

Such studies are also difficult to interpret because of what are called confounding factors. Maybe people who eat more meat are poorer. Maybe they smoke, drink too much alcohol or don’t exercise. Those things would also lead to bad outcomes, and it’s hard to tease out individual components over time.

If you do trials of people at higher risk — those who have already had heart attacks, for example — it’s easier to see if changes matter. The Predimed trial, for instance, which studied the Mediterranean diet, focused on people who already had diabetes or a number of traits placing them at high risk for heart disease. But these people aren’t necessarily representative of the general public, for whom dietary recommendations are written.

All of this means that observational evidence, which is easier to obtain, will be ruled as “low quality” by some researchers. Others will argue it’s the best we can get, and therefore we should apply different standards to such research.

Should we care about signals like blood pressure, or only major events like heart attacks?

Because big outcomes are rare, research sometimes looks at intermediate measures. Those, like weight, blood pressure, cholesterol levels and more, can change in shorter periods. Some will point to studies in these domains and say that they prove that meat reduction has significant health effects. High blood pressure or cholesterol levels are widely believed to be major risk factors for adverse events. Others will disagree as to how much we should rely on intermediate measures. These new studies focused only on those end-stage outcomes.

If experts are uncertain, should they recommend anything?

Critics of the new meat studies argue that given the authors’ low certainty about their findings, they should have issued no recommendations at all. That’s not unreasonable. When the U.S. Preventive Services Task Force lacks sufficient evidence to publish recommendations on prevention, it gives recommendations an “I” rating, and says the current evidence is insufficient to assess the balance of benefits and harms. That’s all. Maybe that was preferable here, instead of publishing recommendations that people continue their current meat consumption.

Should we look at the individual, or the population?

Even in studies that find statistically significant effects, the absolute benefits in most studies are small. I’ve written about this before. Many will argue, however, that even if there is a small individual benefit, the benefits to the population can be large.