It seems a week can’t go by without another article going viral by someone trying to dismantle the “benefits of breastfeeding” (or really the risk associated with formula use, as the biological norm is breastfeeding). One could almost think that this justification for formula use is being peddled by the formula companies themselves, but alas, it seems to be women fighting to see their use of formula as equal to breastfeeding. The biggest issues in the research according to these many articles?

That the effects for breastfeeding, when present, are small and thus overstated; and

That there is that nasty confound with socio-economic status (SES) in that higher SES women are more likely to breastfeed, less likely to face various diseases, and have higher IQ.

You can read about these in almost any mainstream article that attempts to justify the way in which formula is now the socially “normal” way to feed a child. I’m not going to say these problems aren’t real – they are – but rather that they also ignore one of the bigger problems in breastfeeding research. What is this research problem? Put quite simply, it’s the way in which groups are formed when we’re making comparisons about breastfeeding outcomes.

I’m not surprised no one talks about this because it’s actually a problem in that it results in the underestimation of the effect of breastfeeding. That’s right: This problem actually means that our “small” results may really be much larger and some null results may be type II error (i.e. failing to detect an actual difference where one is present) in that we have erroneously failed to reject the null hypothesis because our groups are made in such a way as to bias our results towards a failure to find any significant differences.

How Grouping Works

When you design a study where you want to compare various groups, you have to have pretty strict rules for who belongs in which group. This is great in terms of trying to make claims if your groups are well-defined and theoretically distinct, but with something like breastfeeding, it’s nearly impossible to get the kinds of groups you want. For example, the most common type of comparisons are any breastfeeding for a certain duration versus no breastfeeding or less than for a certain duration, like any breastfeeding at 3 months versus no breastfeeding at 3 months.

Why would this be problematic, you ask? First you have to think about how much breastfeeding is going on. Is it exclusive? Is it 70% of the time? In these groups you will have the mom that exclusively breastfed for 2 ½ months but stopped when going back to work because pumping didn’t work in the non-breastfeeding group and the mom who breastfed no more than 20% of the time and stopped altogether at 4 months in the breastfeeding group, yet which one do you think would have a bigger impact on their child’s health? This gets to be even more complicated the longer the duration is (e.g., when analyses are done at six months).

When you get into exclusive breastfeeding, which should be simpler on the surface, you also run into problems as you now have the mom who struggled and may have given only one bottle (or only for one week while working with a lactation consultant) no longer eligible for the exclusive breastfeeding group despite the fact that her child is going to be far more like the exclusive breastfeeding child than the child who has formula 90% of the time. Or what of the child who has solids introduced at 5 months? Is the effect assumed to be the same as that of the child who has been receiving 50% formula all along? (Hint: nope.)

The complications here stem from the fact that breastfeeding is not an all-or-nothing act and the shades of gray are very difficult to properly categorize yet likely have important implications for the outcomes of interest. In all these categorizations, however, the bias still leads us away from finding the benefits of breastfeeding/risks of formula because the only “pure” group would be those who followed WHO guidelines and exclusively breastfed on demand for 6 months then continued to nurse for approximately 2 years or more (some children wean before that and they would be biologically normal). That is the biological norm for which humans have evolved (or were created, depending on your perspective) and the effects of breastfeeding need to be examined through that lens.

Any criticisms of small effects in groups that includes less than this for a breastfeeding group – especially groups that include mixed feeding early – presupposes that the effects of breastfeeding are huge (akin to some kind of panacea) or that there is no interaction with formula use or amount of breastfeeding. Yet that’s not what those who study breastfeeding or its impact on health are arguing. In fact, it seems to be brought up as such only by those looking to dismantle breastfeeding research because their job is much easier if you hold breastfeeding up to such a high standard that it simply has to fall apart (or if it stands up to such scrutiny, there’s no sense in saying formula is any kind of equal). The researchers themselves are simply limited by the data they have, yet others seem to assume this is some kind of intentional grouping based on the magnificent power of breast milk.

How should groups be made then? When people talk about the ideal grouping methods, the randomized control trial is considered ideal yet it’s impossible in breastfeeding research. In this, you randomly assign who will fit into each condition. In drug trials, it’s ideal: One group gets the drug, another a placebo (or another drug), and you compare outcomes. In breastfeeding we not only can’t dictate who will or will not breastfeed, but we can’t dictate how someone breastfeeds or for how long and we certainly can’t keep people blind to the status of the women and children in the study. Thus we are left with the incredibly messy task of trying to put people with such diverse situations into well-defined groups, a task that makes our interpretation of results very problematic.

The Previous Research

As mentioned, all of the previous studies suffer from this grouping problem. Whether it’s the randomized control trial (of breastfeeding promotion, not actual breastfeeding) in Belarus (also known as PROBIT)