My reaction, upon reading the study, was: “wow, they’ve done a lot of work, and they’ve got some cool data, but I’m not sure I trust the results”.

The study is called " A century of trends in adult human height ". It's attributed to a research team called the NCD Risk Factor Collaboration, and the corresponding author is Majid Ezzati , a Professor at Imperial College’s School of Public Health. The research team did a meta-analysis of 1472 studies, all of which were based on direct (not self-reported) measures of height. The aim of the project was to estimate the height, at age 18, of people born each year between 1896 and 1996. When the population sampled was older or younger than 18, the authors used a growth model to estimate height at 18 years of age. When no data on a particular birth cohort was available, they projected observed trends forwards or backwards to get some estimate of the heights of the missing cohorts.

Take for example, this figure, that shows the height of men born in 1896 and 1996 at age 18 in countries around the world.





Looking at that figure, one's suspicions should be raised. Estonia and Latvia weren't even independent countries until after World War I. They were part of the Russian empire up to that point. Then there was the Soviet period. How on earth could you get reliable data on the height of Latvians and Estonians born in 1896?

This file lists the studies used by the research team for each country. The Latvian numbers - including that headline grabbing statistic that Latvian women are the tallest in the world - are based on one study carried out in 2008/9 of 1362 Latvian men between the ages of 25 and 74, and 2399 Latvian women. The Estonian numbers are based on a larger sample and more studies, but the earliest Estonian study included in the metanalysis is a 1997 study that sampled people up to 64 years of age - i.e. born in 1933. The 1896 birth cohort number? Just an estimate, calculated by projecting trends back in time 30 or 40 years. Unfortunately there were a few structural breaks in Estonian history that might make doing such projections a bit tricky.

[Update] To you can see just how much guess-work was used, I've replicated a picture that shows the much-hyped "Koreans sprout up" result - I've reproduced the picture for women because it has the horizontal axis labelled - that's the birth cohort. (The vertical axis shows height at age 18).





There is no data at all for people born before 1916. The first 20 years of "sprouting up" are generated by assuming that the 1896 to 1916 period was characterized by the same kind of increase in height as later periods. The data for the people born just after 1916 comes from surveys carried out in 1998 or later - i.e. from measurements of the heights of people up to 80 years old. To estimate the average height, at age 18, of people born in 1918 by observing that cohort in 1998 when they are 80 years old involves some heroic assumptions - assumptions about shrinkage with age, survival rates, etc. It would make a lot more sense to choose a shorter time span for the analysis, and give results that involved a bit less guesswork.

Is this bad science? I would say yes. It's bad science because it oversells the results. The article overstates both the amount of height data the research team has (it's not a century, in many cases it's more like 50 to 75 years, especially for women), and also how recent the data is (in most cases the data is not for the 1996 birth cohort, but rather for earlier birth cohorts). It's bad science because it presents headline grabbing results - and makes them readily available to journalists - without attempting to convey, in ways that are easy for reporters to understand, the amount of uncertainty associated with those results. Are Latvian women tall? Yes. Are they the tallest in the world? We can't know that for sure unless we know the margin of error associated with the estimates of Latvian, Dutch, and other groups' heights. It's hard to put a standard error around the results of complicated projections - but that's an argument against making complicated projections, and disseminating them to reporters, not against reporting standard errors.

It's also bad science because it draws unwarranted conclusions from its results. One widely reported result of the study is that Americans are, apparently, shrinking. Here's an extract from the NPR report on the study:

"There was a time when the U.S. was the land of plenty," says Majid Ezzati, of Imperial College London, who helped to lead the study. "But increasingly over time, the quality of nutrition has worsened."

Income inequality has increased in the U.S. since the 1970s, the Center on Budget and Policy Prioritiesreported. "In some sense, you have a large part of the population who are not getting quality food," Ezzati says. "That drags down the whole place."

That makes it sound as if the Professor Ezzati is certain that declining nutritional quality is responsible for part of the stagnation in US height browth. Indeed, the article itself gives the clear impression that nutritional quality is the primary determinant of international differences in height:

cross- population differences are believed to be related to non-genetic, environmental factors. Of these, foetal growth (itself related to maternal size, nutrition and environmental exposures), and nutrition and infections during childhood and adolescence are particularly important determinants of height during adulthood ("A century of trends in adult human height")

Unfortunately, this is another case when we just don't know. Is the stagnation in US height due to reduced nutritional quality? Is it due to an influx of relatively short immigrants? Is it the result of differential fertility rates, with tall women having fewer children, and short women having more children? It's impossible to tell without a much more sophisticated analysis than is done in this paper.

The other reason to think that the height study might be bad science is that it hasn't gone through the traditional peer review process. The journal eLife, in which it appears, is committed to pain-free publishing. The review process is fast and hands off: "Initial decisions are made in a few days, post-review decisions in about a month, and most articles go through only one round of revision." "The scientist editors who run eLife will give you feedback that’s constructive and fair." eLife may not have a great impact factor, but that doesn't matter, "eLife papers get great media coverage in venues like the New York Times and National Geographic. We make every paper more accessible to a broad set of readers – including students, colleagues in other fields, and the public – through Impact statements, plain language summaries (eLife Digests), and selected expert commentaries (eLife Insights). eLife articles are immediately and freely available to the world – and there’s no cost to publish."

So, in answer to the question posed by the title of this blog post: that height study is bad science. I don't actually blame journalists on this one - they were basically reporting the results that were fed to them. In some ways the more interesting question is why - why would intelligent people doing serious work oversell their results in this way? For better or for worse, there are now growing pressures on researchers to demonstrate that their research matters. Media coverage is one way to show impact - hence the temptation to serve up readily accessible clickbait.