While this result may seem to offer the promise of an easy 13-percent boost in students’ learning, it is critical not to forget that the results may come out completely different if the study were replicated in a different group of children, in a different school, under a different moon. In fact, a systematic review has shown that small, idiosyncratic studies are more likely to generate big findings than well-conducted larger studies. Would that 13 percent gap in student performance narrow in a larger study that controlled for more variables?

In other words, rather than base wide-reaching policy decisions on conclusions derived from 24 kindergarteners, it would seem reasonable, for now, to keep the Jane Austen posters and student art on the classroom wall.

3. Does the study have enough scale and power?

Sometimes education studies get press when they find nothing. For instance, Robinson and Harris recently suggested that parental help with homework does not boost academic performance in kids. In negative studies like these, the million-dollar question is whether the study was capable of detecting a difference in the first place. Put another way, absence of evidence does not equal evidence of absence.

There are multiple ways good researchers can miss real associations. One is when a study does not have enough power to detect the association. For example, when researchers look for a rare effect in too small a group of children, they sometimes miss the effect that could be seen within a larger sample size. In other cases, the findings are confounded—which means that the factor being studied is affected by some other factor that is not measured. For example, returning to Robinson and Harris, if some parents who help their kids with homework actually do the kids’ homework for them while others give their kids flawed advice that leads them astray, then parental help with homework might appear to have no benefit because the good work of parents who help effectively is cancelled out by other parents’ missteps.

It’s always a good idea to check whether a negative study had enough power and scale to find the association it sought, and to consider whether confounds might have hidden—or generated—the finding.

4. Is it causation, or just correlation?

It turns out that the most important way for parents to raise successful children is buy bookcases. Or at least this is what readers could conclude if they absorbed just the finding summarized in this Gizmodo article, and not the fourth paragraph caveat that books in the home are likely a proxy for other facets of good parenting—like income, emphasis on education, and parental educational attainment.

Correlation—in this case, of bookshelves at home with achievement later in life—does not indicate causation. In fact, it often does not. The rooster might believe it causes the sun to rise, but reality is more complex. Good researchers—such as the original authors of the bookcase study—cop to this possibility and explain how their results might only refer to a deeper association.