Prekindergarten is extremely popular. President Obama made expanding pre-K a major policy goal. In the past few years, several states have launched pre-K programs, and large cities such as New York have pushed to make pre-K universal on the promise that seemingly large benefits can come from educating kids at a young age, setting them up for success later in life.

But a recent study of Tennessee’s voluntary program for 4-year-olds from low-income families found that by third grade, kids who went to pre-K fared worse academically than those who didn’t. That shocking finding has triggered a debate among experts; some have called into question pre-K’s long-touted benefits, while critics of the study have sought to reaffirm pre-K as a good investment.

“We haven’t found any sustained effects, either in social and emotional development or achievement,” Vanderbilt University professor Dale Farran, an expert in early childhood education and one of the co-authors of the bombshell study, said in an interview in December. As a result, Farran and her co-authors caution against a rush to expand pre-K programs, saying that the evidence of early childhood education’s benefits is based on research of programs that differ significantly from the large state pre-K programs that are in vogue.

In effect, the researchers think their findings call into question the pre-K revolution. “You have school systems that are pushing pre-K when they have demonstrably failing K-12 systems,” Farran said. “It makes me cringe.”

But a separate team of researchers at the University of Chicago has criticized the Tennessee study in a recent working paper. They view its results as more than just surprising — they claim they run counter to most other pre-K research.

For the Tennessee study, Farran and her co-authors began to partner with the state Department of Education in 2009 on an ambitious randomized controlled trial — a type of experiment that is the gold standard of social science research — to measure how the state’s pre-K program was affecting kids’ academic achievement and social and emotional development. In its experiment, the Vanderbilt team followed more than 3,000 4-year-olds in two cohorts; the kids were randomly assigned to either attend the state’s pre-K program (the treatment group) or not (the control group).

To track academic achievement, a smaller sample of kids in both the treatment and control groups were given the Woodcock Johnson III Achievement Battery — a series of tests measuring literacy (the kids’ skill at letter-word identification, for example), language (picture vocabulary, for example) and math skills (like number identification).

The initial results cast doubt on the benefits of pre-K. Compared with the control group, kindergarteners who had already attended a year of school initially made strong gains. But whatever advantages the pre-K group had accrued subsequently faded, and fast. The researchers found that the control group caught up with the pre-K kids within a year, and by third grade, there was a “crossover” — that is, the kids who never went to pre-K performed better according to a composite of academic scores.

The researchers also tracked teacher ratings of children’s classroom skills related to self-control: the ability to pay attention, levels of social interaction, how much a child enjoys school, etc. By these measures, too, the pre-K group had an initial advantage. These children were rated higher by their teachers in kindergarten. But by the end of first grade, the pre-K group was exhibiting worse behavior. “That was really a surprise to us,” Farran said.

The University of Chicago team — led by Nobel Prize-winning economist James Heckman, one of the godfathers of early childhood education research and a long-time pre-K proponent — has marshaled a battery of arguments to dispute the Tennessee study’s findings. First, Heckman’s team criticizes the study’s methodology, saying that the researchers failed to get a fully random sample of participants. (The Vanderbilt team stands behind its study design, saying that it checks out against selection bias and that Heckman’s team is making a mountain out of a molehill.)

Heckman’s team also maintains that the case for pre-K’s benefits outlined in earlier research is strong. In their new working paper, the Chicago researchers re-analyze a slew of studies — experimental, observational, and in-between — on early childhood education and conclude: “There is a strong case for high-quality early childhood education for disadvantaged children.”

In the Heckman team’s view, the Vanderbilt researchers did not track students long enough to see the long-term benefits of pre-K. They also argue that any negative outcomes could reflect problems with the quality of the Tennessee program, rather than a fundamental problem with pre-K. “There were several voices saying this study was so good, so unique … that it’s better than anything we’ve done before,” Andrés Hojman, one of the Chicago researchers, said in an interview. “That’s not true.”

The pre-K advocates point to randomized controlled trials that began with the Perry Preschool Project in Michigan during the 1960s and the Carolina Abecedarian Project in the 1970s, both of which followed participants into adulthood to measure outcomes. These programs were small (123 and 111 kids, respectively) and intensive — they often provided health care to participants in addition to education. For both programs, participation produced impressive results in children’s social and emotional development, if not academic achievement. Kids who participated had better behavioral outcomes as adults: lower crime rates, better health and higher earnings. Heckman’s team says these long-term benefits, which the Tennessee study couldn’t capture, justify expanding pre-K.

The Vanderbilt researchers maintain that state-based programs — like the one in Tennessee — are a more realistic reflection of modern pre-K than the small, intensive programs made famous in those classic experiments. And the Tennessee results raise a red flag. In the Vanderbilt researcher’s view, if their study is seeing negative achievement effects so early — something that’s unusual in pre-K research — then maybe the purported long-term behavioral benefits in adulthood might not materialize either.

That’s because the Perry and Abecedarian were not typical pre-K programs. Some of these programs started when participants were in infancy; often they were available to parents for much of the year. It’s dubious “to generalize from [those programs] to say states can provide pre-K to 4-year-olds and get the same effect,” Farran said. “What worries me the most is rushing states into scaling up, without thinking about how they’re designed.”

The Vanderbilt researchers are not ideologically opposed to pre-K; they just haven’t seen benefits of pre-K materialize in their experiment. “I haven’t spent my life being opposed to interventions for young children,” Farran said. Like Heckman, she’s been studying early childhood education for a long time. “I’ve spent 40 years thinking about this,” she said.

And yet it might be many more years before the debate over pre-K is settled.

CORRECTION (Jan. 6, 5:50 p.m.): A previous version of this article referred incorrectly to a series of tests given to children under study. The tests are called the Woodcock Johnson III Achievement Battery, not the Woodcock Johnson III “tests of cognitive ability.”