In its first five years, closing schools and shifting students to higher-performing district and charter schools did more to boost Newark, N.J.'s achievement than improvement efforts in the schools overall, according to a new study.

In a working paper released this morning by the National Bureau of Economic Research, researchers from Harvard and Dartmouth universities found that higher-performing schools in the Newark district provided a buffer for the shock of disruption in the early years of the initiative and gave improvement programs within schools more time to gain traction.

The study comes as Newark reverts from state to local control of schools this year, and residents debate how to shape the district in the future.

"Sustainability is the question in Newark going forward," said Harvard economist Thomas Kane, one of the authors of the study. "I don't know the answer to that question, but I do think that the folks in Newark should be recognizing the factors that allowed it to make the progress it has over the years."

There were no half-measures in the highly controversial initiative to overhaul Newark schools in 2010. Backed by the city's then-Mayor Corey Booker, a Democrat, Republican Gov. Chris Christie, and $200 million in matched donations from Facebook CEO Mark Zuckerberg's Chan-Zuckerberg Initiative and others, the reform push begun in 2011-12 introduced a new curriculum, closed and reconstituted schools, expanded the district's charter sector, introduced a new curriculum and teacher evaluation system, instituted a new teacher contract and expanded learning time in several schools. It also closed and reconstituted schools, expanded the district's charter sector, and launched a universal district enrollment that allowed parents to apply to most district and charter schools.

The researchers tracked the achievement levels and student growth rates of both district and charter schools, in comparison to state averages, from 2011-12 to 2015-16.

"One thing surprising for us was yes, achievement levels were low before [the overhaul], but if you compared Newark as a whole to the growth rates of similar students attending similar schools elsewhere in New Jersey, Newark was actually above average in math and at the state mean in English," Kane said.

Student growth rates dropped in math and reading/language arts in Newark's district and charter schools in the first years of the program. Kane said they were not able to point to exactly what caused the drop, but he pointed to disruptions caused by a switch to a new curriculum aligned to the Common Core State Standards, teacher turnover, and shuffling of students as the district closed more than a dozen schools with below-average achievement and growth rates. Student growth rebounded in English/language arts, and by 2016 Newark students showed significantly higher growth than the state average.

Researchers found that 62 percent of the improvement in language arts came from students moving from low-performing and low-growth schools to higher-performing schools, both through forced school closures and the new open district and charter enrollment system. By contrast, growth within existing schools took longer and was slower to pick up steam.

In fact, in math, Kane and his colleagues found no overall difference in achievement and growth from 2010 to 2016, and a decline in within-school growth in math during that time.

The results should raise questions for districts that would use the city as a model for school reform, Kane noted. Its charter schools were among the best in the country before the reform effort, and the district designed its new enrollment system to make it easier for parents to find schools following the closures.

"I'm an economist," he said. "We've been noticing in many other industries, the way productivity growth happens is when more productive firms gain market share. It's less often about existing firms becoming more productive. It's hard to fundamentally change how you organize and do your work. That's the equivalent of what we see here."

Chart: The change in student growth in English/language arts in Newark schools was driven largely by students moving out of lower performing schools. Source: "Assessing the Impact of Newark Education Reforms: The Role of Within-School Improvement vs. Between-School Shifts in Enrollment." National Bureau of Economic Research