More evidence is in that charter schools – at least on average – do no better than regular public schools.

Middle-school students who were selected by lottery to attend charter schools performed no better than their peers who lost out in the lottery and attended nearby public schools, according to a study funded by the federal government and released Tuesday.

This is the first large-scale randomized study to be conducted across multiple states, and it lends some fuel to those who say there is little evidence to back the drive for more charters.

But the study also found more nuanced evidence that the charters that work best are those serving lower-income students, especially in urban areas.

“When you take a look at our findings and then look back at previous studies, they start to follow a pattern,” says Philip Gleason, the study’s director and a senior fellow at Mathematica Policy Research, which produced the study. “Studies that have focused on the largest set of schools find either no or negative effects, but schools in larger urban areas, serving the most disadvantaged students, do have an effect.”

Charters have been a hot topic lately, with a big push from the Obama administration for states to expand the number of charter schools and to replicate those that seem to have the most effect. Charters are publicly funded but largely autonomous, and they are frequently criticized by teachers unions, in particular, since they are not bound by union agreements.

The push to create more charters has been questioned in light of research showing no advantage – or even a negative effect – for students attending charter schools. Such research includes a much-publicized study a year ago from the Center for Research on Education Outcomes (CREDO). Several other, narrower studies – including one on New York City charters and a study that came out last week on charter schools operated by the Knowledge Is Power Program (KIPP) – have showed positive outcomes for charters.

The result, say education researchers, is a heated debate but also a growing consensus that charters, like regular public schools, vary widely in their quality and that they are at their best when serving a more disadvantaged population.

“It’s not surprising that suburban charter schools don’t do anything, because suburban schools are already pretty good,” says Diane Whitmore Schanzenbach, a professor at the University of Chicago’s Harris School who has studied charter schools in Chicago. “At this point,” she adds, “the literature is still trying to figure out, are charter schools better or not? And arguably, that’s the wrong question to even be asking.” More interesting, Professor Schanzenbach suggests, would be research on what makes some charters more effective.

The Mathematica study began to explore some of those questions, and it found some correlation between better performance and smaller charters and those that used ability grouping in classrooms more than surrounding schools did.

The study looked at the performance of students after they had been at a charter school for a year or two.

Such national studies only highlight the broad differences in what is ultimately a state-driven policy, says Robin Lake, associate director of the Center on Reinventing Public Education at the University of Washington in Seattle.

“We’ve learned a lot about how to make charter schools work,” she says, adding that a lot of attention now is focusing on how to fine-tune the authorizing process so that states select the best schools, monitor them well, and then shut down those that aren’t performing sufficiently.

But others believe that, while there still may be a place for some charters, research like this study doesn’t justify the massive public-policy push to create more charters quickly.

“The worry is that President Obama and others are getting seduced by the movement because they’re looking at the results from boutique charters [like KIPP and Aspire] rather than at the wide array of charters that don’t outperform regular schools,” says Bruce Fuller, an education professor at the University of California at Berkeley. Professor Fuller remains “cautiously optimistic” about charters and says they seem to do some things well, such as attracting energetic young teachers. But, he adds, “It’s irresponsible that President Obama would [push] all 50 states to create more charter schools in light of such sketchy evidence.”

Related: