Last week I wrote about the findings from the recently released federal study of the impact of Head Start and placed those findings in the context of research on other preschool interventions. I concluded that whereas other preschool interventions have been shown in randomized trials to increased student achievement at least into the elementary school years, Head Start does not have positive effects beyond the Head Start year.

This week I address universal pre-K, the advocacy movement for states to fund free pre-school education for all children. Proponents of universal pre-K would have states fund a free public education for preschoolers just as they presently do for students in the regular school years. I contrast this with targeted pre-K, the policy of investing state funds in programs for children and families in the greatest need.

The movement for universal and targeted state pre-K has been successful in that enrollment in state-funded pre-K programs for 4-year-olds has doubled in the last 10 years, from 14% to 28% of all 4-year-olds. There has been a concomitant increase in annual spending from about $2.5 to $4.5 billion. Annual spending per child for state pre-K is about $4,000. Thirty-nine states offer state-funded pre-K, with 31 of those states having programs that are targeted for low-income families. When children enrolled in Head Start and other public programs, including special education, are combined with those in state funded pre-K, 42% of the 4-year-olds in the nation are enrolled in a taxpayer funded center-based preschool programs.[1]

This relatively recent explosion of public pre-K programs has been underpinned by research findings from two iconic preschool interventions from 30 to 40 years ago whose participants have been followed into adulthood, the Perry Preschool Project and the Abecedarian Project. These programs are estimated to have generated positive returns on investment (e.g., approximately $13 to the general public for every $1 invested in Perry Preschool per the most optimistic analysis[2]). But does that mean that present day state-funded pre-K programs for 4-year-olds are also good public investments?

We do not have randomized trials on even the short term impacts of state-funded pre-K, much less randomized trials with long term follow-up into adulthood such as we have for Perry and Abecedarian. Thus the best I or anyone sticking close to the evidence can do is to explore whether the expectations associated with state pre-K are reasonable given what we presently know from research. That research includes randomized trials of preschool interventions that aren’t universal pre-K but might be relevant to it, including Perry, Abecedarian, and Head Start, and less rigorous research that has directly examined state-funded pre-K.

In my view, generalizations to state pre-K programs from research findings on Perry and Abecedarian are prodigious leaps of faith. Perry and Abecedarian were multi-year intensive interventions whereas state pre-K programs are overwhelmingly one year programs for four-year-olds. Costs per participant for Perry and Abecedarian were multiples of the levels of investment in present-day state preschool programs, e.g., $90,000 per child for Abecedarian.[3] Both Perry and Abecedarian were small hothouse programs (less than 100 participants) run by very experienced, committed teams, whereas widely deployed present day preschool programs are, well, widely deployed. The circumstances of the very poor families of the Black children who were served by these model programs 30 to 40 years ago are very different from those faced by the families that are presently served by publicly funded preschool programs. For example, nearly half of the four-year-olds in Head Start today are Hispanics, whereas there were no Hispanic children in Abecedarian or Perry. And 40 years ago other government supports for low-income families were at much lower levels and pre-K was not widely available for anyone, much less the poor. Thus, even without the recent findings from Head Start there would be reason to be skeptical that today’s typical state preschool programs for four-year-olds are producing the large benefits that accrued to participants in Perry and Abecedarian.

But we do have the Head Start findings I reviewed last week and we should not ignore them in thinking about state pre-K. Head Start spends about twice as much per child per year as states ($8K per child per year for Head Start vs. $4K for state pre-K). And Head Start includes many program components that are advocated by early childhood experts such as health, nutrition, and parental involvement that are much less prevalent in state pre-K. If a year of Head Start does not improve achievement in elementary school, should we assume that a year of state pre-K does?

Research that has examined the impact of state pre-K programs on student achievement is limited in quantity and considerably less causally persuasive than the research on Perry, Abecedarian, Head Start and a number of other preschool interventions based on randomized controlled trials.[4]

The logic model of universal pre-K is that all children benefit and that the political will to invest in early childhood programs will be greater if all families with young children are served. The logic model for targeted programs is that the maximum benefit will accrue to children with the highest risk factors who receive the most intensive services and that the effect of trying to provide a universal service will be to spread available resources thin. This makes an empirical examination of the effects of a universal vs. targeted program of paramount policy importance. No studies have addressed this question in ways that permit a rigorous side-by-side comparison of targeted/means-tested programs vs. universal programs. There are, however, a couple of studies that at least permit cautious inferences.

A study of universal pre-K in Georgia compared changes in Georgia’s 4th grade NAEP scores before and after the implementation of universal pre-K with changes in the NAEP scores of students in other states in comparable periods in which universal pre-K was not introduced. This was not an experiment and therefore required sophisticated estimation models. Under the estimation model preferred by the researcher there was no overall impact on the achievement of Georgia’s 4th graders of their prior access to universal pre-K. Under a more generous estimation model, the overall effects were very small (about 2% of a standard deviation increase in NAEP math scores, which is far less than a 1 point increase). The researcher concludes that:

the costs of the program … greatly outweigh the benefits in terms of potential increased taxable revenue [which] is at least suggestive that the government’s scarce resources would be better spent on more targeted early childhood interventions that have been shown to be more cost efficient…[5]

A second study examined the impact of the Texas state pre-K program, which is targeted for low income families, on scores on 3rd grade state tests.[6] The researchers compared scores for children at the school district level before and after the introduction of the pre-K program, using a number of statistical controls. Positive effects ranging between 5% and 10% of a standard deviation on test scores were found, with the largest effects for children who were both economically disadvantaged and limited English proficient. Note that the effects were larger than in the Georgia universal program and were largest for children with the greatest risk, thus providing some support for a targeted approach.

Note that even the outer bound of the effect as derived from the most impacted subgroup in the Texas study, 10% of a standard deviation, is probably substantially smaller than policymakers or the general public expect from a compensatory preschool program that is understood in the context of the dramatic results from Perry and Abecedarian and the prevailing sense that preschool works. One way to anchor 10% of a standard deviation is to consider that the black-white achievement gap on NAEP is about 1 standard deviation. Thus you might think about the targeted Texas pre-K program as eliminating up to 10% of the problem that early childhood education is supposed to address. This is not a trivial effect by any means, but it does send the cautionary message that even effective targeted state pre-K programs are not silver bullets for the educational challenges that are associated with children from low-income, minority, and limited English proficient backgrounds.

Advocates of universal pre-K point to research conducted in Tulsa, OK to support their claim that middle-class children as well as children from low-income at-risk backgrounds can benefit from a pre-K program delivered by the state.[7] The research design involved comparing children who had just missed the birth date cutoff for entry into pre-K with those that just made the cut. The finding was that at the end of the year of participation in state pre-K the slightly older group scored higher on cognitive tests than the slightly younger group, which had not yet had access to the state pre-K program. There were differences favoring the state pre-K group for children from middle class as well as low-income backgrounds, but the effects were larger for low income children.

Unfortunately the research design of the Tulsa study is critically flawed when used to draw conclusions about the impact of a state pre-K program in that it requires the implausible assumption that any differences in tested outcomes for the two groups of children after the slightly older group finished pre-K were due to pre-K participation by the older group vs. lack of access to such participation by the slightly younger group of children. However, there likely were many differences in the learning environments of the two groups of children in addition to their differential access to state pre-K. Imagine that you have adopted two infants who are only a few days apart in age but whose birth dates happen to fall on different sides of the age cutoff date for entry into pre-K. Your slightly older child is starting pre-K and will be starting public school in a year whereas your slightly younger child has two years to go before she will start public school. Do you treat the two children the same at home during the year that the older child is in pre-K or are you likely to work a bit harder with that child to make sure she is ready for kindergarten? Do your two children have the same playmates or is your slightly older child likely to be playing with the 4-year-olds with whom she is going to school whereas your slightly younger child is playing with 3-year-olds? All you need to do is feel that you or other parents who happened to share the circumstances in this hypothetical example would treat their slightly older child differently than their slightly younger child during the slightly older child’s pre-K year to invalidate the Tulsa study as an credible estimate of the impact of a state pre-K program on student achievement.

These three studies fall far short of providing a convincing case for investment in universal pre-K: The Georgia study finds impacts that are at best very small and do not pass a cost-benefit test. The Texas study provides evidence for value in a targeted program and is silent on the effectiveness of a universal program. The Tulsa study and other studies that use a design that compares children who just meet or just miss the age cut-off for pre-K can’t estimate the impact of state pre-K because they are comparing children that may differ in many experiences in addition to their participation in state pre-K.

This thin empirical gruel will not satisfy policymakers who want to practice evidence-based education. Their only recourse if they have to act is to do so cautiously and with the awareness that they are going to make some mistakes and need to be in a position to learn from them. They and the general public need to be wary of the prevailing wisdom that almost any investment in enhancing access to preschool is worthwhile. Some programs work for some children under some conditions. But, ah me, which programs, children, and conditions?

States desperately need systems in place that allow them to examine and learn from differences in the performance of individual preschool program providers. Most certainly those systems should include the pre- and post-testing of representative samples of children at the individual center level on skills and dispositions that are known to be important for success upon school entry, and the collection of survey information from the parents of these children on their satisfaction with their child’s preschool experience. No states presently do this. How is any state going to improve its early childhood programs or allow parents to pick and choose based on center quality if the relevant information is not collected and used?