As a teacher I’m told a lot of stories about how education works and how we are going to fix it. We tell the stories because they give us easy-to-understand enemies along with optimistic explanations of our problems. But these stories are not often based in evidence. And when they aren’t clearly wrong, they are much more complicated than the stories on the ground. I’m going to try to collect the stories I’m told and do my best to show why they are probably wrong or at least not clearly right.

1. The US spends a lot on education

Spending is the guiding principle for how most people make sense of education policy. We have very high expectations for what our public schools need to offer and, on top of that, we frequently assume that reform means more spending.

This spiral has led to the United States spending more than almost any other developed country despite having poor relative rankings on international measures of education quality. This misconception drives a lot of the dysfunction and gluttony in the system. We can’t just spend our way out of education problems. We’ve tried and it’s led to education being one of the largest parts of the US federal budget despite the fact that most of us are dissatisfied with the results.

2. Misconceptions about student funding

A common misconception about US education is that the property-tax based funding of local school districts makes it so that poor students are underfunded relative to non-poor students. There’s some truth to this statement. A lot of districts do fund schools based on property taxes and there are large differences in school funding between states. This spending disparity closely matches the actual educational ranking of the states. This, again, makes it look like poorer students are being left out to dry, and although that is the case in some cases, on average US school funding is somewhat progressive.

The combination of state, local, and federal school funding makes it so that the districts attended by poor students are funded 2.5% more than non-poor students. And even within districts, “schools with less advantaged students spend at least as much (and often significantly more).”

source from RCA post

On average, poor students don’t receive less funding, but there are a lot of confounding factors. Although the majority of states are progressively funded, there are some examples like South Carolina and Tennessee that have about the same amount of spending per student.

On top of that there’s also a clear disparity in extra voluntary funding per district. Parents will end up donating huge amounts of money to their children’s school to help pay for extracurriculars, facilities, and support staff. There’s a Vox Future Perfect podcast episode on this (episode named “Your PTA vs Equality”) that details the fight within a wealthy California community in redistributing donations across the district. Basically, parents weren’t allowed to directly donate to their children’s school. Instead any donation would be re-distributed evenly. It’s an extreme example that is unrepresentative of most of the country. Parent donations still make up a fraction of per-student spending and so I’d be skeptical of anyone who presents this story as being a particularly impactful lever for change in US education.

You could also argue that education spending within poor urban districts has less purchasing power than suburban districts. I couldn’t find research on this and it certainly won’t describe every district in the United States, but it might be worth considering.

For example, take the district I teach in, Nashville. Nashville property prices are booming, the city is swelling with new arrivals, and the cost of living is exploding. That means that school property prices are increasing alongside teacher cost-of-living. The district is also becoming increasingly regulated in terms of zoning and building restrictions. That means that any new building could end up being considerably more costly than many suburban areas. The district will also likely need to pay teachers more in order to meet basic quality of life. All of these factors could add up to making it so that every dollar that goes to suburban and rural districts goes a lot further than every dollar spent in urban districts.

Overall, we spend more on poor students than non-poor students. At the very least, we can’t keep scapegoating all of our education problems on school funding. I should say, though, that getting real about education funding could still lead us to spending even more on poor students. We can frame that as closing the gap made by parent donations to richer schools or by just acknowledging that our poor schools just need a lot more funding to be successful. That’s okay and it’s a conversation that’s still worth having.

3. Misconceptions about teacher pay

As a teacher, I feel weird writing about this. I don’t think I’ve ever met a teacher who felt that they are paid enough, this is all despite the fact that we get at least 12 weeks of paid vacation alongside the best benefits of almost any profession.

By almost all objective measures, teachers don’t actually seem to be underpaid in the traditional sense. “Nationwide, the average teacher salary was $60,477 during the 2017–18 school year.” And teachers work around 2 hours less a week than the average profession (“ 40.6 hours during the work week, compared to 42.4 hours for private-sector professionals”). BLS Occupational Information Network studies found that teaching isn’t a particularly stressful job relative to other professions, and teachers typically have pretty solid relative job security. In addition to all of this, teachers don’t make a lot when they leave teaching and education majors have the lowest standardized test scores of any major. Although teachers have extraordinarily high social prestige, it looks like we are paid close to what we are worth on the actual job market.

Now, of course, this comes with some caveats. When we say that someone is underpaid we aren’t making the same claim that an economist is. We are saying that the importance of their job is misaligned with their salary. Teachers score right next to Lawyers and Doctors in terms of occupational prestige but they are paid significantly less. So, in this vague sense, yes, maybe teachers are underpaid. But regardless of that, the blanket push to “fix the system by paying teachers more” is almost certainly misguided, probably ineffective, and also incredibly expensive given that teacher salary is already one of the largest parts of education spending.

4. Misconceptions about teacher supply

Some districts have trouble keeping teachers. Overall though, teacher quit rates are lower than most professions, and the actual supply of teachers is pretty good when you look at scholarly economic studies of the problem. (It should be said that Union funded thinktanks like EPI do argue that there are teacher shortages)

“ Another consequence we would expect if teachers were underpaid is widespread teacher shortages. According to a trend-setting New York Times article from 2015, “districts are struggling with shortages of teachers…[as] a result of the layoffs of the recession years combined with an improving economy in which fewer people are training to be teachers.” Today, an internet search of news media sources finds 262,000 references to “teacher shortages,” often attributing such shortages to low teacher pay. And yet, as Kate Walsh of the National Council on Teacher Quality points out, there is “no real data — because neither states nor the federal government collect the data that’s really needed to pronounce the onset of a teacher shortage.” Instead, claims of a teacher shortage are heavy on anecdote and speculation.”

The trouble is that we have a low supply of teachers willing to work in difficult districts.

All of my teaching friends in Nashville have experienced some shortage in their respective school. Anecdotally, most teachers are looking for cushy positions in suburban districts where classroom management and discipline is easy. When I was growing up I was lucky to attend a high-quality suburban school that just had a culture of treating teachers with respect. There were very few disciplinary issues in the school. It was just a given that you didn’t talk during class. Consequently, there was only a basic disciplinary system. You did something bad, you got a detention. You did something really bad, you got sent to the principal’s office. That was it.

The story in urban schools is different. Arguably, the majority of an urban teacher’s job is classroom management. The successful schools almost always have complicated and multi-tiered disciplinary systems that are mislabelled by journalists and policymakers as “no excuse” policies. Not all disciplinary systems are “no-excuse” policies, but that doesn’t matter, the buzzword has power in the press.

These policies typically look pretty similar across the board. You have a series of leveled demerits that add up to a detention and eventually a suspension or call home. You talk in class, well that’s a level 1 demerit. You say the f-word, that’s a level 2 demerit. You hit someone and that’s a level 3 demerit and you’re going to get sent out and put into a restorative room where you will talk with someone about the situation.

The urban schools that don’t have complicated and tiered systems are frequently mismanaged and ineffective. The students often run the school. Subsequently, with the exception of a very few exceptional teachers who run their rooms through likability and relationships, the majority of classrooms contain very little learning.

A lot of teachers don’t want to work in these schools either because they hate implementing the disciplinary systems or they can’t handle student disrespect.

You really have to have a conviction to work in these schools. But unfortunately, a conviction isn’t enough to meet long-term supply needs in many tough schools. Because of this, you will have schools where a third of teachers will leave before the second semester and well more than half of teachers will not return for the following year. This creates cycles of under-served students who have to deal with long-term substitutes (who never really teach anything) and inconsistent school cultures (due to the revolving faces of administration and teaching in the school).

In sum, it looks like there isn’t a teacher shortage in the traditional sense, but there are a lot of urban and rural schools with a shortage of willing teachers. This obviously needs to be remedied but it won’t be fixed by lying about the statistics.

One possible reform is to push teachers unions and local districts into adopting differential pay according to subject and school. Right now, almost all districts have the same pay schedule across all teachers and all schools despite the fact that we have a relative surplus of humanities teachers and a relative shortage of STEM teachers.

Take Nashville, the gym teacher in the top performing school in the district (e.g. Hume Fogg) is being paid the same as the math teacher in the worst high schools (e.g. Pearl Cohn). Despite the fact that there is a long line of willing and qualified people to be a gym teacher in a magnet school, that person is being paid the same as the teacher working in an undersupplied, state-tested subject in the most at-risk school.

It seems like our explanations for problems are often shaped by whether they lend themselves to easy solutions. This is particularly true when we are considering the reasons for the achievement gap between students of color and white students.

Unfortunately this again is much more complicated than we want it to be. The dominant explanation is that the achievement gap is because of a lack of good teachers in urban and rural schools. But when you look at actual assessments of teacher quality, it appears that poor and non-poor students have roughly the same quality teachers. There are some disparities but these are not big enough to be able to account for the achievement gap.

For the teachers who do last in Title 1 schools, it appears that they are generally high-quality relative to teachers who work in non-poor districts. But as we already know, many urban districts have considerable issues in teacher retention and teacher quit rates. These districts are outliers relative to the national averages. So while there is a way in which teacher-based factors obviously have a large impact on student achievement, it is likely not the factors that we typically consider.

6. Having a doctorate or masters does not make you a better teacher

Another gut reaction to poorly performing schools is to assume that requiring more educated teachers will translate to having more effective teachers. That is not true. Having a masters or doctorate in education does not make you a better teacher. Graduate programs for education are expensive and the majority of evidence shows that they don’t make teachers any better.

7. Bad test scores are not (all) because of bad schools

One naive way of viewing education is to see poor test scores and assume bad schools. This is partially the case and it is obviously up to school districts, schools, and teachers to do as much as they possibly can in order to support their students. The trouble is that we greatly overestimate the influence of schools. All of the best economic estimates find that school-based factors control only around 30% of a student’s academic outcomes.

This does not mean that schools don’t matter. Schools still matter tremendously but this does mean that schools alone can’t be fully blamed for student outcomes.

Schools control what they can and we should obviously be hellbent on improving our schools, but we can’t scapegoat education as the single reason for why particular students end up having bad outcomes. And when we do that we will end up not embracing the full suite of social policies that will be required to truly pull people out of poverty in the US.

8. Is it not easy to measure teacher performance (“just look at student growth!”)

A lot of well-intentioned education reform is built on the idea that we can measure whether a teacher is good and then respond accordingly. If the data tells us who is good and who is bad then we can fire the bad teachers and pay the good teachers more.

A common assumption is that this is a pretty easy feat: just measure the amount of growth that students have from the previous year. This base idea is called “value added modelling” (VAM). And to be honest there are a lot of people who think that we can measure teacher quality to some degree. But almost no one involved thinks that this is easy or that it’s a perfect measure. There are arguably more people who think that VAM doesn’t work and likely never will.

A claim like “VAM accurately predicts test scores” is kind of circular, since test scores are what we used to determine VAM. But I think the people in this field try to use the VAM of class c to predict the student performance of class c + 1, or other more complicated techniques, and Chetty, Rothstein, and Rivkin, Hanushek, and Kane all find that a one standard deviation increase in teacher VAM corresponds to about a 0.1 standard deviation increase in student test scores.”

It ends up that VAM is very difficult to do properly and for the most part a teacher’s VAM does not correlate well to itself year after year. There’s even more interesting research that shows that a teacher’s VAM for their current year is best predicted by the VAM of the teacher for their current students prior-year.

“ Rothstein (2009) tries to “predict” students’ fourth-grade test scores using their fifth-grade teacher’s VAM and finds that this totally works. Either schools are defying the laws of time and space, or for some reason the kids who do well in fourth-grade are getting the best fifth-grade teachers. Briggs and Domingue not only replicate these effects, but find that a fifth-grade teacher’s “effects” on her students in fourth-grade is just as big as her effect on her students when she is actually teaching them, which would suggest that 100% of VAM is bias.”

In other words, it looks like what VAM measures is very well connected to confounding factors that aren’t necessarily related to the teacher’s actual ability. On top of all of this there is also evidence that VAM is very poorly correlated with in-person rubric-based assessments about teacher performance which are likely to be the actual gold-standard for assessing teacher ability.

9. Charter schools are not bad for traditional public schools

One common complaint about Charter Schools is that they hurt the local Traditional Public Schools. This makes a lot of sense because it looks like it is stealing money directly from the schools by decreasing student enrollment.

To be fair, Charter Schools are a big and complicated problem and every instantiation of charter schooling is different. However a lot of the studies that try to measure the impact of Charter Schools on Traditional Public Schools show that Charter Schools often improve the performance of local public schools because of the extra competition for students. Now obviously this not meant to be a final word on the problem, it’s mainly to say that the constant rhetoric about charter school impact is mostly anecdotal and not evidence-based. A clear-headed accounting of the research shows a very different picture.

10. Charter school success is not because of cherrypicking students.

It is commonly argued that charter schools are only successful because they are able to cherrypick the students who get into the schools (sometimes called “cream skimming”). This view is common and has been pushed forward by many news outlets. A recent Reuters report about some of the worst abuses of charter school cherrypicking has had a lot of influence on the general public’s sensitivity to the issue.

In the literature this problem is called selection bias. The idea is that if you can get the best students in your school those students scores will be successful regardless of what the school actually does. Now this can happen in some circumstances. That is always a possibility. However, its prevalence is considerably over reported.

When you look at the research you’ll find that most studies on the subject find that there is either little evidence of cherrypicking in charter schools or the effect is very small. For this particular topic we need to be especially careful of evidence that is overly anecdotal or biased by funding source since the majority of actual scholarly reviews finds little evidence for selection bias.

For instance, education writer Fredrik deBoer recently wrote a book, Cult of Smart, whose entire policy section was built around selection bias. Without even attempting to assess the research on the topic, he came to the blanket conclusion that all Charter School success is due to selection bias.

Selection bias happens, some competitive charter schools have nefarious policies that are built to juice the numbers. We should push back against these practices, but it is entirely dishonest to try and explain argue that charter school success is only because of selection bias.

11. Charter School success is not only because of “no excuse” disciplinary policies

Another common scapegoat for explaining away the success of charter schools is to say that charter schools are only successful due to Draconian “no excuses” disciplinary policies.

Now once again there can be situations where the disciplinary policies of schools can look very tough and borderline unfair to people who do not work in the schools. There is no doubt about the fact that structured disciplinary policies improve the overall success of the school, however there are a lot of ways in which journalist can be unfair in how they assess these disciplinary policies.

The term “no excuse” is typically a misnomer. It doesn’t describe the majority of actual disciplinary policies that charter schools use. “No excuse” or “zero-tolerance” policies typically describe the system’s popularized in the 90s to try to ensure that schools would not have issues with violence or drugs. You would have a “zero tolerance” policy on bringing weapons, drugs, or violence to the school — one time and you’re going to be sent to another school. These policies have been taken to the extreme in some cases and they have a record of being very destructive to some communities.

However, Charter School policies are not necessarily “no excuse” policies. Instead they tend to use complicated multi-tiered disciplinary systems that include a lot of restorative practices along with the use of social workers and counselors to try to mediate problems and provide a long range of incentives to improve student behavior while maintaining school safety.

This is not the case in every school, but there are literally thousands of charter schools across the country. The coverage of them is mostly in bad faith, and what is often labelled as “no excuse” is not necessarily problematic or destructive. At least anecdotally, my own school’s disciplinary system is frequently cited by my student’s parents as their favorite aspect of the school.

To people who do not work in education, this can seem unfair because it is so drastically different from the typical suburban school. However, the alternative is often safety concerns and classrooms where almost no learning occurs.

Disciplinary policies are arguably the most important aspect of a school’s health. In some cases the studies show that without no excuses policies — or, in this case, without strong disciplinary policies — Charter Schools perform as well as public schools. However I do not personally think that tough and structured disciplinary policies should be seen as a charter school failure that should be factored out. When we just throw around the term “no excuse” to describe all charter school disciplinary systems, we are starting with a negative connotation to a practice that is uniformly found to be extremely effective in improving school outcomes.

If anything, we should see all of these studies and put more work into figuring out how to best build disciplinary systems that maximize learning while also prioritizing student mental health. For anyone who actually has experience in education it is very clear that having a very structured school environment can be very helpful and, in most cases, necessary.

12. Preschool does not improve academic achievement

Universal preschool is a very important and commonly misunderstood program. Preschool has not been around for very long, however it is commonly seen as one of the most important factors in how we improve the outcomes of underserved children who do not have a lot of structure at home.

Research on universal preschool pretty convincingly shows that it does not improve academic outcomes. In fact, there is even some evidence that universal preschool can lead to increased rates of ADHD. Despite having no academic improvements and in some cases increased mental health problems. There is some evidence that Universal preschool can have positive impacts on pro-social behavior and later job outcomes.

The important thing is to be honest about the limitations and benefits of universal preschool. Universal preschool does not appear to be a catch-all solution to Poverty. Universal preschool does not appear to improve academic outcomes, however, it may end up having benefits that we did not previously anticipate.

13. Smaller schools and classes are not a fix for the system

The Gates Foundation experimented with building small schools back in the 2000s. The main takeaway was that smaller schools did not actually improve student outcomes. Aside from smaller schools, there is evidence that smaller class sizes can lead to better student outcomes. The trouble is that our school systems are already very costly and ineffective. Small class size is likely to be the most expensive way to improve student outcomes since it would increase the required number of teachers.

Teacher salary is already one of the largest costs in education and each new hire represents a new cost to the system in training and integrating them into existing curricular and disciplinary systems. It is not clear whether this is a scalable solution to “fix” our education problems.

14. Public charter schools don’t hurt minority and poor students

I just want to inject uncertainty into how most people in education should think about these topics. The majority of evidence shows that charter schools, on average, improve student outcomes especially for underserved students of color. If we are to attack charter school systems we will likely be hurting many students who are being benefited by these institutions.

source: charter schools are most effective in urban settings and almost never perform well in rural settings

The trouble is that our ideas about education reform are almost never informed by what works, instead they are built on our political ideology. If you’re conservative, you’re almost certainly going to favor the “market based” incentives for charter schools. If you’re progressive, you’re almost certainly going to despise union busting charter schools and favor more support for traditional public schools. The trouble is that ideology and political theory isn’t good at predicting results.

In the case of charter schools, urban students universally benefit, while nonurban students don’t. The truth is somewhere in the middle, and, to me, we should build policy decisions based on what the evidence has shown to be most effective. Charter schools are good for urban students, they aren’t a savior for our education system. We should support charters in urban districts and push against charters in nonurban districts.

15. Misconceptions about property-tax-based funding for public schools

Property based funding of schools is not likely to be a very effective target for school reform since our current system does not actually have large differences in the funding of poor students. I think that it is more likely that the dysfunction in the schools is best explained by a lack of continuity and efficiency within the schools that serve poor students.

The property tax debate is mostly just an easy but misguided target for explaining the achievement gap. On top of the pure fact that property based taxes have actually still allowed for the progressive funding of schools, we already have examples of states that fund students without a total reliance on local property taxes.

Take Michigan. Michigan has a centralized funding source for their students. The state takes an overall tax and then breaks it up evenly across students within the state (there are extra complexities to this that are addressed in this article). Even with these changes in school funding, it does not appear that the reformed funding strategy had any impact on student outcomes.

In general, funding is not a good proxy for educational quality.

16. It is very difficult to improve teacher quality and professional development is ineffective.

Another common hope for improving student outcomes is to improve teacher quality through rigorous systems of evaluation and professional development. Urban educators are constantly being observed, evaluated, and then pushed to improve their teaching through many hours of professional development each week after school.

I understand why schools feel like this would be effective. We have a bias towards thinking that doing something will always be better than doing nothing. This is called the action bias and we see it commonly in policy. The trouble is that our perception is acutely built for us to assume that our actions are paying off without considering the actual evidence that would show whether something was effective.

In education it can take a long time for interventions to pay off and those interventions will frequently have very small gains. I want it to be true that professional development and rigorous feedback system can improve teaching quality, however I suspect that we are very biased in our implementation of these actual programs.

In a recent RAND study on improving teacher effectiveness, they found no gains in student performance from implementing rigorous systems of teacher feedback, professional development, and mentorship.

“The Intensive Partnerships for Effective Teaching initiative, designed and funded by the Bill & Melinda Gates Foundation, was a multiyear effort to dramatically improve student outcomes by increasing students’ access to effective teaching. Participating sites adopted measures of teaching effectiveness (TE) that included both a teacher’s contribution to growth in student achievement and his or her teaching practices assessed with a structured observation rubric. The TE measures were to be used to improve staffing actions, identify teaching weaknesses and overcome them through effectiveness-linked professional development (PD), and employ compensation and career ladders (CLs) as incentives to retain the most-effective teachers and have them support the growth of other teachers. The developers believed that these mechanisms would lead to more-effective teaching, greater access to effective teaching for low-income minority (LIM) students, and greatly improved academic outcomes. Beginning in 2009–2010, three school districts — Hillsborough County Public Schools (HCPS) in Florida; Memphis City Schools (MCS) in Tennessee (which merged with Shelby County Schools, or SCS, during the initiative); and Pittsburgh Public Schools (PPS) in Pennsylvania — and four charter management organizations (CMOs) — Alliance College-Ready Public Schools, Aspire Public Schools, Green Dot Public Schools, and Partnerships to Uplift Communities (PUC) Schools — participated in the Intensive Partnerships initiative. RAND and the American Institutes for Research conducted a six-year evaluation of the initiative, documenting the policies and practices each site enacted and their effects on student outcomes. This is the final evaluation report.”

Here are their main findings:

“The sites succeeded in implementing measures of effectiveness to evaluate teachers and made use of the measures in a range of human-resource (HR) decisions. Every site adopted an observation rubric that established a common understanding of effective teaching. Sites devoted considerable time and effort to train and certify classroom observers and to observe teachers on a regular basis. Every site implemented a composite measure of TE that included scores from direct classroom observations of teaching and a measure of growth in student achievement. Every site used the composite measure to varying degrees to make decisions about HR matters, including recruitment, hiring, and placement; tenure and dismissal; PD; and compensation and CLs. Overall, however, the initiative did not achieve its goals for student achievement or graduation, particularly for LIM students. With minor exceptions, by 2014–2015, student achievement, access to effective teaching, and dropout rates were not dramatically better than they were for similar sites that did not participate in the Intensive Partnerships initiative. There are several possible reasons that the initiative failed to produce the desired dramatic improvement in outcomes across all years: incomplete implementation of the key policies and practices; the influence of external factors, such as state-level policy changes during the Intensive Partnerships initiative; insufficient time for effects to appear; a flawed theory of action; or a combination of these factors.”

This was a very well-funded study supported by the Bill and Melinda Gates Foundation and it was held over the course of six years. It might be the case that a perfect storm of professional development could end up being effective in other circumstances. It can be very hard to generalize the results of one policy experiment onto other contexts. However, in this case, we have very good reason to believe that expensive systems of professional development do not have as much of a positive impact as we would assume.

It might be more effective to get rid of professional development and drop many systems of feedback and observation because they can negatively affect teacher retention while at the same time being ineffective in improving teacher quality.

A common stressor for teachers in high-quality Charter Schools is that they cannot stay in the school long because the cycles of observation, feedback, and professional development over-stress the teachers. It might be worth dropping these systems all together, but there needs to be more research on this specific tradeoff.

One common belief within the education system is that students know best. The idea is that students can have a very good understanding of whether a teacher is actually good. So the student could end up being a good tool in evaluating the effectiveness of the teacher. However this is not really true.

The majority of studies on student evaluation find that they are negatively correlated with teacher effectiveness. Worst off, it is commonly found that student evaluations are especially biased towards the race and gender of their teachers. Teachers who are black or female are typically more poorly rated than white male teachers despite the fact that the studies control for teacher quality.

I have to administer student evaluations for Teach for America. Although these are not strictly connected to my pay or my status in the organization, these evaluations are still supposedly used by my instructional coach to optimize the feedback that I get on my teaching ability.

I understand the reason why we do this. It makes sense to try to validate the voices of students and, in general, there are a lot of positive mood affiliations with assuming that students know best. However, this is not clearly the case and including student evaluations in teacher quality measurements and/or hiring decisions will lead to problems in terms of teacher diversity and teacher quality. It can also create a counterproductive system of incentives between teachers and students.