One of the few rigorous studies of its kind shows that first-grade math teachers tend to use unproven alternative techniques when there are more math-challenged students in their class.

Believe it or not, there is not much reliable research that indicates which teaching techniques are most effective for teaching young children math. It’s a Russian nesting doll situation where the largest doll is already quite small, says Paul Morgan, associate professor at Pennsylvania State University.

“There’s not a lot of research in elementary grades on math achievement,” he says. “Within those, there aren’t many estimating the effects of instruction in rigorous ways that control for other factors.” Even fewer studies consider how teaching techniques can affect students with learning difficulties.

The lack of research in this area is alarming for a few reasons. In the near term, a 2009 study by Morgan and his colleagues found that students who exhibited difficulties with math in kindergarten continue to show poor achievement throughout elementary school. In the long term, other studies have shown that students who are bad at math go on to be paid lower wages and are more likely to be unemployed, even if they have higher reading skills.

Morgan’s latest study, published today in Educational Evaluation and Policy Analysis, shows that teachers tend to use different instructional techniques when their classrooms have more math-challenged students. Unfortunately, these techniques either had no positive effect on students’ achievement, or they helped students who were already doing well in math while leaving those who struggled even further behind.

Teachers tend to use different instructional techniques when their classrooms have more math-challenged students. Unfortunately, these techniques had no positive effect on students’ achievement.

Morgan and his colleagues analyzed a data set from 1998-99 to conduct one of the most rigorous investigations on this topic to date. The nationally representative sample includes 13,393 children attending 3,635 classrooms in 1,338 schools. Teachers filled out reports on how often they used different teaching techniques and taught certain math skills throughout the school years.

The robust data set made it possible to control for a dizzying number of potentially confounding factors at the individual, classroom, and school levels. They included factors that account for a huge amount of the variation in academic achievement, including past achievement (in this case, kindergarten math scores), parents’ socioeconomic status, and parents’ education levels.

Researchers identified four main teaching techniques. The most traditional were categorized as "teacher-directed activities," where teachers first demonstrate how to solve problems and then ask students to practice on their own. “Student-centered activities,” on the other hand, allow students to generate their own mathematical ideas and apply them to real-life situations, often working in groups. In the “manipulatives/calculators” technique, students work with physical objects (M&Ms, pennies, etc.) to symbolize math problems and/or used calculators. Lastly, the “movement/music” category included techniques in which students physically moved around or used music.

The results indicated that teacher-directed activities were the most effective techniques in terms of improving students’ academic performance. Student-centered instruction only improved outcomes among students who did not previously exhibit difficulty learning math.

Most troublingly, teachers who taught classrooms with higher percentages of students exhibiting math difficulties were more likely to use manipulatives/calculators and movement/music techniques. When students are struggling with topics or losing interest, it might seem intuitive to teachers in the classroom to use physical objects, music, or movement to attempt to engage these students. But empirically, manipulatives/calculators and movement/music techniques were not associated with academic gains at all.

A possible reason for the discrepancies, Morgan hypothesizes, is that student-centered learning and other alternative techniques tend to demand more, cognitively, from students. Those who already have trouble learning math can be distracted by handling M&Ms or moving around while attempting to grasp mathematical concepts. In student-centered instruction, students have to juggle interacting with their peers while considering different solutions to a problem. This could have big implications for Common Core, the educational policy championed by the Obama administration and adopted by 45 states. The Common Core standards emphasize using different approaches to solve the same problem and applying mathematical concepts to real-life situations—both of which are categorized as student-centered in Morgan's study.

While the researchers recognize that using a data set from more than a decade ago is a major limitation, Morgan points out that first-grade teaching techniques and math skills haven’t changed much since then. Larger factors like No Child Left Behind and the economic recession of 2008 might mean results would be different today than in 1998, but the benefits of having such a robust data set to control for outside factors offset these limitations. Additionally, Morgan acknowledges that the effect size of these teaching techniques is relatively small. However, considering many of the factors affecting student performance are out of educators' hands (socioeconomic status, family history, etc.), any controllable factor—such as teaching technique—should be fully examined.

“We’re all trying to help kids as much as we can," he adds. "No study is perfect, every study has its limitations. But we don’t want to be making decisions for kids based on anecdote. We should try to use empirical findings and evidence, as limited as it may be, to guide our decision making.”