Does the insertion of tech devices and software–aka “edtech”–into classrooms significantly improve students’ learning? Do headline-attracting tech initiatives such as “A tablet for every student” ever live up to the hype?

Larry Cuban, Professor Emeritus of Education at Stanford University, has spent much of his career considering these questions. Even though he is now retired, his continuing interest in edtech led him recently to research and write a new book, The Flight of the Butterfly or the Path of a Bullet? Using Technology to Transform Teaching and Learning.

In an interview for the New Books Network, I recently had the privilege of discussing this book–and also the general lessons from his decades of study–with Dr. Cuban.

To conduct his research, Cuban first sought examples of “exemplar” teachers who had a reputation for integrating technology successfully into their K-12 classrooms. Because he was interested in assessing the best possible success stories for edtech, he also chose among teachers within Silicon Valley school districts. Cuban recognized that schools in the Valley offer an ideal proving ground, as they are surrounded by a technophile community and major tech players (such as Google), many of which donate devices and services to area schools.

After identifying exemplars in the area, Cuban sat in on 41 classes and observed how teachers and students made use of technology in their everyday activities. Typical devices included tablets or Chromebooks provided to each student, as well as use of interactive white boards by teachers.

As he monitored specific classrooms, Cuban sought to determine whether the technology had been fully adopted and integrated. More importantly, he tried to assess whether tech had made a real difference in education, or if it simply provided a different method to reach a conventional result.

Cuban dedicates much of The Flight of the Butterfly to descriptions of exemplar teachers in action. Larger questions aside, his profiles offer positive, inspiring accounts of dedicated teachers who have seamlessly integrated edtech into their lesson plans. For example, he describes seventh-grade teacher John DiCosmo’s lesson on John Steinbeck’s The Pearl:

“The forty-five minute lesson format used the whole group, small group, and independent activities in a fast-moving sequence, each seldom lasting more than ten to fifteen minutes each. As the lesson unfolded, DiCosmo segued from whole group discussions on the book and preparation of a video trailer to listening to a reading of the first chapter of ‘The Pearl’ to students independently taking notes to a competitive, rousing vocabulary game. Students easily shifted from jotting down notes to looking up words on their tablets…. The back-and-forth between old and new technologies appeared seamless to me.”

But at the end of the day, did the students appreciate The Pearl and improve their language skills better than they would have fifteen years ago in the same classroom, without devices?

Consider the evolution of the black board, a fundamental piece of school technology. Over the past two centuries, the original slate boards were replaced by green boards, then dry-erase white boards, followed by the now-pervasive interactive white boards. As Cuban asks, does this gradual conversion “represent a minor or major change in teaching practice?” Certainly, each step in the board’s evolution improved efficiency, but the board’s essential function and effect may not have changed.

After surveying the teachers in his study and adding his own observations, Cuban concluded that other contemporary edtech had similarly improved efficiency, but also had modified education only incrementally. To be sure, some of the teachers in the study said that technology had truly transformed their classroom experiences–but even then, those major improvements were limited to the individual class and teacher.

This leads to another theme from The Flight of the Butterfly and Cuban’s larger body of research: it is exceedingly difficult to reform education at a broader level, such as a school district. Cuban reminds us of the complexity of school organizations, with all of their politics, turnover, funding issues, conflicting ideologies, and heterogeneous groupings of individual teachers. In the midst of that complexity, high-level efforts to apply top-down reform typically fail.

For the same reasons, system-wide initiatives to deploy technology (such as computers in every classroom) are often “Oversold and Underused,” as another Cuban book title puts it.

But because of the dedication of so many good teachers, along with their personal, variable experiments with edtech and other trial-and-error innovations, improvements do occur over time. Those modifications can gradually, randomly combine to result in significant changes in how students learn and retain knowledge. These improvements, then, tend to follow the indirect, random path of a butterfly’s flight, rather than the direct, instantaneous flight of a bullet’s path, which administrators and policymakers may seek.

To hear more about the book, Professor Cuban’s reflections on his career, and some of his thoughts on higher education, you can listen to the interview here.