I had a teacher once who called his students "idiots" when they screwed up. He was our orchestra conductor, a fierce Ukrainian immigrant named Jerry Kupchynsky, and when someone played out of tune, he would stop the entire group to yell, "Who eez deaf in first violins!?" He made us rehearse until our fingers almost bled. He corrected our wayward hands and arms by poking at us with a pencil.

Today, he'd be fired. But when he died a few years ago, he was celebrated: Forty years' worth of former students and colleagues flew back to my New Jersey hometown from every corner of the country, old instruments in tow, to play a concert in his memory. I was among them, toting my long-neglected viola. When the curtain rose on our concert that day, we had formed a symphony orchestra the size of the New York Philharmonic.

I was stunned by the outpouring for the gruff old teacher we knew as Mr. K. But I was equally struck by the success of his former students. Some were musicians, but most had distinguished themselves in other fields, like law, academia and medicine. Research tells us that there is a positive correlation between music education and academic achievement. But that alone didn't explain the belated surge of gratitude for a teacher who basically tortured us through adolescence.

We're in the midst of a national wave of self-recrimination over the U.S. education system. Every day there is hand-wringing over our students falling behind the rest of the world. Fifteen-year-olds in the U.S. trail students in 12 other nations in science and 17 in math, bested by their counterparts not just in Asia but in Finland, Estonia and the Netherlands, too. An entire industry of books and consultants has grown up that capitalizes on our collective fear that American education is inadequate and asks what American educators are doing wrong.

I would ask a different question. What did Mr. K do right? What can we learn from a teacher whose methods fly in the face of everything we think we know about education today, but who was undeniably effective?