Not long ago, Wall Street attracted many of the smartest people in America—brilliant economists and mathematicians, but also the nation's most promising college graduates. Not anymore. Five years after the financial crisis, evidence suggests that students have lost their enthusiasm for beginning careers in finance.

In 2008, Harvard sent 28% of its graduating class to firms where they would become bankers, traders or investors. By 2010, Harvard was sending just 17% of its graduating class. Yale graduates entering the finance business fell to 14% from 26% during the same period. Even Princeton, traditionally the most finance-friendly school, fell to 35.5% from 40%.

The media have taken note. "Harvard MBAs Flee Wall Street, Take Pay Cut," ran a headline in Bloomberg Businessweek in July. The month before, MarketWatch noted a "Silicon Valley vs. Wall Street" talent war in which "more graduates with 'quant' skills are choosing tech over finance."

No doubt the financial crisis, with its widespread layoffs, damaged Wall Street's allure for graduates. And Silicon Valley certainly holds out the promise of getting paid well and maybe even making a fortune—while not having to wear a suit or tie. As the U.S. economy slowly recovers, maybe working in finance will also recover some of its attraction for graduates.

But I worry that it won't—that Wall Street has suffered reputational damage, thanks to a few bad actors, that can't be undone simply by waiting for memories to fade and an economic boom to kick in. I'm concerned that those of us in financial services have forgotten who we serve—and that the public knows it.