Rising college costs and a sagging economy are taking the biggest toll on a surprising group: upper-middle-income families.

According to a Wall Street Journal analysis of recently released Federal Reserve data, households with annual incomes of $94,535 to $205,335 saw the biggest jump in the percentage with student-loan debt from 2007 to 2010, the latest figures available. That group also saw a sharp climb in the amount of debt owed on average.

The...