The smaller your student loan debt, the more likely you are to default. That's what the latest report from the New York Fed, highlighted by Kevin Drum at Mother Jones, shows:

Nothing on this chart is good news. Going into default on a student loan means payments are at least nine months overdue. And default rates are startlingly high for everybody who started paying back loans in 2009: 26 percent of borrowers have been in default, higher than the historical average for previous classes.

But it's not the people with the biggest loans who are defaulting at the highest rates. It's the people with the smallest loans.

Much of the conversation around student debt focuses on big numbers: the average student who graduated from a four-year college in 2013 with loans had $28,400 in debt. And it's true that a five- or six-figure loan balance can feel like an albatross around your neck and stop you from buying a house, a car, or moving out of Mom and Dad's basement.

But a larger balance also indicates that you probably graduated from college (and a very high student loan balance suggests you probably earned an advanced degree). People with small balances, on the other hand, are more likely to have borrowed to pay for a semester or two before dropping out. And those small loans can grow over time as interest and penalties accumulate.

This chart shouldn't be surprising. The single best predictor of whether you'll be able to pay back your student loans is whether you graduate. Without a degree, you still have to make the student loan payments, but you're not reaping the economic return on investment. The share of borrowers who drop out is large: nearly 30 percent of students with loans who started college in 2003 didn't finish.

As Drum writes, this is the worst possible scenario:

The very lowest debt levels are associated with students who drop out after only a year or so. They have the worst of all worlds: only a high school diploma and a low-paying job, but student debt that's fairly crushing for someone earning a low income. The next tier of debt is likely associated with students at for-profit trade schools. These schools are notorious for high dropout rates and weak job prospects even for graduates. The middle tier of debt levels is probably associated with graduates of community colleges and state universities. Graduates of these schools, in general, get lower-paying jobs than graduates of Harvard or Cal. Conversely, high debt levels are associated with elite universities. Harvard and Cal probably have pretty high proportions of students who earn good incomes after graduation.

He's right on the first point, but the rest is a stretch. Debt levels and college experiences don't line up quite that neatly, and neither do college experiences and early-career earnings. The typical student at a for-profit college borrows more than the typical student at a community college. Elite universities, with their generous financial aid, aren't the real debt machines. Those would be nonprofit colleges with small endowments whose students don't come from super-wealthy families.

But Drum is absolutely right that college dropouts struggle the most when it comes to debt. So the big numbers in the student debt crisis seem frightening — but it's the small ones we should really worry about.