It’s no secret that falling behind on student loan payments can squash a borrower’s hopes of building savings, buying a home or even finding work. Now, thousands of retirees are learning that defaulting on student-debt can threaten something that used to be untouchable: their Social Security benefits.

According to government data, compiled by the Treasury Department at the request of SmartMoney.com, the federal government is withholding money from a rapidly growing number of Social Security recipients who have fallen behind on federal student loans. From January through August 6, the government reduced the size of roughly 115,000 retirees’ Social Security checks on those grounds. That’s nearly double the pace of the department’s enforcement in 2011; it’s up from around 60,000 cases in all of 2007 and just 6 cases in 2000.

The amount that the government withholds varies widely, though it runs up to 15%. Assuming the average monthly Social Security benefit for a retired worker of $1,234, that could mean a monthly haircut of almost $190. “This is going to catch an awful lot of people off guard and wreak havoc on their financial lives,” said Sheryl Garrett, a financial planner in Eureka Springs, Ark.

Many of these retirees aren’t even in hock for their own educations. Consumer advocates say that in the majority of the cases they’ve seen, the borrowers went into debt later in life to help defray education costs for their children or other dependents.

Harold Grodberg, an elder law attorney in Bayonne, N.J., said he’s worked with at least six clients in the past two years whose problems started with loans they signed up for to help pay for their grandchildren’s tuition. Other attorneys say they’re working with older borrowers who had signed up for the federal PLUS loan—a loan for parents of undergraduates—to cover tuition costs.

Other retirees took out federal loans when they returned to college in midlife, and a few are carrying debt from their own undergraduate or graduate-school years. (No statistics track exactly how many of the defaulting loans fall into which category.)

Most consumer advocates and attorneys who work with seniors in this predicament told SmartMoney.com that their clients were unwilling to speak on the record, because of shame or fear. But they all stress that stakes involved can become very high for older people on a budget.

Deanne Loonin, a staff attorney at the National Consumer Law Center in Boston, said she’s been working with an 83-year-old veteran whose Social Security benefits have been reduced for the past five years. The client fell behind on a federal loan that he signed up for in the ‘90s to help with his son’s tuition costs; Loonin said the government’s cuts have left the client without enough cash to pay for medications for heart problems and other ailments.

Roughly 2.2 million student-loan debtors were 60 and older during the first quarter of 2012, and nearly 10% of their loans were 90 days or more past due, up from 6% during the first quarter of 2005, according to the Federal Reserve Bank of New York. “It’s really a unique problem we haven’t had to face before, and it’s only going to grow,” said Robert Applebaum, founder of Student Debt Crisis, a nonprofit advocacy group in Staten Island, N.Y.

The threat of Social Security cuts adds to the overall financial woes faced by the aging baby boomer generation. Almost 45% of people aged 48 to 64 won’t save enough money to cover basic needs and uninsured health care costs in retirement, according to the Employee Benefit Research Institute. Experts say reducing Social Security benefits could set them back even more.

That same generation has been slammed by the soaring cost of college, whether for their kids or themselves. As SmartMoney.com reported this spring, tuition and fees have almost tripled in the last twenty years, growing far faster than wages.

Of the more than $1 trillion in outstanding student-loan debt, federal student loans account for about 85%, according to the Consumer Financial Protection Bureau. (Private student loans account for the rest; private lenders can garnish a borrower’s wages, but can’t touch Social Security.)

Unlike other consumer debts, student loans typically can’t be wiped out in bankruptcy. And changes in the law over the last couple of decades have given Uncle Sam more power to pursue defaulters, said William Brewer, president of the National Association of Consumer Bankruptcy Attorneys.

The Debt Collection Improvement Act of 1996 empowered the federal government to offset Social Security payments of defaulted student-loan borrowers. An earlier law, the Higher Education Technical Amendments Act, essentially removed any time limits on the government’s ability to collect from the defaulters. The Supreme Court upheld both provisions in a 2005 ruling.

The government’s withholding power also extends to Social Security disability benefits. Tammy Brown of Redding, Calif. said that the government has been taking $179 out of her Social Security disability check each month for the past five years. Brown, 52, became disabled in 1986 after being involved in a car accident. Unable to work, she fell behind on her student loan payments. She said the Social Security check is now too small to cover her food and medical bills, so she quit taking prescription pain pills. “It’s kind of hard to live on this amount of money,” she said.

Attorneys who specialize in defending debtors have seen a sea change as the government has stepped up its enforcement. For most of the past four years, Joshua Cohen, an attorney in Rocky Hill, Conn., has represented clients in their dealings with banks and collection agencies. It was only in the past year, Cohen said, that he started getting a growing number of calls from retirees. Now, Cohen said, “I’m getting calls from all over the country from people desperate for help.”

The Department of Education, which provides federal student loans to borrowers, say it tries to work out payment plans with people who fall behind on their loans. Justin Hamilton, a spokesman for the department, said that accounts aren’t sent off to collections until almost two years of non-payment; if collection doesn’t yield results, the loan balance goes to the Treasury Department, which can reduce Social Security checks. “It’s when people aren’t making any attempt whatsoever [to pay] that they start heading down that road,” Hamilton said.

For its part, the Treasury Department said it reaches out to borrowers twice to set up a payment plan or otherwise resolve their debt before offsetting money from their Social Security check. And the Treasury won’t withhold money from monthly checks that total $750 or less, said Ronda Kent, deputy assistant commissioner for debt management services at the Treasury Department’s Financial Management Service.

Advocates of the borrowers say many of them have extenuating circumstances. In many cases, they say, family agreements unravel. For instance, adult children tell their parents they’ll repay the loan end up dropping the ball without informing them.

There can also be breakdowns in bureaucratic paper shuffling, said Mark Kantrowitz, publisher of FinAid.org, a student-loan tracker. In some cases the Department of Education loses track of the borrower, sending debt notices to the wrong addresses; sometimes, said Kantrowitz, the government can’t track down the borrowers until they begin receiving Social Security.

Student-loan experts say that changes in payment plans are partly to blame for why an aging population is still dealing with college loans. The repayment period on federal student loans can be extended to 30 years, Kantrowitz notes, if borrowers owe $60,000 or more. Another eight years can be added on for borrowers facing unemployment or other economic hardship; during those years, payments aren’t required but interest accrues.

Compared to present-day retirees, younger generations are in deeper debt, which means stories of Social Security garnishment could become more commonplace when they enter retirement. Borrowers in their 20s and 30s owe roughly $600 billion, according to the New York Fed. They’re also leaving college with more debt than their predecessors: 66% graduated this spring with debt, and their student loans averaging $28,720, up from $9,320 in 1993, according to FinAid.org.

“It’s entirely possible that the way student loan debt is growing, this could get worse,” said Rich Williams, higher education advocate at the U.S. Public Interest Research Group, a nonprofit consumer group.