Jonathan Kirn / Getty Images

So here’s my prediction: America’s debt is going to be downgraded and the ramifications of that will be worse than many think.

And don’t count me as a debt worrier. I don’t actually think the U.S.’s debt is a problem, at least not yet. But to those people who do fret about the debt, which apparently includes the ratings agency Standard & Poor’s, the deal to raise the ceiling is not going to do enough to end the concern that the U.S. has borrowed too much. Moody’s said on Tuesday that while the debt deal will allow the U.S. to retain its AAA rating for now, the ratings agency put U.S. bonds on its list of investments that may get downgraded in the future. Even with the deal, the U.S. is likely to add as much as $3 trillion in debt in the next five years. Worse, the fact that politicians weren’t able to come to an agreement to raise taxes, something nearly all economists agree is essential to dramatically lowering our debt, is likely to indicate to the ratings agencies and investors that if we get in a real pinch we may not be able to come to an agreement that would actually pay off our debt. The most damning thing the debt ceiling standoff showed are the serious problems in Washington, and a functioning government is essential to a top credit rating.

(LIST: 10 Smarter Ways to Reach Your Retirement Goals)

So while I’m not sure it will happen tomorrow, S&P may wait until the dust settles to get some cover, but in the next few months I expect to see the U.S. downgraded from our current AAA status, which is the highest rating. Worse, the impact of the downgrade will be more costly than people think. Here’s why:

Some have predicted that a downgrade of U.S. debt may not have that much effect on bond yield of Treasuries. Ratings on their own don’t determine the interest rate that many countries have to pay to attract investors. Japan, for instance, has a lower rating than the U.S. and yet has a significantly lower interest rate, around 1.1% on its 10-year bond, versus a recent 2.6% for U.S. Treasury bonds. Australian bonds on the other hand have the same rating as the U.S., yet the country pays a much higher interest rate to investors of – nearly 5% – than does the U.S. What’s more, the rating agencies reputation has been badly damaged by the financial crisis and housing bust, when a large number of AAA rated mortgage bonds by S&P and others eventually ended up turning out to be nearly worthless. So some say investors will ignore the ratings change.

(MORE: Deal or No Deal, the U.S. may be Downgraded)

Perhaps. But the examples of Japan and Australia are outliers. And at least so far, the ratings agencies do still seem to matter. On average a higher rating does tend to lead to a lower bond yield. So expect to see U.S. bond yields rise when the downgrade happens. How much? According to a recent chart put together by the New York Times, average bond yield of a debt of a country with a AAA rating is about 3%. The average yield of the bonds of the countries in the next three categories, which is where the U.S. will land is 4.15%. How will a 1.15% higher interest rate hit your wallet? Here’s how:

Taxes

A higher interest rate will mean that the U.S. Federal Government will have to pay more to borrow. Not all of the country’s debt will have to be refinanced, but if it did a 1.15% higher interest rate would cost the country roughly $170 billion more a year in borrowing costs. The cost to the average American household, if it were passed along in higher taxes, would be about $1550 a year.

Buying a House

Mortgage rates tend to follow 10-year Treasury rates. So a higher yield on 10-year Treasuries that would come with a ratings downgrade is also likely to force you to pay more to buy a house. A 1.15% higher rate on a $200,000 loan would raise your interest payments by $36,573 over the course of the mortgage. It is likely over time that the spread (which is the difference) between Treasuries and mortgage rates would narrow because a lower rating for U.S. bonds would mean the difference in risk between mortgages (generally seems as more risky than Treasuries) would have narrowed.

Saving for Retirement

Not always, but higher Treasury yields are generally associated with higher rates of inflation. If that is the case you will need to be putting away more money for retirement because you dollars will be worth that much less 30 years from now. If you are 35 and make a salary of $75,000, you will have to save nearly 8,000 more a year, or about $650 a month, to have enough for retirement. And that is under the rosiest scenario. Higher interest rates are generally not good for the stock market. So if stocks rise slower than they have in the past, you may have to put more away to account for that to have the same at retirement.

And that is just three categories. Other lending rates may go up as well, though credit card rates are not as tied to U.S. bond rates as say mortgages. The point is it won’t just be Uncle Sam who is going to come out of this debt ceiling debate poorer. It will be all of us.

Stephen Gandel is a senior writer at TIME. Find him on Twitter at @stephengandel. You can also continue the discussion on TIME‘s Facebook page and on Twitter at @TIME.