The recent Republican tax cuts and the bipartisan agreement to raise federal spending have done a number on already high budget deficits. But the outlook could be even worse than we thought.

The nonpartisan Congressional Budget Office now projects that annual US budget deficits will cross the trillion-dollar mark in 2020, two years sooner than what the agency was projecting just 10 months ago.

A trillion-dollar deficit is especially striking given that there is no recession or financial crisis driving the country to spend so much more than what it takes in.

But it's not nearly as eye-popping as $2.1 trillion.

Yet that's where budget deficits are headed as soon as 2028 if Congress decides to extend the tax cuts and higher spending levels past their respective expiration dates. Many of the tax cuts are set to expire after 2025, while higher spending levels expire after two years.

Related: The deficit report card is in. Trillion dollars is now the new normal

Two days after CBO's latest projections came out, House Speaker Paul Ryan announced his retirement after serving in Congress for two decades. During that time, Ryan was chairman of both the House tax and budget committees and was the most high-profile deficit hawk in the Republican Party. Yet the annual deficit steadily rose as a share of the economy during his time as Speaker.

With or without Ryan, however, it's not crazy to think Congress could extend the costly tax and spending policies that would lead to $2 trillion deficits. Lawmakers in recent years have been loathe to let tax cuts expire. And cutting spending is not their strong suit.

The 2001 and 2003 Bush tax cuts, for instance, were extended during the Obama administration and then largely made permanent when Congress faced the so-called fiscal cliff in 2012.

In 2011, Congress struck a much-criticized deal to force lawmakers to reduce deficits under threat of automatic cuts if spending caps were violated. Those caps, however, have been overridden a few times, most recently this winter, when lawmakers voted to raise limits on defense and nondefense spending.

And since 2013, instead of actively agreeing to increase the legal US borrowing limit to pay the country's obligations, Congress opted instead to just suspend the debt limit six different times.

If lawmakers take the route of least resistance, the CBO estimates that in ten years, the debt — which reflects the accumulation of deficits over the years — could reach 105% as a share of the economy, higher than at any time except right after World War II. And it would be much higher than the 96% of GDP that the agency estimates if the tax cuts and spending increases expire as scheduled.

For comparison's sake, this year the country's debt is expected to hit 78%.

Then again, there is a chance lawmakers might act to moderate deficits before they cross the $2 trillion mark.

Maya MacGuineas, who runs the Committee for a Responsible Federal Budget, can see Congress extending some but not all of the expiring tax cuts and taking other measures to curb deficits before 2028. In that case, "I would bet our debt will be around 101% of GDP by the end of 10 years," MacGuineas said.

Even then, however, budget deficits north of $2 trillion might not be far off, she noted.

"I think they will certainly cross it but it will be a few years longer than a decade."