For the vast majority of his career, Joe Biden was a proud deficit hawk—a man who fretted for decades about federal spending and, on occasion, bragged about his own personal attempts in the 1980s “to freeze all government spending, including Social Security,” in the name of fiscal prudence.

Now, in the midst of a Democratic primary that has seen the party move left on nearly every major policy issue, Biden’s rivals are attacking him for that record, and rightly so: Biden’s past willingness to put popular entitlements on the table could be a serious political liability in a race against Donald Trump, who has repeatedly promised not to cut Social Security benefits for the elderly. Just as importantly, the former vice president’s history on the budget raises serious questions about whether he would cave to sudden demands for austerity if he were ever confronted by a Republican Congress, much as he was seemingly ready to do during the Obama administration.

So far, the criticisms about Biden’s record on federal spending and Social Security have primarily come from Bernie Sanders and his supporters. Unfortunately, not all of their accusations have been fair. On January 7, for instance, the Vermont senator sent out a newsletter that claimed Biden had “lauded Paul Ryan for proposing cuts to Social Security and Medicare” during a 2018 speech at the Brookings Institution. As Politifact reported, this was untrue; in his own slightly muddled way, Biden appeared to be sarcastically mocking Ryan for proposing tax cuts so sweeping that they could only be paid for with large entitlement cuts. At the New York Times, Paul Krugman seized on the distortion to accuse Sanders of “smearing” Biden and adopting “Trumpian” tactics.

And yet, as Krugman admits, there’s no getting around the fact that Biden spent the vast majority of his time in Washington beating the drum for budget cuts. Was he just a product of his times? Sort of. Biden joined the Senate at a moment when New Deal liberalism was giving way to fiscal conservatism and skepticism about big government. After Ronald Reagan’s victory, plenty of Democrats tacked to the center on issues such as crime and spending, and over the coming decades, debt-panic and a grave willingness to “tackle entitlements” slowly became the mark of supposedly serious Washington politicians.

But Biden was especially energetic about it all. He began sponsoring bills designed to curb the budget in the late 1970s. In 1984, as deficits mounted thanks to Reagan’s tax cuts, Biden warned of looming economic catastrophe and co-sponsored legislation that would freeze all federal spending for a year, including Social Security.

The idea flopped, but Biden still pushed similar spending pauses throughout the decade. He also backed a line-item veto, another favored fad among deficit hawks. In 1995, he chose to side with the new Republican majority in Congress and backed a balanced budget amendment to the constitution, warning that the national debt was threatening to consume the rest of the federal budget. “Unless this thing gets focused,” he told the New York Times, “by the time we face the music, everything I care about is going to be gone.” The amendment ultimately lost by a single vote in the Senate, because Democrats said that they were worried it would lead Congress to raid the Social Security Trust Fund in order to limit the deficit. Biden, along with other members of his party, had fought to tweak the amendment so that it would exclude the retirement program. But after that effort failed, he put his concerns aside and voted yay with the GOP anyway. When Republicans took a second doomed shot at the balanced budget amendment in 1997, he supported it again.

Biden was not always on the side of the deficit slashers and debt alarmists. He helped kill a balanced budget amendment in 1986, for instance, telling reporters that Congress had just passed a major piece of legislation meant to limit the deficit, the Gramm-Rudman-Hollings Act, and that lawmakers should wait to see how it worked (he also co-sponsored legislation to protect Social Security from the effect of that law). In 1995, the same year he got on board with the GOP’s constitutional amendment, he actually voted against a bill that would have balanced the budget within seven years, which President Bill Clinton later vetoed on the grounds that it was too draconian. Around that time, Biden gave a typically florid speech in which he insisted that he was as worried about the deficit as anybody, but suggested that the GOP needed to compromise on its time frame:

I am one of those Democrats who voted for the constitutional amendment to balance the budget. I have introduced, on four occasions—four occasions!—entire plans to balance the budget, knowing I’m not president, and I’m not the leader, for illustrative purposes. I tried with Senator Grassley back in the ‘80s to freeze all government spending, including Social Security. Including everything. And the truth is, the last election did one thing. I don’t know if it made you guys the majority party for long, I don’t know. We’ll find out. But I know one thing it did. It made sure that there was nobody left in the left of my party who said, in fact, we don’t care about moving the budget toward balance.

In a lot of ways, Biden’s remarks here perfectly symbolize the politics of that era, during which the word “liberal” was treated as a dirty word and lots of Democrats adopted center-right politics on issues like crime, welfare-reform, and the deficit in order to hold moderate voters and fend off the GOP’s worst plans. There were also good reasons for Biden to vote as a centrist; we think of Delaware as a solidly blue state today, but until 2001, the state’s other senator was a Republican. At the same time, there’s little evidence to suggest that Biden was anything but sincere in his positions. One notable thing about Biden’s Senate career is that he showed relatively little interest in economic issues, focusing more on foreign affairs and criminal justice, which fit with his committee assignments. Yes, he was a driving force behind the bankruptcy reform bill that has come to be seen as a sop to credit card companies in his home state of Delaware. But other than that, Biden’s main economic policy cause really seemed to be the debt.

Now, decades later, it’s easy to imagine how Biden’s old words and causes could come back to haunt him against Trump. One of the president’s central campaign promises, the thing that made him look like a moderate in the eyes of many voters, was that he would not touch Medicare or Social Security benefits for the elderly (though disability is another story). Cut to a future Trump ad featuring ready-made C-SPAN clips of Biden waving his arms and boasting about trying to freeze the program, as Sanders staffer David Sirota was happy to highlight on Twitter a few weeks ago.

JOE BIDEN: “When I argued that we should freeze federal spending, I meant Social Security as well. I meant Medicare & Medicaid. I meant veterans’ benefits....And I not only tried it once, I tried it twice, I tried it a 3rd time & I tried it a 4th time.”https://t.co/TpoSdmjGhA pic.twitter.com/MIAx4nO9Qa — David Sirota (@davidsirota) January 2, 2020

It doesn’t help that Biden continued to talk about cuts to entitlements deep into his Senate career. Intercept reporter Ryan Grim has noted that when Biden campaigned for president in 2007, he told Meet the Press host Tim Russert that he would consider benefit and eligibility changes to Social Security and Medicare to deal with the deficit, and boasted about helping to negotiate the deal that raised the retirement age to 67. His current campaign platform actually calls for slightly expanding Social Security benefits for seniors. And yet, during that recent Brookings speech, he still somehow managed to suggest—albeit in an aside—that both Social Security and Medicare would need “adjustments“ to stay around, which most pundits on the left have interpreted as a euphemism for “cuts.” (This isn’t the first time Biden’s comments on the stump have clashed with his official platform; he’s talked about bringing back Obamacare’s individual mandate, even though his health care plan says nothing about doing so.)

It’s exceedingly unlikely that a Democratic Congress would ever try to slash Social Security or Medicare. One important question, however, is what will happen if President Biden comes face-to-face with a Republican Congress—or even just a GOP-held House or Senate—that tries to force his hand on spending cuts. (Mitch McConnell has essentially said he will wait until a Democrat is in office to try to slash entitlements, because it’s too unpopular for the GOP to do on its own.) We know what transpired during the Obama administration: The White House tried to compromise. It agreed that the deficit was a clear and present danger—despite the fact that the economy was still depressed after the recession—and attempted to negotiate a grand bargain of spending cuts and tax increases. Biden himself led discussions with a bipartisan group on behalf of the administration, and appeared ready to give up trillions in budget reductions, though they wouldn’t have targeted the major entitlements. As late as 2014, Obama caved to Republican requests and included a proposal to slow down the growth of Social Security benefits in his annual budget (he later dropped the idea in the face of liberal pushback). Would a Biden administration travel down the same road, and try to compromise? Compromise, after all, is what Biden is running on.

Ideally, a Democratic president in 2021 would not only oppose budget cuts today, but would jettison most of Washington’s obsession with so-called fiscal discipline. The national debt is, truly, among the least of our problems right now. We may need to fully pay for ongoing expenses like a healthcare expansion. But there are plenty of urgent priorities with big, one-time price tags—infrastructure, a transition to green energy—that could be deficit financed without causing any significant economic issues. Certainly, the rock bottom interest rates on U.S. treasury debt suggest we have plenty of room to borrow.

It unfortunately might be too much to ask for that kind of leader. But at the very least, the next person in the Oval Office should be able to stand up to the debt scolds on the big issues, like entitlements or austerity. If they can’t, the sick and elderly will suffer, the economy will suffer, and their presidency will suffer.

One tragedy of the Obama administration was how, following the failure of the grand bargain, the GOP still steamrolled it into passing major spending cuts that helped turn the economic recovery into a crawl, and may well have contributed to a Trump win in the end. Perhaps Biden learned a lesson from that experience, and would resist repeating it. But our ex-VP has talked extensively about his desire to return to an era of bipartisan cooperation. And whether we’re talking about Social Security or another major program, the budget is one area in which those instincts are truly scary.