Washington (CNN) Senate Democratic leader Chuck Schumer criticized President Donald Trump for abruptly ending a second round of infrastructure talks at the White House on Wednesday, saying that it's clear the President is not interested in tackling the issue and that Trump was "looking for every excuse" to get out of working on it.

"We are interested in doing infrastructure, it's clear the President isn't," Schumer said at a news conference after the meeting ended with Trump criticizing Democrats to reporters for their investigations into his administration.

Schumer added, "He is looking for every excuse, whether it was, let's do trade first, or whether it was he's not going to pay for any funding, or whether today, that there are investigations going on. Hello, there were investigations going on three weeks ago when we met and he still met with us. But now that he was forced to actually say how he'd pay for it, he had to run away."

House Speaker Nancy Pelosi and Schumer had gone to the White House for a second round of talks on infrastructure Wednesday -- a meeting that was already set to unfold against the backdrop of rising tensions between Democrats and the White House over the administration's resistance to congressional investigations . And that was before Pelosi told reporters that Trump engaged "in a cover-up" in his calls to end the Democratic-led probes.

The meeting had been expected to grapple with the thorny question of how to pay for a deal after Pelosi and Schumer announced at the end of last month's meeting with the President that Trump had agreed to a top-line price tag of $2 trillion for infrastructure spending, but that they would need to meet again to discuss how to pay for the plan.

But in the end, the meeting lasted only five minutes, a Democratic aide told CNN, a sign of how this possible point of bipartisan cooperation appears sidelined amid the broader investigations into the President.

In remarks to reporters not long after the meeting was set to start, the President turned to criticizing Democrats , explaining that he had said Democrats he could not work with them until their investigations were over.

Trump said that he "walked into the room" and told Pelosi and Schumer, "I want to do infrastructure. I want to do it more than you want to do it. I'd be really good at that, that's what I do, but you know what, you can't do it under these circumstances so get these phony investigators over with."

Schumer said that "to watch what happened in the White House would make your jaw drop."

At the same news conference, Pelosi said she and other Democrats went into the meeting with a "spirit of bipartisanship," but that the President "just took a pass" on continuing their discussions on infrastructure.

"For some reason, maybe it was lack of confidence on his part that he really couldn't come match the greatness of the challenge that we have, didn't, wasn't really respectful of the...Congress and the White House working together. He just ... took a pass. And it just makes me wonder why, why he did that."

Further complicating any effort to reach a deal, the President sent a letter to Pelosi and Schumer the evening before they were set to meet for a second round of talks asking them to take up the US-Mexico-Canada trade agreement before tackling infrastructure.

Congressional Democrats have expressed concerns over the trade deal and have called for changes to the agreement. As a result, the President's last-minute request on its own threatened to derail the potential to strike a deal on infrastructure.

In a joint statement later in the evening on Tuesday, Pelosi and Schumer made no mention of the trade agreement and instead focused on how to pay for an infrastructure deal, emphasizing that they expect Trump to come to the table with a proposal.

"On Wednesday, we look forward to hearing the President's plan for how to pay for this package," the top congressional Democratic leaders said.

Congressional investigations complicate talks

Infrastructure could be a rare area of bipartisan cooperation given that both Democrats and the President have long talked about investing in the nation's crumbling infrastructure as a top priority. But the sticking point in any deal will be how to pay for it and after the abrupt end to Wednesday's meeting and the breakdown in talks it may be that any hope for compromise is over.

The fact that hostilities between Democrats and the White House continue to escalate over congressional oversight efforts could continue to make it more difficult to reach a compromise. Some House Democrats are calling for an impeachment inquiry as the administration blocks oversight efforts. Pelosi has argued in recent weeks that Trump is effectively building a case on his own for impeachment, but has also advocated for a cautious approach and worked to tamp down on impeachment talk on the Hill. The President and his allies, meanwhile, argue that Democrats are targeting him unfairly and out of purely partisan political motives.

Wednesday's meeting also came as congressional leaders and the White House are attempting to negotiate a broader fiscal deal involving the debt limit and looming budget caps -- a set of talks that might need to be settled before it is possible for Democrats and the President to determine how to pay for any potential infrastructure deal.

When Pelosi and Schumer returned to the White House to talk infrastructure, they were again joined by members of House and Senate Democratic leadership and congressional Democrats whose committees would be expected to play a role if a deal comes together.

According to aides, the congressional Democrats who were planning to attend in addition to Pelosi and Schumer are: House Democratic Majority Leader Steny Hoyer; House Democratic Majority Whip James Clyburn, who told CNN that he will be attending; Assistant Speaker Ben Ray Lujan; House Ways and Means Committee Chairman Richard Neal; House Transportation and Infrastructure Committee Chair Peter DeFazio; Senate Democratic Whip Dick Durbin; Assistant Senate Democratic Leader Patty Murray; Senate Democratic Chairwoman of Policy and Communications Committee Debbie Stabenow; Senate Finance Committee ranking member Ron Wyden and Senate Environment Public Works Committee ranking member Tom Carper.

Would Mulvaney have made a difference?

There was expected to be one key difference in Wednesday's meeting compared to their last face-to-face with Trump three weeks ago: Mick Mulvaney.

The acting chief of staff — who is known on Capitol Hill for his hawkish fiscal policies — was across the country the last time Trump sat down with the Democratic leaders and agreed on a $2 trillion infrastructure plan. That's a price tag that Mulvaney was less than enthusiastic about, sources said, and he has told Trump it's a number that will be difficult to finance.

Mulvaney has advocated internally for public-private partnerships to leverage funding from corporations, a source familiar with the infrastructure discussions told CNN. The infrastructure plan Trump unveiled in early 2018 would only have used $200 billion in direct federal spending and relied mainly on state and local governments to foot the rest of the $1.5 trillion bill.

Democrats want to increase significantly the proportion of the plan funded by the federal government, and White House aides have discussed a gas tax as a possible way to raise revenue for infrastructure.

The meeting comes as Trump's point person on Capitol Hill, Shahira Knight, leaves the White House ahead of an expected slowdown in the White House's legislative agenda.

This story has been updated to include additional developments Wednesday.