Washington (CNN) A wide array of House and Senate Republicans are not yet ready to endorse President Donald Trump's bid for a second term, a reflection of the deep uncertainty on Capitol Hill over his political standing amid growing problems at home and abroad.

In interviews with a cross-section of more than two dozen GOP lawmakers, ranging from rank-and-file members, conservatives and party leaders, many refused to say they'd back Trump's re-election bid -- a surprise declaration given that members of Congress are typically quick to endorse sitting presidents of their own party without hesitation. Hardly any would offer a categorical endorsement of the President.

"I don't know what the world is going to look like," said Senate Majority Whip John Cornyn, a Texas Republican, when asked if he'd endorse Trump for re-election. "But let's say it's not something I've given any thought to."

Asked several days later if he had given thought to it, Cornyn demurred.

"I haven't even thought about that election," said Cornyn, No. 2 in the Senate GOP conference. "I'm worried about the midterm election."

He's not alone. Many lawmakers sought to avoid the topic altogether.

"Look, I'm focused on opioids," said Sen. Lamar Alexander, the veteran Republican from Tennessee, referring to efforts in Congress to deal with the drug epidemic. "And I was just reelected myself three years ago. So, I'm focused on that."

And others said they were still uncertain the President would ultimately stand for re-election -- even though the White House and Trump himself have repeatedly said he would do so, as he's hired a campaign manager, has been raising money and holding campaign rallies in anticipation of 2020. Unlike past presidents who have waited to announce their reelection bids, Trump made clear immediately after taking office that he's running again.

Still, many Republicans aren't certain he'll do it.

"That's a little loaded," said Rep. Bill Huizenga of Michigan when asked if he'd support Trump for re-election. "One: we need to make sure that he's actually moving forward and wants to go after this -- so when he makes a declaration, then I think that would be a time to determine whether there are others (who) run or not."

White House officials did not provide a comment for this story. Trump, however, may have little concern about whether his party in Washington is fully behind him. Few lawmakers backed his primary bid in 2016, and many abandoned him in the general election after the now-infamous "Access Hollywood" tape emerged showing him talking crudely about groping women.

Come 2020, things may be no different.

The conservative South Carolina Rep. Mark Sanford, when asked about backing Trump's re-election bid, said: "I'm worried about my own race right now."

And the moderate Adam Kinzinger of Illinois had a similar refrain.

"That's 2020 -- pretty far away," he said when asked if he'd back Trump for re-election.

GOP skeptical Trump would lose to primary challenge

Moreover, Trump could face a primary challenger in 2020, though many Republicans on the Hill don't think it's possible that any viable challenger will emerge. Arizona Sen Jeff Flake, a frequent Trump critic who is retiring at year's end, won't rule out a potential primary challenge against the President. And he contended that Trump's staunch support within the party could weaken by the 2020 primary season.

"I wouldn't gauge what support there is a year-and-a-half from now from what support there is now," Flake told CNN. "Certainly, now, this is Donald Trump's party. The base is with him in a big way. Believe me, we all know. But that's not to say that will hold."

It's highly unusual for sitting presidents to face primaries --- much less serious contenders. In 1992, George H.W. Bush fended off a challenge from Pat Buchanan, who ran a populist campaign against the sitting GOP president but didn't win a single primary. In 1980, President Jimmy Carter faced a fierce primary from then-Sen. Ted Kennedy, who eventually conceded after a bitter fight. And in 1976, Ronald Reagan tried to pull off an upset against President Gerald Ford but fell short at the GOP convention. All three sitting presidents lost their reelection bids.

This time around, few Republicans think that potential challengers -- like Flake or Ohio Gov. John Kasich -- would have a chance to win.

If Kasich runs, Ohio Sen. Rob Portman appears likely to side with Trump, whom he said is "very popular in Ohio among Republicans."

Portman said he plans to endorse "our standard-bearer, and I assume that will be Mr. Trump."

But asked if there's a possibility he wouldn't back Trump, Portman added: "I mean I would assume that he'll run for re-election but I don't know. Some would say that he's going to get his work done and move back to the private sector, so who knows, who knows? It's just speculative at this point."

Awaiting the midterms

What puts Republicans on Capitol Hill in an awkward spot is the fact that Trump's approval numbers remain rock-solid among core GOP voters, standing at 85% among Republicans in a recent Quinnipiac poll. But if they side too closely with Trump, they risk alienating the broader electorate, where his poll numbers have been historically low.

It's his shaky standing that could prompt a Democratic wave in this fall's elections, with the GOP at serious risk of losing the House and potentially even the Senate. If that were to happen, some believe, Republicans in the Washington will begin to search for a new GOP candidate come 2020.

"Wait until the midterms," said the outgoing GOP Rep. Charlie Dent, a moderate from Pennsylvania. "If we get wiped out, the question is going to be: 'Should we do that again?'" referring to backing Trump for re-election.

And even some loyal Trump supporters are cautious about 2020, waiting for the midterms to play out first.

"I've supported the President in the past and support him now but three years from now?" said Louisiana Sen. John Kennedy, a freshman Republican. "I think the midterms are a long ways away in terms of politics; I don't get involved that far ahead."

Sen. John Thune, the No. 3 Republican in the Senate, asked if he would support Trump in 2020, said: "Well that's a long ways off. I want to get through 2018 first."

But Thune, who called on Trump to drop out in 2016 after the "Access Hollywood" tape emerged , added a note of optimism for Trump, saying that he expects "most of us would probably be behind him" if "he continues to get things done for the American people."

Others wouldn't go that far, including Rep. Mario Diaz-Balart, a Republican from the pivotal swing state of Florida, who said "it's premature" to say if he'll endorse Trump's re-election bid.

"I'm focused on working and doing what I do and so to talk about what might happen in that time I think is premature," Diaz-Balart said. "We have one President, he's President until the next election, and I will continue to work with him like I work with everybody else to get things done."

Yet despite Trump's public refrain that the party is more united than ever, he has yet to win over some Republicans who didn't back him in 2016. And some have no plans to back him now -- at least not yet.

"I did not endorse the President for the Republican nomination in 2016," said the moderate GOP Sen. Susan Collins of Maine. "I supported first Jeb Bush and then John Kasich. So, again, I think it is far too early to make a judgment of that type."