(CNN) Lawmakers and other politicians were at odds Sunday about whether they think Congress will be able to reach an immigration deal amid the outcry over President Donald Trump's remarks about immigrants from certain countries last week.

While some expressed optimism about negotiating a bipartisan deal, they also bemoaned the backlash over Trump's reported remarks, which have stalled immigration talks as Congress grapples with spending legislation to avoid a government shutdown at the end of this week.

Democratic Sen. Joe Manchin of West Virginia said Sunday that he was "encouraged" by the bipartisan effort, but added, "We've got to move on" from the controversy spawned by Trump's reported Oval Office comments in a meeting with lawmakers Thursday, in which the President said certain African immigrants come from "shithole countries."

Congress and the White House shouldn't "let it stop the whole procedure," Manchin said in an interview on CBS's "Face the Nation ."

"I mean, first of all, we have a bipartisan recommendation, a bipartisan legislation, that's been worked on, worked out, after they sat down last week with the President," he said. "And it seems to do everything that was asked of them. ... I would be very, very encouraged that, hopefully, all of my colleagues are going to look at this in a very positive way. And if the President said what he said, as far as you give me something in a bipartisan way, something you work out, and I'll sign it, we hope to get that done and move on."

Read More