Rep. Trey Gowdy, R-South Carolina, one of the four authors of the GOP memo released Friday, told CBS News' "Face the Nation" that he believes a surveillance warrant for former Trump campaign adviser Carter Page would not have ever been authorized without the existence of the controversial "Steele dossier."

Gowdy's comments come on the heels of the Friday release of a memo drafted by Gowdy, House Intelligence Committee chairman Rep. Devin Nunes, R-Calif., and two investigators on the committee. The memo, declassified by President Trump and released by Republicans on the House Intelligence Committee Friday, outlines how Foreign Intelligence Surveillance Act (FISA) warrants were obtained for former Trump campaign official Carter Page, and what relationship the Trump dossier compiled by former British intelligence officer Christopher Steele had with those FISA warrant applications.

Gowdy said the public will "never know" if the surveillance against Page was justified, saying the warrants relied on three pieces of information -- the dossier, a reference to a Yahoo News article and other information available to the FISA court judges who approved warrants.

But, Gowdy, the only Republican on the House Intelligence Committee who has seen the actual FISA warrant applications -- Nunes has not -- emphasized that the memo does not discredit special counsel Robert Mueller's investigation into Russian election meddling and any ties to Trump associates.

"Not to me, it doesn't, and I was pretty integrally involved in the drafting of it," Gowdy said, asked if the memo has an impact on the Russia probe. "There is a Russia investigation without a dossier."

The dossier has "nothing to do" with the June 2016 Trump Tower meeting involving Donald Trump Jr., Trump son-in-law and senior adviser Jared Kushner, and a Russian lawyer, Gowdy noted. Gowdy also said the dossier did not form the basis for the case against George Papadopoulos, the former Trump campaign aid who has pleaded guilty to lying to the FBI about foreign contacts. The dossier has nothing to do with obstruction of justice, Gowdy noted.

"So there's going to be a Russia probe, even without a dossier," he said.

Meanwhile, Gowdy said "it would" concern him if President Trump makes any move to fire Deputy Attorney General Rod Rosenstein.

The South Carolina Republican and chairman of the House Oversight and Government Reform Committee said he has confidence in the deputy attorney general, of whom Mr. Trump has been publicly critical, and he would be concerned if the commander-in-chief attempts to fire him.

"It would. Again, I'm not in the Senate. I don't have advice and consent. And the president has not sought my counsel on this," Gowdy told CBS News' White House and senior foreign affairs correspondent Margaret Brennan.

Pressed as to whether he thinks Rosenstein should be fired, based on the information he has seen in compiling the memo, Gowdy said no.

"I don't," Gowdy said. "I think it is fair to ask the deputy attorney general, what did you know at the time you signed one of the applications? I think it is fair to ask, what FISA reforms are you going to implement to make sure we don't have this fact pattern come up again? I don't judge people based on a single decision that they make throughout the course of an otherwise really stellar career."

Gowdy also expressed confidence in FBI Director Christopher Wray, whom President Trump selected to replace fired FBI Director James Comey last May.

"I've had my differences with Rod Rosenstein and I still think that he is fully capable of helping run a Justice Department that we can all have confidence in," Gowdy said. "I'm actually really impressed with Chris Wray and I say that even though we are on totally opposite sides of this issue and probably will always be. He doesn't think the memo should have been publicly disseminated. I have real questions about the process that the bureau went through in 2016, but I also think he's the person to lead the bureau. I think he's doing a good job."

Gowdy went on to express his support for Mueller, whose investigation he said is independent.

"Well I'm actually in a really small group, I think, of Republicans that think that this FISA process is suspect and wrong and should not have taken place. But you still have a Russia investigation even without it," Gowdy said. "So I don't know how many other Republicans feel that way. I am on record as saying I support Bob Mueller 100 percent. I think you would have a Russia -- look, Russia tried to interfere with our election in 2016 with or without a dossier. So you need an investigation into Russia. You need an investigation into the Trump Tower and the Cambridge Analytica email, separate and apart from the dossier. So those are not connected issues to me. They may be for Republicans, but they're not for me. I say investigate everything Russia did but admit that this was a really sloppy process that you engaged in to surveil a U.S. citizen."

Gowdy's claims would seem to conflict with Mr. Trump, who took to Twitter Saturday morning to claim the GOP memo "totally vindicates 'Trump.'" The president frequently describes the Russia probe as a "witch hunt," and did so on Saturday morning.

Gowdy also opened up about his upcoming departure from Congress. His announcement that he would not run for reelection in 2018 sent shockwaves throughout Washington.

"You know, I'm just -- I enjoy the justice system more. I enjoy being fair," Gowdy said, explaining the reason for his departure. "I enjoy the pursuit of fairness as a virtue and I'm just more comfortable in that system. My wife hates it when I say this but I was a pretty good prosecutor, I think. But I've been a pretty lousy politician. So, I've done it for seven years. I'm really grateful for the opportunity to do it but it's time for me to -- whatever time I've got left, I want to spend it in the justice system because that's where my heart is."

You can watch Gowdy's interview with Margaret Brennan on Sunday on "Face the Nation" (@FaceTheNation). Click here to check your local listings.