President Donald Trump's outgoing National Security Adviser Lt. Gen. H.R. McMaster blasted Russia and President Vladimir Putin's "growing" confidence and "aggression" in what he said were his last public remarks as he transitions out of his role and makes room for his replacement, John Bolton.

"For too long some nations have looked the other way in the face of these threats. Russia brazenly and implausibly denies its actions," he said during a dinner hosted by the Atlantic Council at its headquarters in Washington D.C. on Wednesday evening. "And we have failed to impose sufficient costs."

The comments came just hours after Trump said, "Nobody's been tougher on Russia than I have."

While McMaster highlighted some of the administration's actions against Russia, critics point to Trump's repeated praise for Putin and his stated desire to work with him, not deter him.

"I think I could have a very good relationship with Russia and with President Putin, and if I did, that would be a great thing," Trump said Tuesday. "And there's also a possibility that that won't happen. Who knows?"

McMaster has been a strong critic of Russia.

Under his leadership, the National Security Council recommended Trump expel 60 alleged Russian intelligence officers over the poisoning of an ex-spy and his daughter in the United Kingdom — an action that Trump took in solidarity with over two dozen U.S. allies. Russia has denied involvement.

The Trump administration has also sanctioned Russia for its cyber attacks, including on the 2016 presidential election, and for its incursion into eastern Ukraine — to whom the U.S. is now selling lethal arms, a step the Obama administration never took.

But McMaster said the U.S. must do more.

"We are acting, but we must recognize the need for all of us to do more to respond to and deter Russian aggression," he said, laying out four steps in particular: integrating the government's various tools to respond, enhancing cyber defenses, greater allied investment, and a strong commitment to western values.

"We must strengthen our resolve, cooperate to share responsibility, catalyze positive change, and compete effectively in new arenas," he said, warning, "There’s nothing inevitable about the course of human events and history, and there is no arc of history — there is no so-called end of history — that will ensure our success."

But he had this message for Putin: "Russian aggression is strengthening our resolve and our confidence. We might all help Mr. Putin understand his grave error."

Tuesday's address was not the first time McMaster has had tough words for Russia — but most recently, when he spoke out, his own boss fired back at him.

"The evidence is now incontrovertible" that Russia interfered in the U.S. election, he said in February after special counsel Robert Mueller indicted 13 Russian officials and laid out their campaign in detail. Hours later, Trump tweeted, "General McMaster forgot to say that the results of the 2016 election were not impacted or changed by the Russians and that the only Collusion was between Russia and Crooked H, the DNC and the Dems. Remember the Dirty Dossier, Uranium, Speeches, Emails and the Podesta Company!"

John Bolton will take over on April 9 as National Security Advisor. McMaster, an active duty three-star general, said he was grateful for "the great privilege to serve the United States for 34 years."

McMaster joined Tillerson in criticizing Russia on his way out the door.

Hours before Trump took to Twitter to discuss his ouster, Tillerson was his most critical of Russia, telling reporters traveling back from Africa with him, "I've become extremely concerned about Russia... What we've seen is a pivot on their part to be more aggressive, and this is very, very concerning to me and others, that there seems to be a certain unleashing of activity that we don't fully understand what the objective behind that is."

Trump's top diplomat went on to release a statement that called Russia "an irresponsible force of instability in the world, acting with open disregard for the sovereignty of other states and the life of their citizens."