President Donald Trump's corporate tax cut and his business deregulation efforts are not just an economic "sugar high," former Federal Reserve governor Kevin Warsh argued on CNBC Thursday.

Most economists "never thought the economy could grow this fast," said Warsh, who had been on the president's short-list for Fed chairman before Jerome Powell was chosen.

The government's final reading of second-quarter gross domestic product showed an advance of 4.2 percent on an annualized basis. The Atlanta Fed's GDPNow real-time indicator put the third quarter gain at 4.1 percent, as of Monday.

Taken all together with the first quarter's 2.2 percent increase, the economy stands to grow at an average of 3.5 percent for the first nine months of 2018.

That would be above Trump's goal of economic growth 3 percent-plus. The president has used GDP numbers and the Dow Jones Industrial Average, which closed at another record high Wednesday, as yardsticks of success for his policies.

"Today the economy is the strongest it's been since it's been in the U.S. since 2004" when growth was 3.8 percent, Warsh said on "Squawk Box." "At the end of this year, we may be saying the economy is the strongest since 1999," when growth hit 4.7 percent, he added.

The trajectory of the economy suggests "we're pretty early in a business recovery, long after we had a consumer recovery and a housing recovery," said Warsh, a visiting fellow at the Hoover Institution think tank.

Warsh, who has long-advocated an end to the easy monetary policy needed after the 2008 financial crisis, refused to say whether he would be raising interest rates on the same course if he were Fed chair. "Powell is a good guy. I've known him a long time. And I'm rooting for him. I think he's actually taking the office at much more difficult time than many people think."

Trump has not been shy about letting Powell and the Fed know that he wants the cost of borrowing money to remain low. But the Fed under Powell, who became chairman in February, has hiked rates three times this year — in March, June, and September — with another increase widely expected in December.

Powell added to that likelihood on Wednesday, saying the central bank has a ways to go yet before it gets interest rates to where they are neither restrictive nor loose. "The really extremely accommodative low interest rates that we needed when the economy was quite weak, we don't need those anymore," he said.

Those comments stoked the yield on the 10-year Treasury on Wednesday to highs not seen since 2011. Yields were continuing their march higher Thursday. When yields rise, bonds prices drop due to their inverse relationship.

"I give Jay Powell a lot of credit to introduce some humility into this forecasting process," said Warsh, who before his time at the Fed from 2006 to 2011 worked as an economic advisor in the George W. Bush administration and an M&A vice president at Morgan Stanley.

Powell also said on Wednesday neither he nor other Fed officials are letting the president or politics bother them. "My focus is essentially on controlling the uncontrollable."