(CNN) James Clapper, former director of National Intelligence, said Tuesday that for the first time he can remember, he agrees with Steve Bannon that US options in dealing with North Korea are "limited."

Clapper's comments about the former White House chief strategist follow President Donald Trump's warning to Pyongyang that "all options are on the table" after North Korea fired a missile over Japan early Tuesday.

Tuesday's missile launch was particularly provocative as it was North Korea's first ballistic missile to fly over Japan. Kim Jong Un's regime regularly fires missiles into the sea between its own territory and Japan.

"If the North Koreans intended the trajectory of that missile to go over Japan that is a big Rubicon they've crossed," Clapper told CNN's Chris Cuomo.

Clapper argued there needs to be a distinction between a peremptory military option, contrasted with a reaction to a kinetic damage by a North Korean missile.

"The problem as Mr. Bannon observed, we don't know what their red-line is," Clapper said. "Whether they would just react in kind or pause or an all out cataclysmic attack against the South and that's what makes it such a difficult problem."

However, Clapper said after traveling to North Korea in November, he came away with the conclusion that negotiations are the only viable courses of action.

"We may have to hold our nose and we may have to consider some concessions," Clapper said.