(CNN) Secretary of State Rex Tillerson on Wednesday tried to knock down a report that he had called President Donald Trump a "moron" and considered leaving his post over disagreements with the commander in chief over the summer. But, in the process he may have made things worse for himself.

"I want to dispel with this notion that I have ever considered leaving," Tillerson said in an appearance at the State Department added to his calendar following the bombshell report via NBC News that the secretary of state had called Trump a derogatory name over the summer, had considered resigning and was counseled to remain in the job by Vice President Mike Pence.

While Tillerson's denial that he ever spoke with Pence about leaving his post is important, it's what Tillerson didn't deny that may be even more telling.

Asked whether he had indeed referred to Trump as a "moron," Tillerson said: "I'm not going to deal with petty stuff like that."

Ahem. Cough. Ahem.

Here's the thing: If Tillerson refused to directly deny calling Trump a "moron," you can be almost certain he did in fact call the president exactly that. Think about it: If there is a report that you said your boss was "a big dummy" and you hadn't actually said that, wouldn't you take any chance you could to correct the record? Your job might depend on it!

Tillerson's non-denial comes even as the CNN White House team is reporting that "discord between President Donald Trump and Secretary of State Rex Tillerson is at an all-time high." And, you can be sure that Tillerson's refusal to publicly acknowledge that he didn't call Trump a moron will only make things worse.

"For weeks, conversations with Tillerson friends outside of Washington have left the impression that he, despite his frustrations, was determined to stay on the job at least through the end of the year. That would allow time to continue efforts to reorganize the State Department and would mean he could claim to have put in a year as America's top diplomat.

But two sources who spoke to CNN on condition of anonymity over the weekend said they would not be surprised if there was a "Rexit" from Foggy Bottom sooner that that."

At the time, State Department spokeswoman Heather Nauert denied any rumor of a potential Rexit, saying : "[Tillerson has] been very clear he intends to stay here at the State Department. We have a lot of work that is left to be done ahead of us. He recognizes that. He's deeply engaged in that work."

Speculation about Tillerson's future died down in the wake of his return to Washington. But, over the weekend, talk of tension reemerged when Trump seemed to undercut his Secretary of State on North Korea.

"I told Rex Tillerson, our wonderful Secretary of State, that he is wasting his time trying to negotiate with Little Rocket Man," Trump tweeted Sunday morning

I told Rex Tillerson, our wonderful Secretary of State, that he is wasting his time trying to negotiate with Little Rocket Man... — Donald J. Trump (@realDonaldTrump) October 1, 2017

Remember that Tillerson was at one point seen as the crown jewel of Trump's Cabinet. The former head of Exxon was exactly the sort of person that Trump had promised his administration could and would recruit into public service.

"He's a world-class player," Trump said of Tillerson before formally nominating him as a Secretary of State. "He's in charge of an oil company that's pretty much double the size of its next nearest competitor."

JUST WATCHED Tillerson sworn in as secretary of state Replay More Videos ... MUST WATCH Tillerson sworn in as secretary of state 02:39

"You bring the unique skills and deep deep insights...into foreign diplomacy our nation needs to foster stability and security," Trump said when Tillerson was formally sworn in as the nation's top diplomat.

Tensions appeared quickly between the two, however, as Tillerson's careful diplomatic approach clashed with Trump's free-wheeling Twitter diplomacy.

JUST WATCHED Tillerson: President Trump speaks for himself Replay More Videos ... MUST WATCH Tillerson: President Trump speaks for himself 00:51

The Charlottesville, Virginia, protests -- and Trump's both-sides-do-it response to the white supremacist violence -- complicated the relationship further. Tillerson seemed to rebuke Trump when he responded to Fox News' Chris Wallace that "the President speaks for himself" regarding the President's comments. Sources inside the White House tried to play down Tillerson's comments -- insisting that what the secretary of state meant was only that Trump is his best spokesman.

In the wake of Tillerson's seeming admission that he called his boss a nasty name, the question turns from whether the secretary of state will quit to whether Trump will part ways with him.

That seems somewhat unlikely given the already-heavy turnover in the top ranks of Trump's administration. Last Friday, Health and Human Services Secretary Tom Price was forced to resign. And, Trump's chief of staff, chief strategist, national security adviser, spokesman and not one but two communications directors have departed the administration in just its first nine months.

"There's much to be done and we're just getting started," Tillerson said on Wednesday.

It's not clear at the moment whether that's a promise or a hope.