It was mid-November, and Donald Trump was once again enveloped in political scandals and legal woes. He and his attorneys were finalizing written answers to Special Counsel Robert Mueller; the administration was trying to defend a lethal-force authorization at a militarized southern border; and the Republican Party was licking its wounds from the 2018 midterm election drubbing.

In the midst of all that drama, however, the president’s mind drifted elsewhere.

News had recently broke that Cliff Sims, a former Trump White House official, had penned a “tell-all” book about life in a backstabbing-filled West Wing. A 2016 campaign veteran, Sims had enjoyed wide access in part due to his personal friendship with Trump. The two men were so close that the president affectionately called him “my Cliff,” according to a source familiar with their relationship.

But when Trump caught wind of the upcoming book, he feared that yet another confidant had the knives out for him. Sims, according to a Politico report, had taken “copious notes” during his White House stint and sources familiar with the book told The Daily Beast that there are indeed explosive details that are likely to upset some key players in Trumpland. Though the book isn’t due out until January—and though Politico reported that it was not a Trump “takedown”—the president began asking associates in November if “we lost Cliff?” according to two sources with knowledge of the conversations. Sims declined to comment on this story.

The paranoia in that moment hinted at a larger, more bitter reality the president and his strategists feel is increasingly confronting them.

As the 2020 campaign rapidly approaches, Trump is surrounded by ex-officials and former friends eager, willing, or motivated to air dirty laundry. The list includes jilted advisers, former senior officials, ex-Cabinet members, once-loyal lawyers, and chiefs of staff. Some have already spoken out against the president, others have signaled their plans to do so, and even more are regarded as potential trouble down the road.

Publicly, Trumpworld has approached the possibility of the president’s former team and allies becoming his current roster of enemies with nonchalance.

“This is not new,” said Marc Lotter, Vice President Mike Pence’s former press secretary, and member of Trump’s 2020 advisory board. “It happened after the first campaign. You had people who were either in power or wanting power, former colleagues or business competitors, who would say he couldn’t do it, wouldn’t do it, it would be a disaster if he got elected. And he keeps pointing to his successes in office. He points to promises made and promises kept.”

But privately there is a healthy degree of consternation within the ranks. A source close to Trump says that he becomes noticeably “terrified” and irate whenever he hears of a negative book on him about to drop, whether authored by a former associate or an established journalist. The president often obsesses over the media coverage of these published works, and directs his lieutenants to wage war on the writers and the credibility of the work itself.

White House spokespeople did not comment on the record for this story. But with 2020 looming, it’s not just books that Team Trump has to worry about. Two of Trump’s political advisers say they view the growing list of allies-turned-foes as a political liability, and predict some will be “seeking revenge” against Trump come campaign season.

Already, the list is quite vast.

Former FBI Director James Comey has described Trump as amoral and a threat to the rule of law. Former top economic adviser Gary Cohn was widely regarded as a key source of highly unflattering information for Bob Woodward’s book Fear, which portrayed the president as a dunce on economic and world affairs. Former National Security Adviser H.R. McMaster has a book coming out in 2020, after leaving the administration on acrimonious terms. Former Veterans Affairs Secretary David Shulkin described Trump administration chaos and efforts by the White House to privatize VA hospitals. Omarosa Manigault Newman, a former Trump friend who also worked as a senior official in his administration, secretly recorded audio of the president, wrote a highly unflattering book, Unhinged, and has since waged a public campaign against Trump, labeling him a mentally unstable racist who is unfit for office. Trump’s one-time lawyer, Michael Cohen, is cooperating with federal officials looking into Trump’s business dealings and campaign conduct and has called Trump untrustworthy, unethical, and a race-baiting crank. Trump’s former secretary of state, Rex Tillerson, recently revealed that he had to stop the president from breaking the law. In his resignation letter, Defense Secretary James Mattis made it clear he had sharp disagreements with Trump’s worldview, including his policy in Syria. And over the weekend, it was conspicuously reported that departing Chief of Staff John Kelly did not think Trump was capable of handling the job.

For Democrats, the prospect of these same individuals amplifying these criticisms during the heat of the presidential campaign is viewed as the equivalent of electoral nirvana. The television ads (“I served with Donald Trump and let me tell you, he’s doesn’t deserve re-election”) practically write themselves.

“I think having people who have served in his administration reinforcing the aspects of Trump that most alienate and turn him off to voters accessible to us will likely push them to vote for a Democratic candidate, presuming we have a credible candidate there,” said Joel Benenson, who helped run Hillary Clinton’s presidential campaign.

Those who have worked with Trump say he understands the dangers of having so many disgruntled ex-employees—including former corporate titans and revered military figures—speaking out during the re-election. And yet, they also recognize that the president is largely unwilling, or incapable, of doing the main thing to prevent such a situation from happening: not humiliate members of his team before (or in some cases after) their banishment.

“ He is not good at firing people. ” — Sam Nunberg

Sam Nunberg, a former Trump political adviser who endured his own dose of Trump Twitter venom, said the president’s primary problem is that “he is not good at firing people.” That was fine, when it was just the Trump Organization, Nunberg noted, when Trump could treat an employee terribly, bungle the firing, and still compel him or her to sign a nondisclosure agreement. But now the stakes are far larger and the team less loyal.

“Our group was so close, we hated each other but it was a close group of people that to go out against him was a big deal,” said Nunberg, who was sued in a since-resolved lawsuit for violating his NDA. “But as the circle gets bigger and it is not just the Trump Org, things get out of his control.”

To this point, Trump and his team have taken steps to keep some of his former aides in mutual good graces. Many of those exiled from the White House or Trump’s orbit under less-than-ideal circumstances have been given cushy employment at allied groups or the re-election campaign. Katie Walsh, Trump’s former White House deputy chief of staff, left the administration in its early months in March 2017, only to get a soft-landing at the Trump-boosting outside group America First Policies. Sean Spicer, Trump’s first White House press secretary, similarly joined the super PAC America First Action as a senior adviser, following months of searching for a new, prominent professional perch.

And as revealed by one of Manigault Newman’s covertly made recordings, Lara Trump, a senior official on Trump 2020, offered the president’s former “Apprentice” $15,000 a month to work for the re-election campaign. “I saw this as an attempt to buy my silence, to censor me, and to pay me off,” Manigault-Newman said in an interview on MSNBC in August. She said she had recorded Lara Trump just days after she was ousted from her position as a senior Trump administration official. The Trump campaign has since sued Manigault Newman over her book.

But a large part of the strategy for dealing with the enemies-formerly-from-within is to simply bank on the capacity to convince the public that their criticisms don’t actually matter. The campaign is expected to emphasize a message that the president has achieved what he set out to do. And both they and even their detractors recognize that, for a good chunk of voters, it doesn’t matter what the critics say, even if those critics are informed by their front-row experience.

“In my view, it would only have impact if [these former Trump hands] backed the Democratic nominee. Republican primary voters could care less,” said Stuart Stevens, who helped run Mitt Romney’s 2012 presidential campaign. “A sort of Swift Boat campaign of those who served with him could be powerful to voters in play in a general election.”

He added, “But would they do it? I doubt it... But they should.”