On Tuesday night, after James Comey got fired, FBI agents tasked with thwarting Russian intelligence operations started drinking.

Two well-connected former FBI employees told The Daily Beast that counterintelligence agents working on the Russian counterintelligence program out of FBI headquarters in downtown Washington met for drinks in the hours after their boss’s firing and shared their concerns: that they would be reassigned elsewhere, and their work on the Russian-Trump associates investigation would come to a grinding halt.

“We do not have any comment,” an FBI spokeswoman said in an email to The Daily Beast Friday morning.

These are worries that have spread through the bureau in the days since Comey was fired: that the new administration will find ways to stymie investigations that could create political problems—especially on Russia. It’s a concern the president himself exacerbated in an interview with NBC News’ Lester Holt that aired Thursday evening.

“And in fact when I decided to just do it I said to myself, I said, ‘You know, this Russia thing with Trump and Russia is a made-up story, it’s an excuse by the Democrats for having lost an election that they should’ve won,’” the president said, discussing his reasoning for firing Comey.

Among current and former agents who worked on Russian counterintelligence, concern about political meddling is palpable.

“It’s complete bananas,” said one FBI source. “Management in counterintelligence are insanely concerned, worried about the overreaching obstruction and political influence from the White House.”

And a former high-ranking FBI official who worked on aspects of the case said there’s “no doubt the investigation can be damaged.”

“This particular case is within HQ with pieces in other field offices,” the source continued. “Hard to stop, but definitely subvert.”

The pace of the FBI’s Russian counterintelligence investigation dramatically picked in recent weeks when the probe into former National Security Adviser Michael Flynn expanded to include his company’s work for the Turkish government, and a round of subpoenas were issued by a Virginia grand jury for related business and financial records. This stems from reports out of Turkey that Flynn had at some point attempted to return money he was paid for work he didn’t end up doing. That gave investigators a money trail to follow. Flynn reportedly failed to disclose this income when he was employed by the White House.

Two sources suggested that aspects of the larger investigation are focused on whether foreign influence was or is currently being exerted at the White House. It is unclear if this is specifically related to Flynn, or other aspects or targets of the case.

“It is not just a historical investigation,” said one former intelligence official who worked aspects of the early stages of the investigation.

Andrew McCabe, the bureau’s interim director, told members of the Senate Intelligence Committee on Thursday morning that nothing would stop the investigation. But his confidence didn’t calm many nerves, in large part because there’s a broad consensus in Washington that his days at the bureau are numbered. In that same hearing, McCabe praised Comey and directly contradicted a White House spokesperson’s assertion that the FBI rank-and-file had turned on their former boss.

“Literally who cares, nothing he said matters. He’ll be gone,” said one congressional staffer.

McCabe isn’t the only top FBI official who could be in trouble.

Several administration sources said Associate Deputy Director David Bowdich could also be in the crosshairs. Sources said he has played an integral role in the investigation that led to the resignation of Flynn. That investigation is ongoing. The FBI website says Bowdich oversees the management of all FBI personnel and budgeting. So if he’s replaced, that could have a significant effect on the resources available to agents working on the Russia investigation.

The FBI declined to comment.

And it’s not just Bowdich. As FBI director, Robert Mueller changed the structure of the bureau’s leadership, adding outside non-agent, non-bureau personnel into the FBI at the rank of assistant director. These became mostly administrative positions, though some had joint oversight of counterintelligence operations. Comey continued this tradition. Sources said Trump could replace people in those positions with his loyalists, who could could slow the investigation.

The larger temporary task force investigating Russian influence investigation could soon be gone, according to a former FBI official.

They could “dismantle it, transfer the agents out, minimally staff it, have DOJ refuse to prosecute,” that official said.

“They could slow down the investigations to a crawl, prevent charges from moving forward to DOJ for prosecution, or any other number of ways the White House could subvert these investigations,” said a former FBI official who worked on Russian investigations.

“You have to remember, these agents have families they need to support,” said a former high ranking FBI official who worked closely with the Counterintelligence division. “The threat of being fired for doing their job is real here.”

And Comey’s firing could have already had an impact, according to Carrie Cordero, a former attorney in the Justice Department’s National Security Division.

“On a really major, highly sensitive, big big big-time case, it does matter the level of the director’s personal involvement,” she said. “It just does.”

And Comey’s support mattered.

“He gave the agents, the investigators, cover politically,” she said. “He said, ‘You go where the facts take you and I will handle the politics of it, I’ll go brief the Hill, I’ll hold off the White House.’ He’s the lineman in football, keeping everybody away from the guys that are trying to run or make the pass. He provided a cover for them to do what they needed to get done.”

Ron Hosko, assistant director of the bureau’s Criminal Investigation Division before his retirement, said the agency could also hit snags at the Justice Department, where the White House’s political appointees will have more sway.

“There are often frustrations in sensitive, important investigations that you end up with prosecutors—either too few, who are unwilling to move forward at the desired pace, or too many and you turn every simple decision into a debate club—and it slows progress,” Hosko said. “Here, I think that is the pulse that you in the media and others ought to be keeping close to: What’s the pace? Are the investigators getting the prosecutors’ support that they need?”

Attorneys in the DOJ’s National Security Division are responsible for securing subpoenas and court orders for the FBI agents working the Russia investigation. Hosko said agents sometimes suspect politics is to blame when Justice Department lawyers don’t move as fast as they would like.

“Prosecutors will sometimes start to debate and question every word in a subpoena and it tends to slow progress,” Hosko said. “And then you start to ask questions about—is this because of something political?”

Hosko also said he believes Dana Boente, the acting head of the DOJ’s National Security Division, has “impeccable integrity,” and wouldn’t let political concerns slow an investigation.

But those reassurances are unlikely to quell the fears of veteran FBI agents investigating the president’s associates.

“The Orange blob in the WH doesn’t care about anyone or anything he can’t control,” said the former high-ranking official. “He’s made that abundantly clear.”