Once outside the administration and free to pursue his political enemies, Mr. Bannon set out on an audacious mission to challenge Republican incumbents he deemed insufficiently loyal to Mr. Trump’s agenda. He vowed to replace Senator Mitch McConnell, the Republican leader, and started backing far-right candidates, some with questionable backgrounds and losing track records at the polls.

His full-throated, unfailing support of Roy S. Moore in Alabama, even after allegations surfaced that the former judge preyed on girls as young as 14, ended in an embarrassing setback: The state sent a Democrat to the Senate for the first time in a generation.

Several people with knowledge of the dynamics at Breitbart said that Mr. Bannon had lost the confidence of executives and writers who had been fiercely loyal to him as he helped transform the website from a scrappy start-up to one of the biggest and most antagonistic megaphones on the right.

But that confidence faded in recent days and weeks as they came to believe he displayed serious lapses in judgment, according to interviews with half a dozen people close to the situation. Some associates and friends described Mr. Bannon as being detached from reality, unable or unwilling to grasp the severity of his falling out with the White House and its potential effect on Breitbart as a business.

His situation at Breitbart grew untenable, said one person close to the situation, in part because Ms. Mercer, whose family finances conservative causes with their hedge fund wealth, became concerned that she could face legal exposure. She feared that some of the website’s cheerleading coverage of populist conservative campaigns — like the Senate race in Alabama — could be construed as corporate contributions to those candidates, which are barred under federal election law.

Numerous people who have worked with Mr. Bannon over the past few months said that his attitude lately had grown more imperious and aloof than normal. And his outward lack of any emotion about his messy public breach with the president and the Mercer family struck some not as the calloused indifference of a political operative but as nihilistic.

When Mr. Trump first denounced Mr. Bannon last week, saying, “He not only lost his job, he lost his mind,” Mr. Bannon insisted to his writers and editors at Breitbart that it would all blow over. When reports began circulating that Ms. Mercer had cut him off, he denied it outright. And when friends started asking him about rumors that his job was in jeopardy, he insisted that everything was fine. “The Mercers haven’t given me money in years,” he told multiple people, playing down their significance in his work and insisting that an initial $10 million investment in the website was all they had provided.