Paul Nehlen, the far-right activist who is challenging the House speaker, Paul Ryan, for his congressional seat, will not get the backing of Breitbart News and Steve Bannon.

A source close to Bannon, the former White House chief strategist who is orchestrating a slate of challengers to Republican establishment figures, confirmed to the Guardian that Nehlen became persona non grata after he appeared on a white supremacist podcast, Fash the Nation, earlier this month.

Bannon had never been particularly enthusiastic about Nehlen despite the wealth of coverage on Breitbart, the source said.

Nehlen is a businessman and so-called “mini-Trump” who was backed by leading rightwing figures including Sarah Palin and Ann Coulter in the 2016 primary for Ryan’s Wisconsin congressional seat. The speaker’s eventual margin of victory was huge – 84% to 16% – despite aggressive pro-Nehlen coverage from Breitbart.

Breitbart featured articles that bashed Ryan as a hypocrite for building a fence around his Wisconsin house while not including funding for a wall on the US border with Mexico, and for opposing Donald Trump’s proposed Muslim ban while sending his children to a Catholic school.

Nehlen had remained close to Bannon’s circle and the website to which Bannon returned after his spell in the White House. Nehlen joined Bannon in endorsing and campaigning for Roy Moore, for example, in the Alabama Senate election that was won this month by the Democrat Doug Jones.

Trump has praised Nehlen. In August 2016, after the billionaire was condemned for criticizing the family of a Muslim American soldier killed in battle, Nehlen praised Trump. The then Republican nominee returned the favor in a tweet.

“Thanks to @pnehlen for your kind words, very much appreciated,” Trump wrote, at a time when Nehlen was still fighting to unseat Ryan.

In recent weeks Nehlen has attracted criticism for controversial tweets on immigration and other subjects. On Tuesday, for example, he tweeted that he was reading The Culture of Critique, a book by Kevin MacDonald widely seen as antisemitic.

Nehlen has repeatedly attacked critics for taking “shekels for hire” and has tweeted various memes that state “it’s OK to be white”.

On Twitter on Tuesday night, Breitbart senior editor Joel Pollak said: “We don’t support him. Haven’t covered him in months.” He added: “He’s gone off the deep end.”

Arthur Schwartz, a Bannon adviser, told CNN: “Nehlen is dead to us.”

On Wednesday, Nehlen told CNN: “Putting America First above all other nations, such as being pro-wall and for freedom of lawful speech, has brought a coordinated attack by globalists from both parties.

“Nevertheless I will continue to stand strong against anti-American sentiment, however it manifests.”