Get breaking news alerts and special reports. The news and stories that matter, delivered weekday mornings.

WASHINGTON - There was a moment when it looked like President Donald Trump might be able to execute a historic political realignment by bringing people like union members into the GOP.

But the nation’s top union leader, AFL-CIO President Richard Trumka, who was initially more optimistic about Trump than most other Democratic-leaning political figures, said Wednesday that won't be happening.

"There’s not going to be a realignment," said Trumka, who resigned from the president's manufacturing council this month in response to the president's handling of the unrest in Charlottesville, Virginia, over the removal of a Confederate monument.

"A lot of the optimism has faded," Trumka told a group of reporters at a breakfast organized by the Christian Science Monitor. "We haven’t seen the things that we were hopeful that we could work with him on."

Let our news meet your inbox. The news and stories that matters, delivered weekday mornings. This site is protected by recaptcha

Despite not supporting Trump in last year's presidential election, Trumka and other union leaders saw common ground with the president's populist, pro-worker rhetoric on the campaign trail. But once he got to the White House, Trump has pursued a more conservative agenda on tax cuts and repealing the Affordable Care Act.

"You had two factions in the White House. You had one faction that actually had some of the policies that we would have supported on trade and on infrastructure - but they turned out to be racist," Trumka said.

"And on the other hand," he continued, "you had people who weren’t racist, but they were Wall Streeters. And the Wall Streeters began to dominate the administration and have moved [Trump's] agenda back to everything he fought against in the election."

President of the American Federation of Labor and Congress of Industrial Organizations (AFLCIO) Richard Trumka speaks during day two of the Democratic National Convention at Time Warner Cable Arena on September 5, 2012 in Charlotte, North Carolina. Alex Wong / Getty Images

The recent ouster of Trump's former chief strategist Steve Bannon was seen as a decisive blow against the populist wing in Trump's White House. Ascendent now are the more establishment voices, typified by aides like economic adviser Gary Cohn, the former president of Goldman Sachs.

Trumka said he’s not sure where the president fits in.

While the AFL-CIO supported Hillary Clinton last year, Trumka criticized her for failing to connect with workers on economic issues. "They didn’t believe what she was saying," he said.

Trump outperformed 2012 Republican nominee Mitt Romney by 3 percentage points among union members, Trumka said, but Clinton underperformed former President Barack Obama by 10 percentage points. The remainder either stayed home or voted for a third-party candidate.

For the 2018 midterm elections, Trumka said the AFL-CIO will focus primarily on communicating with its members, rather than voters at large, since so many union households strayed last year.