WILMINGTON, Ohio/WASHINGTON (Reuters) - Some of Donald Trump’s Hispanic backers distanced themselves from the Republican nominee on Thursday for standing by a hardline approach to illegal immigration in a key speech after indicating for weeks that he may soften his approach.

Trump tried to clarify confusion about immigration, his signature policy issue, in a speech on Wednesday. He said the only way undocumented foreigners could live in the United States legally if he is elected on Nov. 8 would be to leave the country and apply for re-entry.

But the businessman, trailing Democratic rival Hillary Clinton in opinion polls, did back away from earlier promises to deport immediately the 11 million immigrants in the United States illegally and said he would prioritize those with criminal records.

While polls show a large majority of Hispanic voters oppose Trump, the withdrawal of support from among his small group of Latino backers underscores how difficult it is for Trump to broaden his support with minorities and moderate voters.

Alfonso Aguilar, who recently organized a support letter on behalf of Trump, said he felt “disappointed and misled” by the fiery speech and withdrew his backing.

“For the last two months he said he was not going to deport people without criminal records. He actually said that he was going to treat undocumented immigrants without criminal records in a humane and compassionate way,” Aguilar told CNN. He is the president of the Latino Partnership for Conservative Principles group.

Trump used his Wednesday appearance in Phoenix to clarify his stance on illegal immigration. But instead of moderating his message as many expected, Trump returned to the hardline rhetoric that powered him to victory in the Republican presidential nomination race over 16 rivals, heartening conservatives drawn to Trump by the issue.

Some members of a council Trump formed last month to advise him on Hispanic issues expressed reservations about or cut ties to the New York real estate developer’s candidacy after the Phoenix speech.

Jacob Monty, a Texas attorney and member of the group, said he was withdrawing his support and would not vote in the election.

Slideshow ( 8 images )

“There was nothing pro-business in that speech,” Monty told MSNBC. “We were hoping for some glimmer of the Donald Trump that we met with a week and half ago, but it never came.”

“SCAM”

Panel member Ramiro Pena, a Baptist pastor in Texas who spoke at the Republican National Convention in July, wrote in an email to party leaders that he believed Trump would lose the election and that the advisory panel was a “scam.”

But other Latino advisers, including Florida pastor Mario Bramnick and Kentucky State Senator Ralph Alvardo, said they would continue working with the Trump campaign.

Republican National Committee Chairman Reince Priebus downplayed the fallout from Trump’s speech, telling CNN that the nominee made clear he first wants to build the border wall and deal with criminal elements, then have a “humane conversation” about other illegal immigrants.

“Somehow or another no one is talking about that piece,” Priebus said.

At a campaign rally on Thursday in Wilmington, Ohio, Trump said his immigration plan would treat everyone with “dignity, respect and compassion” but prioritize compassion for American citizens and include some kind of ideological screening.

“We only want to admit those into our country who share our values and love our people,” Trump said.

Slideshow ( 8 images )

Trump gave his Phoenix address, which was flagged as a major policy speech, just hours after he met with Mexican President Enrique Pena Nieto in Mexico City.

At a joint press conference after the meeting, Trump said the pair discussed his campaign promise to build a border wall but not which country would pay for it. Pena Nieto said on Twitter on Wednesday night he had “made it clear” Mexico would not be paying for the wall.

Trump supporters at the Wilmington rally said they approved of the candidate’s immigration policies but moderate Republicans in Arizona, where Latinos make up more than 30 percent of the population, told Reuters they were less swayed by his message.

Clinton’s campaign called Trump’s immigration speech a “disaster” and said it would begin running advertisements in Arizona, a sign it sees a chance of winning a state that has long backed Republican presidential candidates.

Clinton raised about $143 million in August for her presidential bid and the Democratic Party, her campaign announced. Trump has not yet released his fundraising totals for the month of August.