Just two weeks ago, the Republican National Committee and the Donald Trump campaign announced that they were forming the “National Hispanic Advisory Council for Trump” with 23 Latino members of the GOP.

On Thursday, multiple members of the group said they were withdrawing their support from the Republican nominee’s campaign after he vowed to enact hard-line immigration policies in a speech on Wednesday night.

It’s not hard to see why some people on the council might feel betrayed. Many of them met with Trump on August 20 in a meeting at Trump Tower. During the meeting, he was said to have expressed support for a rethink of his immigration message—even contemplating giving undocumented immigrants a path toward legal status.

Yet Trump gave a very different message to a roaring Phoenix crowd on Wednesday night, emphasizing that he would crack down on visa overstayers, cut federal funding for “sanctuary cities,” bar a path to legalization for undocumented immigrants, and create a new “special deportation task force” to bring the number of deportations up from President Obama’s already sky-high levels.

On Thursday, at least four members of the group either said they were resigning or harshly criticized the speech, and some reports suggest there are more to come.

Grace Flores Hughes, a member of the committee and a former federal government official in multiple presidential administrations, told me in a phone interview on Thursday that she was “withdrawing support for Trump.”

“As long as he continues with this sort of rhetoric, no, none of us are going to support him,” Hughes said. “He’s too right-wing.”

She emphasized that she would not back Hillary Clinton and she was never a member of Trump’s campaign staff—just part of an advisory board convened by the RNC.

“The majority of people who are here illegally are here to seek a better life,” Hughes said. “The majority of them are good, hardworking, honest people. I think we need to concentrate on that and talk about that.”

Another member of the council, Jacob Monty of Texas, said in a Facebook post that he had suggested a sensible immigration plan to Trump. “He rejected that tonight and so I must reject him,” Monty wrote. “He was moving toward a [reasonable], pro business and [compassionate] immigration plan. Tonight he was not a Republican but a populist, modern day Father Coughlin who demonized immigrants. He must want to lose. He can do that without me.”

Texas pastor Ramiro Pena, another member, said that the council was a “scam” in an email to RNC officials that was first reported by Politico.

“I am so sorry but I believe Mr. Trump lost the election tonight,” Pena wrote. “The ‘National Hispanic Advisory Council’ seems to be simply for optics and I do not have the time or energy for a scam.”

A fourth member, Alfonso Aguilar, the president of the Latino Partnership for Conservative Principles, told Politico he was also considering ditching the advisory committee.

“It’s so disappointing because we feel we took a chance, a very risky chance,” Aguilar said. “We decided to make a big U-turn to see if we could make him change. We thought we were moving in the right direction … we’re disappointed. We feel misled.”

The Trump campaign did not immediately respond to a request for comment.