Aug. 18, 2010 -- The increasing flaps over the proposed Islamic center in New York and immigration raise the question of whether the Republican party is alienating two voter bases that the Bush administration went to great lengths to woo.

Former president George W. Bush heavily courted Muslims in his first campaign, winning 78 percent of the Muslim vote in 2000, according to the Council on American-Islamic Relations. That number dropped off sharply in 2004, following the wars in Afghanistan and Iraq, even though the Bush administration tried to appease American Muslims by making a distinction between terrorists and the rest of the Muslim community.

The Bush administration made an even bigger push to attract Hispanics to the party. Bush selected Alberto Gonzales as the first U.S. attorney general of Hispanic heritage, supported amnesty for illegal immigrants, and even spoke Spanish on the campaign trail. This push paid off in the elections. Bush captured 35 percent of the Hispanic vote in 2000 and 44 percent in 2004.

But support for Republicans from both minority groups looks to be uncertain amid the increasing fervor over the cultural center and the immigration issue.

Muslims, who have already turned overwhelmingly to Democrats in recent elections, are likely to turn away further. Republican Muslims say the debate will alienate their constituency if it continues to brew.

"One of the strengths in the United States has been separation of church and state," said Saghir "Saggy" Tahir, a Pakistani-American Republican who has served in the New Hampshire House of Representatives for nearly a decade. " I do believe there will be an adverse effect of dragging the religion into the politics."

Tahir, who is the only Pakistani-American not born in the United States to be elected to a state assembly as a Republican, said it will be difficult for him to build up support for the GOP in his constituency and state if the issue continues to drag on.

"I have talked to a lot of people... They are Republicans but they don't like anybody to come to the religious part of it and that definitely alienates them, and these are those businesses that have helped elect Republicans even though they were Democrats," Tahir told ABC News.

Muhammad Ali Hasan, a film director and commentator who ran twice for office in Colorado on the Republican ticket and founded Muslims for Bush, said attacks by the GOP on the Islamic center are an "assault" on minorities.

"This is not the party of Ronald Reagan or George W. Bush," Hasan, who considers himself a lifelong Republican, told ABC News. "W. Bush condoned that Islam was a religion of peace. Ronald Reagan was someone who supported amnesty (for illegal immigrants). Both men supported opportunities for minorities, for Muslims, for Latinos."

But Democrats are also divided over the location of the Islamic center. Senate Majority Leader Harry Reid, D-Nev., split from President Obama over the issue, when Reid said the mosque, which is part of a larger proposed community center, should be built elsewhere.

"The Constitution gives us freedom of religion," Reid said Tuesday, but added, "I think that it's very obvious that the mosque should be built someplace else."

Speaker of the House Nancy Pelosi, however, said she "respects the right of people in this country to express their religious beliefs in their property" and that the issue should be left up to New Yorkers. But she added that she joins those who have called for looking at "how is this opposition of the mosque being funded."

"There's no question that there is a concerted effort to make this a political issue by some," Pelosi, D-Calif., told reporters, adding that she wants to know how this is "being ginned up."

Obama said on Friday he believes "that Muslims have the same right to practice their religion as everyone else in this country. And that includes the right to build a place of worship and a community center on private property in Lower Manhattan."

As far as the issue of the Islamic Center is concerned, the Republican party has many voices and voters will understand that, said Republican consultant Ron Kaufman.

Some Republicans, specifically Bush administration officials, have spoken out against the outcry that's been created since the president made his remarks, and expressed concern that the GOP is taking the issue too far.

Ted Olson, who served as solicitor general under President Bush and whose first wife died in the plane that crashed into the Pentagon, today expressed his support for the Islamic center.

"I think probably the president was right about this. I do believe that people of all religions have a right to build edifices or structures or places of religious study where the community allows them to do it under the zoning laws and that sort of thing," Olson told MSNBC's Andrea Mitchell. "I don't think it should be a political issue."

New Jersey Gov. Chris Christie called on both parties -- especially his fellow Republicans -- to temper their attacks, although he wouldn't comment on where he thinks the mosque should be built.

"We cannot paint all of Islam with that brush," said Christie. "We have to bring people together. And what offends me the most about all this, is that it's being used as a political football by both parties. And what disturbs me about the president's remarks is that he is now using it as a political football as well. I think the president of the United States should rise above that."

At a time when the economy and jobs remain Americans' top concerns, turning the Islamic center into a campaign issue could also make Republicans seem out of touch with the issues that Americans care most about.

On the immigration front, many Hispanics, the fastest growing minority group in the United States, are outraged at a new Arizona law that requires law enforcement to ask people for their immigration documents if they have "reasonable suspicion" that someone being detained is an illegal immigrant.

Similar laws are being considered around the country, and the crackdown in states such as Texas and Arizona that have a sizeable Latino population could divert Hispanic votes to Democrats.

Even Marco Rubio, the Republicans' star Latino candidate who is running for the U.S. Senate seat in Florida, initially opposed the Arizona law although he switched his position in May and said he would have voted for the law if he were in the Arizona state legislature.

Some Republican consultants accuse the Democrats of hyping both the Islamic center and immigration issues. Kaufman, who worked both on President George H.W. Bush's campaign and as an adviser to the younger Bush, said the Arizona law is being blown out of proportion by Democrats and that the party is still attracting Hispanic voters and more diverse candidates.

"If you stoke the flames, as Democrats have done, they're making it a bigger issue, a more hostile issue and a more divisive issue than it really is. And in the end, I think voters will understand that," Kaufman told ABC News. "Across the country, we have more diverse, more interesting candidates -- women, Hispanic, Muslim. We have by far the most diverse Republican set of candidates in my lifetime."