EDGARTOWN, MA - AUGUST 14: U.S. President Barack Obama delivers remarks during a press briefing at the Edgartown School on August 14, 2014 in Edgartown, Massachusetts. The president, vacationing on the island, spoke about the unrest in Ferguson, Missouri and Iraq. (Photo by Rick Friedman-Pool/Getty Images) President Barack Obama delivers remarks during a press briefing at the Edgartown School on Aug. 14, 2014 in Edgartown, Mass. (credit: Rick Friedman-Pool/Getty Images)

WASHINGTON (AP) — President Barack Obama is facing potential rifts with members of his own party in tough re-election contests as he barrels toward a fall fight with Republicans over his ability to change immigration policies.

If Obama takes the broadest action under consideration — removing the threat of deportation for millions of people in this country illegally — the short-term risks appear greatest for Senate Democrats in conservative-leaning states. Weeks before the November vote, they could find themselves on the hot seat for their views not only on immigration but also on Obama’s use of his presidential powers.

Wary of what could be coming, some of those lawmakers have said Obama should act with caution.

“This is an issue that I believe should be addressed legislatively and not through executive order,” said Sen. Kay Hagan, D-N.C., one of the top targets for Republicans trying to retake control of the Senate.

Sen. Mark Pryor, D-Ark., another vulnerable incumbent, said in a statement that he also is “frustrated with the partisanship in Washington. But that doesn’t give the president carte blanche authority to sidestep Congress when he doesn’t get his way.”

Such statements have immigration advocates on edge.

A coalition of advocacy groups, in a letter to congressional Democrats on Friday, said immigrant families should not have to wait until after the November elections for relief. The organizations said any attempts by Democrats to delay or dilute administrative changes “will be viewed as a betrayal of Latino and immigrant communities with serious and lasting consequences.”

The letter was released because of advocates’ concerns that leading Senate Democrats may be shifting their positions because of political considerations after previously urging Obama to act.

A spokesman for Sen. Chuck Schumer, D-N.Y., declined to say Friday whether Schumer still believes Obama should act by October, as Schumer had said before. A spokesman for Sen. Dick Durbin of Illinois, the second-ranking Democrat, said the timing of executive action on immigration was up to Obama. Senate Majority Leader Harry Reid’s spokesman had no comment on timing.

Still, Obama looks determined to move forward on his own despite the political risks for Democrats.

He is irritated by House Republicans’ inaction on immigration legislation passed last year by the Senate. The crisis over unaccompanied minors arriving in South Texas does not appear to have deterred him, and the slowdown of arrivals at the border may be shifting the issue away from the spotlight anyway.

The exact contours of Obama’s plans remain unclear.

Advocates and lawmakers who have talked with administration officials anticipate that he could expand a program that granted work permits and deferred deportation to more than 700,000 immigrants brought illegally to the U.S. as kids. It could be extended to include parents of those children, as well as parents of U.S. citizens, or potentially others — groups that could add up to perhaps 5 million people.

During a news conference this month, Obama was not specific on his immigration plans. He did say that in the absence of congressional action and in order to address the crisis involving unaccompanied youths, he had to shift resources on his own and exercise prosecutorial discretion.

“I promise you the American people don’t want me just standing around twiddling my thumbs and waiting for Congress to get something done,” Obama said.

Some GOP leaders worry that opposition to a comprehensive overhaul will harm their party in the 2016 presidential race, where Latino turnout is higher than in midterm elections. Hispanics are a fast-growing sector of the presidential electorate and backed Obama overwhelmingly in 2012.

But Republicans also see a nearer-term chance to translate Obama’s potential executive actions into electoral success in November. Republicans need to win a net of six seats in order to take control of the Senate for the remainder of Obama’s term. The GOP already is all but assured of maintaining control of the House.

As Republicans meet with voters in their districts during the summer break, lawmakers have raised alarms about the scope of Obama’s potential plans. In some cases, they are hearing clamors for impeachment in return.

“It is up to Congress to actually go back and restrain this guy,” one voter told GOP Rep. Andy Harris of Maryland at a town hall meeting on the Eastern Shore. Harris had warned that Obama could expand an existing deportation relief program to 4 million or 5 million more people, “competing with Americans for work.”

Republicans have tagged Obama as an “imperial president” who goes around Congress rather than working with lawmakers, and House Republicans have moved to sue him over it. The prospect of the president making a unilateral move on a contentious issue such as immigration has Republican consultants salivating.

“President Obama’s executive amnesty would inject adrenaline into an electorate already eager to send him a message of disapproval,” said Brad Dayspring, communications director for the National Republican Senatorial Committee.

Also problematic for Obama: His apparent plans to act on his own authority come after years of saying that he did not have the legal justification to proceed without Congress.

“If, in fact, I could solve all these problems without passing them through Congress, I would do so. But we’re also a nation of laws,” he said in November. A heckler had interrupted a speech he was giving in San Francisco, prodding him to halt deportations, which have reached record highs on Obama’s watch.

Since then the White House has apparently concluded otherwise.

Democratic pollsters argue that any executive action by Obama could give a political boost to Democrats, not just from newly energized Latino voters but from an electorate at large that would welcome any action from gridlocked Washington.

“Voters are so sick of the do-nothing Congress they don’t mind if there’s an imperial president,” said Democratic pollster Celinda Lake. “They would just like someone to get something done about something.”

(© Copyright 2014 The Associated Press. All Rights Reserved. This material may not be published, broadcast, rewritten or redistributed.)