For all the talk about the need for voter-identification laws, you’d think millions of Americans were impersonating dead people to get their candidates elected, or casting multiple ballots after breakfast, lunch, and dinner.

Not even close.

Voter fraud—the foundational premise for controversial new voter-ID laws—is far more rare in the U.S. than ID proponents would have us believe. The Department of Justice doesn’t even compile figures of how many people have been convicted of fraud in the last decade, and says prosecutions have focused on tampering by election officials and local politicians, not on voters themselves. A New York Times investigation found that between 2002 and 2005 only 96 people were indicted for federal election-related crimes, and 70 of them were convicted. Of those, 41 were campaign employees and government officials, and just five were voters who cast multiple ballots.

Yet impersonation is the only type of election crime that strict new voter-identification laws can actually prevent. Since 2011, 10 states have passed such laws, which typically require people to show a photo ID when they vote. Before the 2010 interim elections, only two states—Indiana and Georgia—had voter ID laws on the books.

Lee Rowland, counsel at the Brennan Center for Justice’s Democracy Program, says the public’s confidence in their democracy is “undermined” by these laws. “The real fraud,” she says, “is the extent to which fake stories about supposed fraud have been used to justify restrictive voter-ID laws.”

According to the center, new photo-ID laws passed in the last legislative cycle since 2010 will make it more difficult for about 5 million Americans to vote. More than 21 million Americans do not have government-issued photo ID now necessary for voting in states like Florida and Pennsylvania (PDF).

The question is who benefits if these people don’t—or can’t—vote. “It’s no coincidence that these laws are almost completely passed by Republicans,” says Tova Wang, a Senior Democracy Fellow at Demos, a nonpartisan policy think tank in New York. That’s because voters without photo ID tend to be young people, the elderly, and minority populations, particularly African-Americans, who tend to vote Democratic.

Republicans say it’s a nonpartisan issue about upholding the sanctity of the vote. “Americans of all political persuasions can agree that it is the integrity of the vote that safeguards the integrity of our democratic process,” says Kirsten Kukowski, a spokeswoman for the Republican National Committee. “When that integrity is compromised, we must act.”

Last month, RNC Chairman Reince Priebus called new voter-ID laws “common-sense reform [that] is not partisan but practical.”

Voter fraud was once a real issue in the U.S. During the Bleeding Kansas crisis of the 1850s, gangs intimidated voters into casting their ballots for and against the extension of slavery. But in the last 50 years or so, says Harvard historian Alexander Keyssar, voting fraud “hasn’t been a problem.” The mechanics of voting came under renewed scrutiny after the 2000 Florida election recount. Since then, “states have been vigilant not to be the next Florida,” says the Brennan Center’s Rowland. In an unexpected twist, this year, Florida itself might be “the next Florida,” she says.

The state’s newly elected Republican governor, Rick Scott, signed a trifecta of new voting rules into law in May 2011 under House Bill 1355. “When you go out to vote, you want to make sure that the other individuals that are voting have a right to vote,” said Scott at the time.

The bill addresses a handful of voting practices that critics say ultimately makes it harder for people to participate in elections. Groups that conduct registration drives, such as the League of Women Voters, now have to submit completed voter applications within 48 hours of filling them out, or face fines up to $1,000. People who move to a different county within the state are required to cast provisional ballots—historically counted only about half the time—if they fail to notify their local election board of their move before Election Day. And early voting, which helps people who, for various reasons, don’t vote on Election Day, is now restricted to the period between the 10th and third day before the general election. This includes the Sunday before Election Day, which in 2008 brought many African-Americans to the polls for church-voting drives known as “Souls to the Polls.”

If fewer people vote early, polls could be inundated, says Ion Sancho, elections supervisor for Leon County, home to the state’s capital, Tallahassee. He says Florida doesn’t have enough polling locations to accommodate the 8 million or more voters expected to cast ballots in the general election. Sancho also says that in 2008, 13 percent of all new voter registrations came from third-party groups.

Florida’s elections officials have been disseminating information about the new laws ahead of the November election, hoping to avoid a flashback to 2000 after a decade of relatively drama-free elections in the Sunshine State. But some fear the result of the new law will be vast numbers of voters disenfranchised—in a state where fraud is rare. The Florida Department of Law Enforcement has investigated 178 claims of election fraud since 2000. “You’re more likely to get hit by lightning than to encounter voter fraud in Florida,” says Sancho.

A spokeswoman from Governor Scott’s office didn’t respond to requests for comment.

Chris Cate, a spokesman at the Florida Department of State, says the new law has “improved accountability” in the voter-registration process.

Others take a more global perspective. When HB 1355 was being argued in the state Senate last year, Republican Sen. Mike Bennett posed the question: “Do you read the stories about the people in Africa? The people in the desert, who literally walk two- and three-hundred miles so they can have the opportunity to do what we do, and we want to make it more convenient?” He continued, “This is a hard-fought privilege. This is something people die for. You want to make it convenient?”