A tough new law in the state of Wisconsin requiring voters to carry photo ID before they can cast their ballot is being challenged in a federal lawsuit that claims thousands of poor, black and elderly people could be disenfranchised. At the same time the American attorney general, Eric holder, has given a strong speech promising to defend the rights of previously marginalised voters.

The legal action, lodged in the federal court for the eastern district of Wisconsin, opens a new front in the battle over voter registration before next year's presidential election. Civil rights groups are warning that a wave of legislative restrictions introduced in more than 30 states amounts to a concerted attack on voting rights in America on a level not seen since the days of segregation.

The lawsuit has been brought by the American Civil Liberties Union against Scott Walker, the Republican governor of Wisconsin, and other state officials. It is the only active federal challenge to the imposition of ID laws on voters.

The suit comes as attorney general Eric Holder made his most combative speech in Austin, Texas, on voting rights. Speaking symbolically at the library named after Lyndon Johnson, the president who signed the Voting Rights Act in 1965, Holder pledged to resist any attempt to roll back the advances in equality that had been made almost 50 years ago.

"Are we willing to allow this era – our era – to be remembered as the age when our nation's proud tradition of expanding the franchise ended?" he said.

Thirty-four states have tried to introduce photo IDs into the electoral process, ostensibly as a means to counter fraud, with nine introducing restrictions into law. Of those, Wisconsin, which passed its new law in May that will become effective from February next year, is seen as one of the most draconian.

Under its terms, voters who used to be able simply to provide proof of residence to register to vote now have to jump through an intricate series of hoops before they can cast their ballot. Only a limited list of photo identification is accepted, and the law applies both to people voting in person in voting booths and by absentee ballot.

Among the 20 plaintiffs listed in the action are Ruthelle Frank, 84, who has voted in Wisconsin elections since 1948. But she has no form of photo ID acceptable under the new law, and does not have a copy of her birth certificate.

Another plaintiff, Barbara Oden, 57, was trapped in a catch-22. In order to get an eligible photo ID, she was advised that she needed to obtain a social security card as proof of citizenship; but when she turned up at the relevant office she was told she couldn't have a social security card because she didn't have a photo ID card.

The lawsuit catalogues a Kafkaesque world in which voters are being made to travel across the state in a search for the correct documentation. Justin Luft, 20, travelled twice to an office to get a driver's licence – an eligible photo ID under the voter law – only to be told on both occasions that he could not have one because he didn't have a social security card.

"This law will leave thousands of people discouraged and disenfranchised," said Jon Sherman, a lawyer with the ACLU's voting rights project. "What's so galling about this is that we've already had this conversation in the US – we don't make people run around and pay money in order to cast their ballot."

The lawsuit argues that the photo ID law is unconstitutional under the 14th amendment, which bans the imposition of burdens on the right to vote, and the 24th amendment, which says that the right must not be removed because someone has failed to pay their taxes.

The complaint says numerous groups are at risk of being disenfranchised, including poor people who may not be able to afford petrol to travel to the driver's licence office, or the $135 to obtain a US passport. There are more than 12,000 families in Wisconsin who have no income of any sort other than food stamps.

Other potentially vulnerable groups are students, elderly people and the disabled. African Americans are also disproportionately represented among those potentially deprived of the right to vote; the NAACP earlier this month petitioned the UN over what it says is the greatest threat to voter participation in America since the early 20th century.

In his speech Holder urged Americans to "call on our political parties to resist the temptation to suppress certain votes in the hope of attaining electoral success and, instead, achieve success by appealing to more voters".

"We need election systems that are free from fraud, discrimination, and partisan influence – and that are more, not less, accessible to the citizens of this country." He promised to resist attempts to nullify measures introduced in the 1960s designed to prevent states, particularly those in the south, from disenfranchising black voters.

The department of justice is already examining, under those 1960s provisions, measures introduced in Texas and South Carolina to require photo identification at the polling booth.

States that have tried to impose photo identification requirements argue that it is necessary to combat fraud at the ballot box. But studies have shown that fraud is not as ubiquitous a problem as has been made out.

A Department of Justice study found that of 300m votes cast between 2002 and 2007, there were only 86 cases of confirmed fraud.

"You have to consider the cost of contracting the electorate by hundreds of thousands of people. In order to kill a fly, they are using a nuclear weapon," Sherman said.

The other states that already have photo ID requirements on the statute books are: Alabama, Georgia, Indiana, Kansas, Mississippi, South Carolina, Tennessee and Texas.