The world tried to move on after the second world war, but one group, having survived the ghettoes and the death camps, was not about to let Nazi foot soldiers walk free. Jonathan Freedland reports on the Jewish avengers who tracked down and executed their tormentors

After more than six decades, with libraries full of books, archives packed with testimony and warehouses crammed with documents, you might think we know all there is to know about the Holocaust. And it's true that, as that event slips from memory into history and the remaining survivors enter their last years, the flow of new revelations about the Nazi campaign to eradicate the Jews has slowed to a trickle. There was a surge of fresh information when the Soviet records were opened up after 1991, but there is no further comparable cache of papers waiting to be mined. The key facts are in place, as thoroughly documented as any event in human history: the Nazis wanted to rid the world of Jews and, through both mass shootings and gas chambers, they succeeded in murdering six million of them, including more than a million children.

And yet there is at least one aspect of the Holocaust that has barely registered on the world's consciousness, a remarkable fact - and, causally connected to it, an extraordinary story. This fact is shocking, even if it is hardly new, and the story it triggered is stunning: a tale of heroism and violence, a burning quest for justice and revenge that reads more like ancient legend than contemporary history.

First the fact. The dirty little secret of the Holocaust, which many would regard as the greatest crime in human history, is that no one was punished for it. Of course, "no one" is not literally true. There were some, at the very top of the Nazi state, who were famously called to account at the Nuremberg trials. But that amounted to 24 individuals. And it takes more than 24 people to kill six million.

What of the men who operated and policed the death camps, who closed the doors on the gas chambers, who administered the pellets of lethal Zyklon B? What of those who manned the ghettos or drove the trains, those who used rifle butts to herd batch after batch of Jews towards vast pits dug from the earth, first stripping them naked, then shooting them in the back, under strict orders to use no more than one bullet per victim, so that many of those who fell into the pits were not yet dead but buried alive - so that witnesses later spoke of the pits seeming to move and writhe, to breathe, for days afterwards? What of those guilty men?

There were tens, if not hundreds, of thousands of them, and all but a tiny proportion eluded justice. Consider this numbing statistic. After the war, allied officials identified 13.2 million men in western Germany alone as eligible for automatic arrest because they had been deemed part of the Nazi apparatus. Fewer than 3.5 million of these were charged and, of those, 2.5 million were released without trial. That left about a million people - and most of them faced no greater sanction than a fine or confiscation of property that they had looted, a temporary restriction on future employment or a brief ban from seeking public office. By 1949, four years after the war, only 300 Nazis were in prison. From an original wanted list of 13 million, just 300 paid anything like a serious price.

Why were more of the guilty not punished? "Because it would have been a never-ending task," says David Cesarani, research professor at Royal Holloway, University of London, and a leading authority on the Holocaust. He cites the British attempt to convict those responsible for the killing at Belsen. The trial took nine months and left the British exhausted. "That was just one camp and there were, what, 70 camps, with hundreds of people at each one. To say nothing of the Gestapo officers and the men of the Einsatzgruppen [the mobile killing units]." Pursuing all those responsible for the slaughter of the Jews would have meant trying thousands upon thousands of people - and it would have ended in the jailing of almost the entire adult male population of Germany. "The allies put their hands up in despair."

And not only the allies. Watching this was a motley group of survivors of the ghettos and the camps, Jews who had somehow cheated death. They were led by a handful of young warriors who had earlier, when barely into their 20s, formed the armed Jewish resistance to Nazi occupation - first in the ghettos, where they had scarcely a weapon between them, then as partisans in the forests of Nazi-held eastern Europe. These young men and women had been shaped by the Shoah: most had lost their entire families and all had witnessed unimaginable horrors.

Now, at war's end, they saw that the guilty were about to walk free. The world wanted to move on; the Americans, especially, were anxious to absorb western Germany into a new alliance against the Soviet bloc. But these fighters were not ready; before they could be at peace, they would first have to avenge the blood of their fellow Jews.

Some accounts suggest the group that would come to be known as the Nokmim, Hebrew for avengers, was born in the spring of 1945 in Bucharest. A Passover gathering of survivors was addressed by Abba Kovner, of the Jewish uprising in the Vilna ghetto, who would go on to become the uncrowned national poet of the State of Israel. He spoke passionately, invoking Psalm 94, in which God promises that he shall deal with the enemies of the people of Israel: "He will repay them for their iniquity and wipe them out for their wickedness." This, Kovner suggested, was the fate that should be meted out to the Germans. And if the courts of international justice would not do it, then the Jews should do it themselves.

The story of what followed has been recounted in several books, most notably Rich Cohen's The Avengers, which benefited from Cohen's family connection with some of the group's lead players, and Forged In Fury, written by the BBC's former Jerusalem correspondent, Michael Elkins, and first published in 1971. One former member of this clandestine movement, Joseph Harmatz, wrote a revealing memoir, From The Wings, a decade ago.

These accounts differ on some details but agree on the essentials. Calmly, what Harmatz calls the Vengeance group set about implementing death sentences they themselves had passed. Perhaps posing as allied military police serving the postwar occupation authorities, they would identify a Nazi who had melted back into civilian life, stage an arrest and spirit him away. Some of these ex-SS men would be strangled, others hanged - all the better for passing off the death as a suicide. Hangings might take place in a garage, with the subject forced to stand on a car roof while his neck was placed in a noose attached to an overhead beam: an Avenger would drive the car away and the man would be left swinging.

Elkins describes former high-level Nazis found dead in roadside ditches, apparently cut down by a hit-and-run driver. Others met their end in car accidents caused by mysterious mechanical failures. One senior Gestapo man was waiting in a hospital bed for a minor operation when somehow kerosene got into his bloodstream. Estimates vary as to how many Nazis died in what Elkins calls this "first hunting season", and the Avengers were understandably coy about naming specific names, but it seems as if dozens of Nazis were killed by Jewish vigilantes.

Not all of these assassins were Holocaust survivors. Cesarani believes that some of the most accomplished were members of the Jewish Brigade, the section of the British army comprised of Jewish volunteers from what was then Palestine, as well as the UK, South Africa and the rest of the empire. With access to military intelligence and transport, and with the right to travel freely across postwar Europe, "they were extremely efficient in tracking people down and conducting extrajudicial executions," he says. If little is known about them, that's because the men of the brigades were impeccably discreet. Says Cesarani, "They kept their mouths shut and took their secrets to their graves."

Forged In Fury makes clear that this effort endured long after the immediate aftermath of the war, stretching at least into the 50s. The Nokmim went to Spain, Latin America, Canada - any place where Nazi murderers had found a bolt hole. If ex-Nazis had Odessa, the old boys' network that helped them gain safe passage out of Germany and start new lives abroad, then the Avengers were Odessa's nemesis.

Elkins describes how, for example, the Nokmim tracked down to suburban Winnipeg, Canada, one Alexander Laak, responsible for the deaths of 100,000 Jews at the Estonian concentration camp of Jägala. They waited for Laak's wife to leave on a cinema trip, then confronted him with his crimes and their intended punishment - and let him do the decent thing and take his own life with a rope.

But the Avengers did not confine themselves to individual executions. Their largest operation was aimed at Stalag 13, a detention centre for former SS men in Nuremberg. The Vengeance group discovered that bread for the detainees was supplied by a single bakery. One of their younger members, Arye Distel, who had blue eyes and Aryan looks, got a job as a trainee baker, then worked up a plan to poison one morning's consignment of loaves.

Once they had got hold of the lethal fluid - which they codenamed "medication" - Distel smuggled it over several days into the bakery, stashing the bottles under the floorboards. Harmatz describes how, on a Saturday, at the change of shift, three extra comrades crept inside to join Distel, and together they brushed the poison on to some 3,000 loaves.

Did it work? Just check the New York Times for April 20 1946, where on page six you will find an Associated Press report that begins as follows: "Nineteen hundred German prisoners of war were poisoned by arsenic in their bread early this week in a United States camp and all are 'seriously ill', United States headquarters announced tonight." How many of those SS men actually died following the poisoning at Stalag 13 has never been verified, but some put the figure at several hundred, others at a thousand.

And yet that effort was a mere Plan B, a fall-back for a larger and more terrifying scheme. Kovner and the hardcore of the Vengeance group planned to inflict on Germany not the targeted killing of guilty men, but the same fate the Nazis had inflicted on the Jews: indiscriminate killing on a massive scale.

Their chosen method was the poisoning of the water supply of five German cities. Cohen says the targets were Munich, Berlin, Weimar, Nuremberg and Hamburg, and before long the Avengers had placed workers inside the cities' water filtration plants. Harmatz recalls days and nights studying blueprints of waterworks, working out how to avoid shutting off the supply to American residential areas, ensuring only Germans were struck. "Everything was in place, waiting for permission to proceed," he writes.

Kovner sought moral backing for his project, travelling to Palestine to consult the leaders of the Jewish state-in-waiting. He met Chaim Weizmann, who would become Israel's first president and who had begun his career (at Manchester University) as a research chemist. Once Weizmann heard from Kovner the horrors of the Holocaust, he could mount no resistance: he gave his blessing to the Avengers, even offering them help in acquiring the poison. (Several sources suggest Weizmann approved only Plan B, rather than the more deadly, and arbitrary, Plan A.)

Yet Plan A never happened. Kovner, with two canisters of poison in his backpack, was arrested by British military police while on a ship bound for Europe: he had been betrayed. The Avengers came to believe that the highest echelons of the Zionist hierarchy had heard of Plan A and were determined to thwart it, not least because they feared such a massive slaughter would fatally undermine the moral claim Zionism was making after the Holocaust - that as a people who had endured near-annihilation, the Jews had won the right to a home of their own.

Plan A, and indeed the Nokmim effort, raises vexed questions for Jews. Were those Avengers right to take the law into their own hands, to ensure that the guilty paid for their crimes and that future generations would know Jews could not be killed with impunity? Or were they resorting to a crude distortion of Old Testament justice - "An eye for an eye" - that let them sink to the level of their persecutors?

Yet, despite the weight of these questions and the books that have been written, the story of the Avengers has not yet become a central part of the Holocaust narrative. That may have disappointed Kovner, who died in 1987. One former comrade, Gabik Sedlis, told Cohen that the leader kept one eye on his place in history, hoping to be ranked alongside the ancient defenders of the Jewish people. "Two thousand years from now, he wanted people to talk about Judah Maccabee and Abba Kovner."

That goal has not yet been realised. But it might be, if at least the work of the Nokmim gets what those men and women wanted for themselves and their fellow Jews - if, in other words, future generations do their story justice.

· Jonathan Freedland's latest novel, based on the story of the Avengers, is The Final Reckoning, published next week by Harper Collins under the name Sam Bourne and priced £6.99.