The war crimes trial of two sons of Libya's former dictator, Muammar Gaddafi, begins amid tight security in Tripoli on Monday, in a case causing sensation at home and controversy among rights groups.

Saif al-Islam Gaddafi and his younger brother, Saadi, are accused of orchestrating a campaign of murder, torture and bombardment of civilians during Libya's eight-month civil war in 2011.

Appearing with them are Gaddafi's former spy chief, Abdullah al-Senussi, two former prime ministers and 34 senior officials: much of the dictatorship's surviving elite.

The trial is going ahead despite much of the country being gripped by violence and the blockading of oil production by rebel militias. Libya's interim prime minister Abdullah al-Thinni resigned on Sunday saying he had faced threats and could not continue, just weeks after he was appointed to the post.

The prime minister said he resigned because of a cowardly attack on his family that had made his position untenable, underlining the chaos of Libya and its domination by militias. It is unclear how congress, opposed by regions in the east and west of the country, can find enough MPs to elect a new prime minister, with only 76 of 200 attending and elections unlikely before August.

Security fears mean that the Gadaffi trial has moved to the capital's maximum security al-Hadba prison, which has been ringed with armoured cars, barbed wire and machine-gun nests.

In a sign of the power of the militias, units holding Saif al-Islam in the mountain town of Zintan have refused to hand him over to the authorities. Instead, he will stand trial over a video link.

The defendants are accused of masterminding a chain of massacres in the early days of Libya's revolution, and of later rounding up, torturing and killing hundreds of opponents. The two brothers are also accused of plundering state coffers to fund extravagant playboy lifestyles abroad.

Until the revolution, Saif al-Islam enjoyed the high life, being entertained at Buckingham Palace and enjoying links with Prince Andrew and Tony Blair.

Saadi, 40, is best known for his failed attempts to become a professional footballer, being signed by three Italian Serie-A clubs in quick succession, but managing just two appearances in three years. Prosecutors say he was responsible for troops firing into the crowd during a Libyan cup final in 1996.

Investigators have released few details of the case, but documents filed by Libya with the international criminal court show an extensive charge sheet, more than 200 witnesses and 40,000 pages of evidence.

Central to the case are telephone intercepts allegedly recording the accused ordering war crimes, and a video that judges in The Hague say is genuine, showing Senussi ordering his followers "to be ready to destroy these filthy groups altogether".

But rights groups say violence against judges and lawyers, which have seen the rule of law suspended across much of the country, leave a question mark over the trial. "Militias and criminals have harassed, intimidated, threatened and in some cases assassinated judges, prosecutors, witnesses," said Hanan Salah, of Human Rights Watch.

Concern about due process was heightened this month with the release of a video made by prison guards, apparently without a lawyer present, showing Saadi supposedly confessing to his crimes.

The trial is also controversial because the international criminal court, which has charged Saif al-Islam and Senussi, has yet to agree to Libyan requests to try both at home.

Judges at The Hague ruled that Saif al-Islam must be sent to the ICC, and his lawyers are appealing against a decision that Libya is fit to try Senussi, 64. His ICC legal team, led by Ben Emmerson QC, say Libya refused them permission to visit their client.

"How can you have a fair trial if they don't let the lawyers visit Senussi or even speak to him by phone?" said Amal Alamuddin, one of the ICC defence team.

Libya insists the trial will be fair and transparent. "I can assure you that the trial will be according to the correct legal procedures," said Ahmed Lamin, a government spokesman.

And Libyans themselves are eager to see the trial of their former tormentors. "This case is going to be an example, not just for the old guys on trial, but for the new guys in power," said a Tripoli photographer Magdi el-Nakua. "The message for the new guys is that any abuses, no matter how you see yourselves, you will be made to account for them."