Three months after the killing of Muammar Gaddafi, concerns are mounting about the mistreatment and torture of prisoners held by Libyan militiamen who are operating beyond the control of the country's transitional government, as well as by officially recognised security bodies.

Amnesty International warned that prisoners from Libya and other African countries have been subject to abuse. The warning comes against a background of anxiety in western capitals about Tripoli's failure to tackle security and political issues.

This week's fighting in Bani Walid, a former stronghold of the Gaddafi regime to the south of the capital, has fuelled fears that tribal rivalries and armed clashes could explode into a wider conflict. Last week, the president of the National Transitional Council, Mustafa Abdel-Jalil, was mobbed by demonstrators in his Benghazi office.

Ian Martin, the UN's special envoy to Libya, told the security council on Wednesday that the Bani Walid fighting did not indicate a resurgence of pro-Gaddafi sentiment, but added this warning: "The former regime may have been toppled, but the harsh reality is that the Libyan people continue to have to live with its deep-rooted legacy."

Navi Pillay, the UN human rights chief, said that more than 8,500 detainees were being held by militia groups in about 60 centres.

The aid agency Médecins Sans Frontières has added its voice to the chorus of concern by announcing that it had halted work in the coastal city of Misrata because staff were being asked to patch up detainees during torture sessions. "Patients were brought to us in the middle of interrogation for medical care, in order to make them fit for more interrogation," said MSF's Christopher Stokes. "This is unacceptable. Our role is to provide medical care to war casualties and sick detainees, not to repeatedly treat the same patients between torture sessions."

Amnesty said its delegates in Libya had met detainees in and around Tripoli, Misrata and Gheryan who showed marks indicating they had recently been tortured. Injuries included open wounds on the head, limbs, back and elsewhere.

Donatella Rovera of Amnesty said: "After all the promises to get detention centres under control, it is horrifying to find that there has been no progress to stop the use of torture. We are not aware of any proper investigations into cases of torture, and neither the survivors nor relatives of those who have died in detention have had any recourse to justice or redress for what they have suffered." Some detainees were scared to speak, fearing harsher torture.

Allegations of human rights abuses are especially embarrassing to Britain and other governments which took part in the Nato air campaign in Libya, which was mandated by the UN to protect civilians and backed by the Arab League, but strongly criticised by Russia and China as "regime change by stealth".

"The MSF report is shocking and the Libyan authorities should thoroughly investigate the claims," said a spokesman for the UK Foreign Office. "We condemn all human rights abuses and have repeatedly made clear that the transitional government must live up to the standards that it has set for itself and make a clean break with the past."

Libyan and foreign experts highlight the difficulties of disarming and integrating militia units while there has been so little progress at the political level. Tripoli residents say the militiamen are now less visible than before, but still prone to turf wars that often spin out of control.

Peter Cole of the International Crisis Group said: "The NTC and government have started distributing application forms to the militias encouraging them to join the army and security services, but they are undermined by excluding the militias from the political process and dialogue. They are clear that they will keep their weapons and run their own affairs until such a time as they judge the government to be performing well. This will make it much harder for the government and international community to clamp down on things like prisoner abuse, which is an issue that will fester over time and make social integration that much harder."

Analysts worry about a tight political timetable that requires a new electoral law and elections by the summer, as well as about rising resentment against the NTC and its failure to purge Gaddafi-era officials – so called "opportunists".

"We are at a critical juncture," said Tripoli businessman Muhannad Alamir. "The economy is improving and things are getting better. But they could also get worse. The Bani Walid issue blindsided us. It was exaggerated by the media and underplayed by the NTC."

Human rights abuses were due in part to the lack of proper command structures.

"We hit rock bottom under Gaddafi so anything is an upgrade," he added. "You can't undo 42 years in three months."