The families of thousands of civilians killed by US forces in Afghanistan have been left without justice or compensation, Amnesty International has said in a damning indictment of the US military as it withdraws from the country.

In a report published on Monday, Amnesty said it had gathered evidence of "a deeply flawed US military justice system that cements a culture of impunity" in dealing with Afghan civilian deaths and injuries caused by US-led Nato operations since 2001.

President Hamid Karzai has often castigated US forces for civilian casualties. Nato's International Security Assistance Force (Isaf) says it takes all allegations of civilian casualties seriously and investigates each alleged incident.

Amnesty researchers interviewed 125 Afghans who had first-hand information on 16 attacks that resulted in civilian casualties, as well as collating data from 97 reported incidents since 2007.

An Amnesty spokesman said its data that thousands of civilians had been killed by US forces was based on UN reports on civilian casualties, a Science magazine investigation in 2011 and other sources, but it gave no total death toll.

"After any incident in which civilians have been killed by US forces, (the US must) ensure … wherever there is sufficient admissible evidence, suspects are prosecuted," said the Amnesty report entitled Left in the Dark.

It detailed a US bombing in 2012 when women were collecting firewood in the mountains of Laghman province. Seven women and girls were killed and seven more injured.

Ghulam Noor, who lost his 16-year-old daughter Bibi Halimi in the attack, brought the bodies to the district centre after hearing Nato forces claimed that only insurgents had been killed.

Afghan relatives look on during a press conference in Kabul. Photograph: Wakil Kohsar/AFP/Getty Images

"We had to show them that it was women," Noor told Amnesty. "I have no power to ask the international forces why they did this. I can't bring them to court."

Amnesty said villagers filed complaints with the provincial governor, but international forces are immune from Afghan legal processes and no one ever contacted family members to investigate the attack.

Karzai, who will step down when his successor is chosen after an ongoing dispute about election results, welcomed the report after inviting Amnesty representatives to the presidential palace on Sunday.

"I'm very happy that you have focused on something that is the main point of disagreement between Afghanistan and the US," he said, according to a palace statement. "I believe that civilian casualties should never happen. Together with you, we should stop them."

Amnesty emphasised in its report that the "vast majority" of civilian deaths are caused by the Taliban and other armed groups in Afghanistan, which remains in the grip of a fierce Islamist insurgency. It said its report concentrated on the US rather than other members of the Nato coalition since it was the largest national force and was implicated in the majority of civilian casualties.

"Amnesty International is aware of only six cases over the last five years in which members of the military have been criminally prosecuted for unlawfully killing Afghan civilians," it said.

In the most high-profile killing, the US army sergeant Robert Bales was sentenced to life in prison after gunning down 16 villagers in 2012.

US-led foreign troop numbers in Afghanistan have declined from a peak of 150,000 in 2012 to 44,300 now – of whom 30,700 are American.

All Nato combat soldiers will depart by the end of the year, though a follow-up support mission of about 10,000 troops is planned if the next president signs security deals with the US and Nato.

The deal with the US would continue to give so-called "immunity" to US troops, who would be prosecuted under their own legal system.

The Isaf press office in Kabul referred enquiries about the report to the US Department of Defense, which was not immediately available to comment.