A Canadian project to encourage women in Afghanistan to run for political office was poorly designed and executed, according to a newly released evaluation.

The $5.6 million project "had limited impact in changing women's capacity to influence decision-making processes," concludes the report, which suggests too little of the money went to the women it was supposed to help.

The initiative, Promoting Women's Political Participation in Afghanistan (PWPPA), was approved by the Conservative government of Stephen Harper in 2013, and delivered between May 2014 and July 2016.

Canadian Armed Forces Capt. Ashley Collette poses for a photo with children during a patrol in Afghanistan. After the Canadian military left Afghanistan in 2011, Ottawa approved a $5.6M project to help Afghan women get into political office. (Capt. Ashley Collette/Associated Press)

The project followed Canada's 2011 withdrawal of combat troops from Afghanistan and the March 2014 end of its military mission to train Afghan soldiers. The women's initiative signalled Canada's continuing civil support for the troubled country.

Global Affairs Canada contracted the project to the National Democratic Institute, (NDI) based in Washington, D.C., which carried out most of the fieldwork in difficult conditions as security deteriorated.

Global Affairs hired a consultant group based in Milan, Italy, last October – more than a year after the project ended – to determine whether the money had been well spent.

Hobbled from start

The $114,000 evaluation, delivered in April 2018 and recently released to CBC News after a request under the Access to Information Act, suggests the project was hobbled from the beginning.

"[T]he design of the project shows several weaknesses, including the lack of reflection on some crucial elements such as structural and long-term gender discrimination," said the report by Lattanzio and Seefar.

It also cites "a limited engagement of men in the project, a flawed implementation of the risks and mitigation strategies, and limited attention to the lack of political background and professional experience of the women."

The consultants also found most of the money went to the salaries and benefits of the NDI workers and consultants delivering the project, with little committed to field activities directly benefitting Afghan women — a situation the report calls "disproportionate."

The project had called for further training of women who managed to get elected to the 34 provincial councils in Afghanistan. "However, these follow-ups were considered insufficient," said the report.

"On more than one occasion, the participants involved in the evaluation complained about the brevity of the project and the lack of adequate follow-up."

The project had also planned to help prepare women candidates for the 2015 national parliamentary elections, but those elections were delayed and still have not happened. They're now set for October this year.

Substantive analysis is hardly ever done. - Nipa Banerjee , professor of international development, on the lack of proper program evaluation at Global Affairs Canada

Fresh security concerns in the unstable country also hurt the project between 2015 and 2016. NDI had to close offices temporarily in Kabul and Kandahar, and permanently in Kunduz, and road travel between provinces was restricted because of security threats.

At the time, Global Affairs Canada considered whether to cancel the project, given these and other significant challenges on the ground, but decided to press ahead.

"In spite of these challenges, Global Affairs Canada believed that cancelling the project would adversely affect women's political participation in Afghanistan," said spokesperson Krista Humick.

The department internally rated the success of the project a 3.5 out of five, which means the goals were "mostly met," she said.

"Despite the uncertain election calendar and re-profiling of project activities, NDI contributed to enhancing the capacity of women candidates running in the PC (provincial council) elections, enhancing their campaign skills and knowledge of electoral processes and regulations," Humick said in an email.

A spokesperson for the non-profit NDI, Raissa Tatad, declined to provide on-the-record comment, saying the group had already provided input to Global Affairs Canada's response to CBC News' questions.

Not as rigorous

Canada has committed more than $2 billion in aid to Afghanistan, and in 2015 published an overall evaluation of programs covering 2004—2005 to 2012—2013.

But Ottawa's assessments of achievements has not been as rigorous as in the United States, where in 2008 Congress established a Special Inspector General for Afghanistan Reconstruction, or SIGAR, to conduct independent, hard-nosed audits of aid programs which are then made public.

Nipa Banerjee, a senior fellow of international development at the University of Ottawa, said the evaluation reflects much of her 35 years' experience in foreign-aid work, including years spent in Afghanistan.

"There is a very serious design problem" in the project, she said in an interview, which lacks "proper indicators with which one could assess if the desired results have been achieved. That is the major problem."

Nipa Banerjee, senior fellow of international development at the University of Ottawa, said Canada needs an independent office to properly evaluate the success of aid programs in Afghanistan. (uOttawa.ca)

Banerjee also noted the finding that the project did not take into account the special cultural circumstances of Afghanistan, such as the need to persuade Afghan men to support its goals and methods.

Banerjee said her current academic work in assessing the success of Canada's aid projects has been hampered because desired outcomes are not clearly articulated, and independent outcome-measuring assessments are rare.

"Monitoring is hardly done," she said. "Substantive analysis is hardly ever done."

Banerjee said Canada needs to create an entity like SIGAR to provide Canadians with independent accountability for foreign aid programs and spending.

Follow @DeanBeeby on Twitter