WASHINGTON — World Bank managers are failing to properly implement, and in some cases ignoring, the recommendations of its own independent watchdog, according to evaluators.

In a report presented Thursday, the Independent Evaluation Group, the body tasked with evaluating the performance of all parts of the World Bank and making recommendations on how it can improve, accused bank management of failing to act on most of its advice.

While the bank is not obliged to take up IEG’s recommendations, it does develop Management Action Plans outlining the steps it will take to address the issues raised. Progress against these plans is measured through a dedicated tool launched in 2012, which includes progress ratings on the actions taken, ranging from “complete” to “negligible.”

Q&A: Summing up why evaluation matters When it comes to any line of work, substantial evaluation of the systems, processes, and outcomes is vital to ensure improved performance. In the global development sector, that need for evaluation is no different. Speaking to Devex, IEG’s Caroline Heider explains how evaluation can contribute to better understanding of development.

According to IEG’s review, which looked at four years’ worth of evaluations resulting in 147 recommendations up to 2016, only 20 percent of the bank’s follow-up actions were classed as “complete,” and only 52 percent were marked as “high or complete.”

More than 90 percent of actions were rated as “substantial or above,” which World Bank chief executive Kristalina Georgieva argued should be taken as a positive finding.

The report pointed to a mismatch between IEG expectations of how their recommendations would be implemented and the reality within the bank. It stated that “Management Action Plans are not always aligned with the intent of IEG's recommendations,” and that “action plan items have often been unambitious and insufficient to achieve the outcome envisioned in the recommendation.” Both IEG and World Bank managers surveyed as part of the report said tracking the implementation of IEG recommendations was a “box-ticking exercise,” and not something designed to bring about improvement.

Looking specifically at action items coming out of IEG environmental evaluations, the report found that workshops or further training were marked as the action item in five cases, but that in four of those cases it never took place. In the one case where a workshop was held, its effect was unclear. In 10 cases, new tools or guidance were set to be created and while these were “largely achieved,” they “saw limited use by operations,” according to the report.

Speaking at the launch event for the report, IEG Director General Caroline Heider said there is “a lot more that needs to be done” in order to ensure the bank is getting the “most out of independent evaluation and maximizing the learning that can be gained from our work.”

She stressed that both the bank and IEG need to “work together on this” and said the watchdog had taken steps to become less “isolated.” She added: “It does take two to tango and we would welcome a more engaging stance from the leadership and the management” of the bank.

However, Georgieva disagreed with the report’s interpretation of the findings: “I tend to slightly disagree with the observation you are making that we don’t take recommendations seriously,” she told Heider. She added, about the fact that 90 percent of actions were rated “substantial or above,” that “you choose to take ‘substantial’ as negative. I’m not sure I agree.”

Furthermore, while Georgieva emphasized the value of IEG’s work, which she described as crucial to making the bank “do better,” she faulted the evaluators for producing too many recommendations, which she said the bank could not possibly hope to implement. She called on IEG and the bank to work together to prioritize the most significant recommendations, noting that “not all recommendations are born equal.”

Heider hit back, saying “it is the number of [bank] actions which is so large, not so much the number of recommendations.” But she agreed that prioritizing and being more “strategic” about which to pursue would “move us all into a much better place.”

In a response included in the report, the World Bank agreed that the management action plan process “could be improved to enhance its focus and effectiveness.” This includes making it more adaptive and less rigid, and being more selective about which recommendations to pursue.

The bank and IEG have already started working on new ways of improving the bank’s engagement around major evaluations, such as by introducing “adaptable action plans” that can be updated as projects develop, the report stated. IEG is also piloting incorporating World Bank feedback into its recommendations while in draft stage. Such reforms could help “shorten the feedback loop and give more real-time information from an independent point of view,” Heider said, while being clear that it would not undermine her department’s independence or lead to bank managers dictating recommendations.

IEG was set up in 1973 and produces about 60 evaluations every year, including an annual results and performance report, and smaller, project-level assessments.

Ultimately, it is up to the bank’s executive board, to which IEG reports, to hold management accountable for effectively implementing the watchdog’s recommendations. This was confirmed by Otaviano Canuto, executive director and chair of the board’s committee on development effectiveness, who said the board would “keep its eyes open” when it comes to ensuring action plans are implemented, but acknowledged that it cannot follow the smaller details.