We would like to thank a number of people for their valuable contribution to the development, coordination, and production of the WaSH Performance Index and this report: Catarina de Albuquerque, Pete Andrews, Robert Bain, Clarissa Brocklehurst, Edmund Cain, Elizabeth Christenson, Chris Dunston, Michael B. Fisher, Georgia Kayser, Peter Gleick, Katie Hall, Shaheen Kassim-Lakha, Pete Kolsky, Kaida Liang, Rolf Luyendijk, Ashley McKinney, Patrick Moriarty, Jan-Willem Rosenboom, Ron Ross, Darren Saywell, Katherine F. Shields, Heather Skilling, Marissa Streyle, and Yael Velleman.

Webcast The Water Institute recorded a special webcast from the Gillings School of Global Public Health, launching the WASH Performance Index. The webcast featured special guest speakers Catarina de Albuquerque, a world-renowned international human rights lawyer and advocate and the vice chair of Sanitation and Water for All (SWA); and Ed Cain, Vice President of grant programs for the Conrad N. Hilton Foundation. To view the recorded webcast online, please visit http://bit.ly/1D5TAPG .

This report describes the technical basis and rationale for the initial WASH Performance Index. It presents country performance and trends for the overall index and for each of its components –improving water access, water equity, sanitation access, and sanitation equity.

To answer the first question, we use the rate of change in coverage of improved water and sanitation. To answer the second, we use the rate of change of the gap in coverage of improved water and sanitation between rural and urban settings. To evaluate country progress, country rates of change are compared against best-in-class country performance at different levels of water and sanitation coverage.

Our proposed solution is the WASH Performance Index. The Index compares country performance to best-in-class performance. It enables fair comparison of countries at different stages of WASH development by comparing progress made by countries at similar levels of water and sanitation coverage. The Index is designed such that it can be calculated with already-available data and could be refined in the future to account for new nationally representative data on hygiene, water safety, improvements in service levels, and WASH in non-household settings for which data collection initiatives are anticipated.

Like the MDGs, proposed monitoring approaches to assess progress toward these targets and indicators focus on levels of coverage [5]. Important additional insights would be obtained by complementing coverage monitoring with performance and allow fair comparison of countries. There is a need for an instrument to measure country performance in improving access to and equity of WASH and measure progressive realization of the human rights to water and sanitation [6].

Deliberations are underway to develop targets and indicators for WASH as part of future Sustainable Development Goals (SDGs). The forthcoming SDGs provide potential for convergence of human development and human rights policy related to WASH. Proposals for targets and indicators for the SDGs call for expanded monitoring on hygiene, WASH in non-household settings (schools and health care facilities), universal access, and improvements in service levels [5].

Water and sanitation are recognized as human rights. The principle of progressive realization of human rights requires that each government takes steps “to the maximum of its available resources, with a view to achieving progressively the full realization of the rights” [4]. The UN General Assembly’s 2010 Resolution on the Human Right to Water and Sanitation calls upon governments “to scale up efforts to provide safe, clean, accessible and affordable drinking water and sanitation for all” [4].

Water, sanitation and hygiene (WASH) are essential to human health and development [1]. Water and sanitation feature prominently in human development policy, most recently the Millennium Development Goals (MDGs) [2, 3]. Monitoring of WASH in the MDGs focused on coverage of water and sanitation in countries.

Methods

We consulted WASH stakeholders and experts to design the WASH Performance Index.

The Index assesses country performance in water access, water equity, sanitation access, and sanitation equity.

Country data compiled by the WHO/UNICEF Joint Monitoring Programme (JMP) were used to calculate rates of change for each component.

Frontier analysis was used to identify best-in-class performance at different levels of water and sanitation coverage.

The most recent rate of change from each country was compared to best-in-class performance among countries at similar levels of water and sanitation coverage to generate a benchmarked value, enabling fair comparison.

The index value is the sum of the component benchmarked values.

Country trends were calculated for each of the components.

We examined associations between component values and country characteristics (e.g. GDP per capita, percent urban, world region as defined by the World Bank) and governance indicators (e.g. government effectiveness, control of corruption, regulatory quality, rule of law).

Consultation

We consulted experts and stakeholders in designing the Index and incorporated feedback from a series of events: a think tank at the University of North Carolina at Chapel Hill (UNC) in March 2014, a workshop at the September 2014 Stockholm International Water Week, and a workshop at the October 2014 UNC Water and Health Conference. At the UNC think tank, nine participants representing WASH stakeholders from multilateral organizations, donors, NGOs and academia recommended that the Index should be simple for clarity of communication and focus on country-level water and sanitation coverage. In preliminary versions of the Index, we considered including input data, such as data on governance from sources such as the UN-Water Global Analysis and Assessment of Sanitation and Drinking-Water (GLAAS) report [7]. Think tank participants recommended separating these two types of data to enable comparison. In Stockholm over 30 WASH stakeholders and experts suggested the Index use already-available data and be able designed so as to be able to incorporate future data. At the UNC Water and Health Conference workshop, eight participants suggested the Index align with the indicators and data sources for the Sustainable Development Goals therefore making it ‘future proof’.

Index components

The WASH Performance Index is the sum of country performance values for the following components: water access, water equity, sanitation access, and sanitation equity. The access components use the rate of change of access to an improved water source or sanitation facility, where access is the proportion of the population using improved water or sanitation (i.e. coverage). The equity components use the rate of change of the gap in coverage between rural and urban settings. Countries have a decreasing gap between rural and urban coverage (i.e. increasing equity) or an increasing gap between rural and urban coverage (i.e. decreasing equity) (Figure 1).



Figure 1. Examples of increasing and decreasing equity in access to water and sanitation between rural and urban areas

Frontier Analysis

A country approaching 100% coverage can only improve water and sanitation slowly, while countries at intermediate levels of coverage can often increase coverage more rapidly. This is also seen with other technologies, such as the uptake of mobile phones [8]. When levels of coverage are compared with rates of change, we tend to see rates increasing at low levels of coverage, plateau at intermediate levels of coverage, and slow as they approach 100% coverage. This is illustrated in Figure 2, in which the best-performing countries represent a performance frontier at which best-in-class performance is demonstrated and against which countries at the same level of coverage can be compared.

Frontier analysis, a technique used to study efficiency or best-in-class performance, enables identification of top performing countries. It has been used to measure performance of “decision making units” such as schools, factories, and hospitals and has been applied to measure human rights realization [9, 10].



Figure 2. Country performance on WASH using frontier analysis to describe the performance frontier at which best-in-class performance is achieved

Data sources

Data on improved water sources and improved sanitation facilities were obtained from the WHO/UNICEF Joint Monitoring Programme (JMP). The JMP, the agency charged with international monitoring of drinking water and sanitation, categorizes a drinking-water source as improved if “by nature of its construction or through active intervention, [it] is protected from outside contamination, in particular from contamination with faecal matter”. An “improved” sanitation facility is, “one that hygienically separates human excreta from human contact” [11].

National level data on the proportion of the population with access to improved water and sanitation (coverage points) were obtained from JMP Country Files. The JMP coverage points are compiled from nationally representative sources including Demographic and Health Surveys (DHS), Multiple Indicator Cluster Survey (MICS), World Health Surveys (WHS), and national censuses [12].

Calculation of rates of change

We calculated the line of best fit between each series of three consecutive coverage points for each country (i.e., a three-point moving average). Countries can have more than one rate if they have four or more coverage points – each corresponding to a different time in their development. When there were multiple coverage points from the same year for a country, data for that year were averaged to generate one coverage point. The country rate of change is the slope of the best-fit line. A three-point moving average was selected rather than the slope of all available coverage points to capture change in rate over time. The 2015 Index is based on the most recent three coverage points for each country. This process was performed for each of the components. Data from 212 countries and territories were reviewed from the latest JMP update in 2014 [13]. Country data were excluded if countries achieved 100% coverage or if there were insufficient coverage points (less than three). We were able to calculate rates for 138 countries for water access, 129 for water equity, 133 for sanitation access, and 126 for sanitation equity. An Index value was calculated for each country if values from all four components were available. We were able to calculate Index values for 117 countries.

Calculating the performance frontier and identifying best-in-class performance

We conducted frontier analysis using the FEAR software package in R version i386 3.1.1 [14]. We followed frontier analysis best practice and used the software to identify outliers which were removed when defining the performance frontier [15, 16]. The software used the rates of change from all countries to identify performance frontier points, each representing best-in-class performance.

We used Microsoft Excel® to generate a straight line between the performance frontier points to define best-in-class performance values at any level of coverage. Since countries can no longer improve once they reach 100% coverage or achieve equity, the line defining the performance frontier ended at 100% coverage and 0% rate of change.

Rates of change from years that coincided with or followed within three years of a country’s involvement in an armed conflict (with more than 1000 deaths) were not used in defining the performance frontier. To identify conflict states, we consulted the UCDP/PRIO Armed Conflict Dataset [17]. Conflict states were not assigned a rank for these years, but for reference, they were compared against best-in-class performance. Conflict states are denoted at the bottom of the ranking lists and in the annex (overall index and for each component) with asterisks.

Comparing best-in-class performance between countries

To generate a value for each component that enables country comparison we used the following equation:

Country component value = (country rate) ÷ (best in class performance rate)

This compares country rates to best-in-class performance and generates a value between -1 and 1, enabling fair comparison between countries. Values between 0 and 1 represent progress while values between 0 and -1 represent regression. Values of 1 lie along the performance frontier, reflecting best-in-class performance. Outliers were manually assigned a value of either 1 or -1. We repeated this process for each country and each of the components.

The most recent country component values were used in calculating the 2015 WASH Performance Index.

Trends in performance

While we report the most recent component values for each country, these values change over time. Trends in these values show whether country performance is improving or deteriorating. To examine trends in performance, we calculated the slope of all available values from each country for each component. For all countries where slopes could be calculated, we created three equal groups: either improving, unchanged, or deteriorating. Countries with only one value for any given component were not categorized (listed as “N/A” in tables). Countries with values that tracked along the performance frontier for all available values (i.e. maintaining best-in-class performance over time) were not categorized. Trends were further grouped based on their most recent component value in the 2015 Index. Countries with positive values in the 2015 index (with values between 0 and 1) were grouped and countries with negative values (with values between -1 and 0) were grouped. Trends should be interpreted alongside component values because, for example, a country might have a positive trend but negative component value or vice versa.

Assembling the Index

We followed index best practice and considered each component of the WaSH Performance Index as an equal-weighted value since we have no justification to weight any component more than another [18]. We use the latest country values for each component for the overall Index. For each country, the WaSH Performance Index is the sum of best-in-class performance for water access, water equity, sanitation access, and sanitation equity. Countries without all four components were not ranked.

Correlation between components and country indicators

We assessed correlations between component values and country indicators to explore potential underlying drivers of performance (Table 1). Country characteristics and governance indicators, representing the enabling environment, from publicly available data sets were used [19, 20]. The enabling environment is “a favorable culture of internal coordination and communication; policy and institutional behavior that guides behavior of water and sanitation service providers with clear and enforceable service standards, and resources to provide effective water and sanitation services” [21].

We used the latest available data for each indicator. To assess correlation between components and country indicators, we conducted univariable linear regression analyses for each of the components.

Table 1. Country characteristic and governance indicators

INDICATOR INDICATOR DESCRIPTION Source: World Development Indicators (2013) [19] Gross domestic product (GDP) per capita (in 2013 USD) GDP per capita reflects the amount of resources available for investment (in 2013 United States Dollars). Gross National Income (GNI) per capita GNI is defined as “the sum of value added by all producers who are residents in a nation, plus any product taxes (minus subsidies) not included in output, plus income received from abroad such as employee compensation and property income.” Under-five mortality rate Under-5 mortality rate is defined as “the probability per 1,000 that a newborn baby will die before reaching age five.” Primary education Primary education is defined as the number of primary education years completed by the population. Urban population (% of total) “Urban population refers to people living in urban areas as defined by national statistical offices. It is calculated using World Bank population estimates and urban ratios from the United Nations World Urbanization Prospects.” World region World region as classified by the World Bank. Regions are Africa, East Asia and Pacific, Europe and Central Asia, Latin America and Caribbean, Middle East and North Africa, and South Asia. World Bank income classification Classification defined by GNI per capita in 2013. Classifications are: Low-income (less than $1,045), middle-income ($1,045 to 12,746), and high-income ($12,746 or more). Lower-middle-income and upper-middle income economies are separated at a GNI per capita of $4,125. Source: Worldwide Governance Indicators (2014) [20] Control of corruption Control of corruption “captures perceptions of the extent to which public power is exercised for private gain, including both petty and grand forms of corruption, as well as capture of the state by elites and private interests.” Voice and accountability Voice and accountability “captures perceptions of the extent to which a country’s citizens are able to participate in selecting their government, as well as freedom of expression, freedom of association, and a free media.” Political stability and absence of violence Political stability and absence of violence “measures perceptions of the likelihood of political instability and/or politically motivated violence, including terrorism.” Governance effectiveness Government Effectiveness (GE) reflects government commitment and effectiveness in implementing programs. Regulatory quality Regulatory quality “captures perceptions of the ability of the government to formulate and implement sound policies and regulations that permit and promote private sector development.” Rule of law Rule of law “captures perceptions of the extent to which agents have confidence in and abide by the rules of society, and in particular the quality of contract enforcement, property rights, the police, and the courts, as well as the likelihood of crime and violence.”

Future development of the Index

Data for other proposed SDG targets and indicators such as hygiene, improvements in service levels, water safety, and WaSH in non-household settings are not yet available. Such data will likely be collected and become available within five years [12, 22]. A minimum of three different years of data for these new aspects must be collected so that rates of change can be calculated for the WaSH Performance Index. Data for an indicator can only be included when they are available for a sufficient number of countries. Our initial choice of indicators was influenced by data availability in order to maximize the number of countries we were able to assess. We will also assess whether other indicators of equity, such as the relative levels of access among wealth quintiles, might be added to the Index.

We will update the index and rankings in response to new insights and as new data become available (e.g. from JMP). According to the 2014 JMP report, 106 data sets from 63 countries were added, indicating potential for meaningful updates in the WaSH Performance Index rankings [12].