Comparing water point based and household survey based water access estimates with publicly available data

I am excited to share with WPDx a preprint of a study comparing different ways of measuring access to water services. . It builds on the work of the jmpwashdata data science library that I started last year to enable researchers to use JMP WASH data like never before .

The purpose of the study is to determine how water point level estimates for rural basic water service coverage from the Water Point Data Exchange (WPdx) compare to the household level estimates from the Joint Monitoring Programme of WHO/UNICEF (JMP) in multiple geographies. The study describes how these different estimates are produced and proposes the comparison of JMP basic minus services on premises to WPdx basic access. WPdx basic access estimates the population with 1km of an improved water point. Comparing between metrics and triangulating different measured results can be useful to validate conclusions and inform decision making. This study finds a relatively strong correlation and linear trend between these two estimates in four countries that suggests that using household surveys and water point inventories together can be useful to decision makers who may only have one or the other data sources or may want to validate the conclusions from one against another. The WPdx basic estimates allow a more granular geographical level of access estimates that can be useful to districts and enable national vulnerability assessments. This could strengthen the type of analysis provided in JMP inequality charts showing the differences between country regions. At the same time, further research is needed to validate these trends at these lower geographical levels. Rural water leaders, including national and local governments, development partners, service providers and civil society should continue to advocate for the publication of water point data and the validation of access estimates on the basis of publicly available information. This plays an important role in improving the quality of both public and private data sets and analyses used by researchers and decision makers.

Read more about the study in the WPDx blog.

Acknowledgements

This study would not have been possible without the contribution of open data on water points by data providers to WPdx. Members of the Water Point Data Exchange (WPdx) working group reviewed both the proposal and findings of this work. Katy Sill of WPdx first recognized the potential of the work, provided invaluable feedback, and responded quickly with explanations about how the WPdx algorithms work while investigating and delivering improvements to the tools when required to make this comparison possible.

Similarly, the National Statistics Offices (NSOs) and the Demographic and Health Surveys (DHS) Program of the United States Agency for International Development (USAID) made it possible to use household survey data from different countries. I would like to thank the Joint Monitoring Programme of WHO/UNICEF (JMP) team for sharing country, regional and global estimates of progress on drinking water, sanitation and hygiene (WASH) in households as well as the estimates for the sub-indicators required to generate those estimates, for providing clarifications about the JMP methodology, and for taking time to reflect on study findings.

This material is based upon work supported by USAID under award number 7200AA18CA00033.

Are national WASH M&E system important for accountability and transparency?

When policy and the roles and responsibilities of service providers and government are unclear, it is difficult for users
and their representatives in civil society to advocate and tackle issues of accountability. Who is ultimately responsible for ensuring basic and safely managed WASH services? When these are clear with clear contractual arrangements, are there also agreed ways to measure and evaluate service performance? Who is measuring and communicating results? It is crucial to hold parties accountable for their actions and promises for this life sustaining services.

Sometimes civil society may fill the evidence gap with their own metrics. But government and service providers may choose to ignore the results on the basis that they are not ‘official’, ‘approved’ or even deemed to be ‘wrong’.

Establishing an information system is complex  and it requires clarifying a matrix of roles and responsibilities so that it is clear who is reporting what and why. Using and communicating results is an important step in holding sector stakeholders accountable .

Establishing a national WASH M&E system in a participatory approach is one way to develop consensus and ensure parties are all working towards the same goals to sustain services. Indeed, for change and improvement at all levels, there is a need to engage in dialogue around sector performance in WASH service delivery.

WASH Web Registry

Have you every wondered who is providing water or sanitation services or some other public service in a particular place or how these public services are regulated? In much of the world, it’s not easy to find out. Perhaps you’ve googled, gone to the useful PSEAU database, spoken with colleagues, or perhaps you visited the local government office. Either way, this is largely a manual affair restricting access to those with time, money, and interest. This public information is not available in one very accessible place and is often inadequate as it does not link to information about roles and responsibilities. Many NGOs and charities do not bother to go to the local government office before installing a hand pump in a community. As result, it is not always clear who is doing what.

For many larger projects, consultants are hired to collect and validate this enabling environment information. Time and money that could be spent analyzing sector/market data to identify strategies to improve WASH services if this information were available and peer-reviewed.

Even at national level public information, such as “Water Supply and Sanitation in [Country X]” pages on Wikipedia are often out of date despite efforts from Susana and others to keep them up to date and relevant. I wrote last year about my continued conviction for a backbone for WASH service data and here is what I am doing about it now.

At WASHNote we have three strategies to break out of organizational and product silos and develop the WASH Web. Soon we will be inviting partners to review our work on #3 “A WASH registry” of peer-reviewed source of information on WASH organizations and what they do. We are taking the existing public but hard to use information, making it available on Wikidata so that it is possible to edit and peer review and critically ensure it is updated on Wikipedia. Wikipedia is an authoritative source of information on many topics and supported widely. It provides a solid and mature platform on which to improve public domain information on WASH.

We are taking action to do this with (1) the open source & Wikipedia community and (2) the WASH communities to make this happen. We need to work together for the following reasons:

(1) Over time, we expect that datasets on WASH services link back to more contextual information about those services, developments in each area and enable the development of a real ecosystem for high quality WASH data. We need help from very experienced wiki and open source communities to make this happen and ensure a WASH data ecosystem.

(2) Data quality is a huge issue. WASH sector professionals, you are the authority for information about your partner organizations, former employers, or your local service providers. We will need people to start update public information and help link different data sources together.

At the end of all this, we would like to see an open WASH Web, which we believe is crucial to be effective to achieve the grand ambitions of SDG6. We envision a time when anyone can see all the WASH activities in a particular service area logged by date, time, and who did what. The basic information that is required to take action to improve service provision.

The WASH Web registry is a start help us know who is doing what. Interested in more information, please contact us about how to get involved.

You can find our Github repository here: https://github.com/WASHNote/WASHWeb/