WASH Learning Companion

We are excited to be developing and testing together with IRCWASH WASH Systems Academy, an AI Learning Companion to help onboard and support WASH professionals to learn about WASH systems and the building blocks. IRCWASH and WASHNote signed a Memorandum of Understanding at the start of 2024 to make this a reality.

It is built from two components:

  1. The Knowledge Companion, that provides a validated set of information and a model of the conceptual framework to be used.
  2. The Learning Companion, a conversational agent that can chat with users via mobile apps such as Telegram, Whatsapp, Signal or Element.

The Learning Companion document  and data repository focuses on WASH systems and their building blocks, and the conceptual framework of the WASH Systems Academy courses (IRC building blocks).

These components are easily extensible and it is possible to switch conceptual frameworks (knowledge graphs) on the fly to be able to map, for example, results from one data source to another methodology. Ultimately, these will be used to help in monitoring and evaluation and to support the development of validated public datasets.

So far, together with a small team, we’ve built a knowledge graph of the key concepts used, a document repository that can be easily cited by the learning companion and a Telegram bot. Much of our effort has been used to build up our user journeys, validate with existing users and develop a evaluation framework for both improving conversations and evaluating the outcomes of the exchanges.

We’ve also researched how to run this AI product in an affordable manner so that it can scale and achieve an impact on WASH services, which is the ultimate goal. Some of this research will be published shortly.

In the coming weeks, we will be testing with users in Rwanda and Ethiopia and we look forward to sharing our findings. As the product matures, the relevant software components will be open-sourced and shared online. More importantly, we are building with open source and out of principle also contribute back our improvements to these original open-source projects so that the bigger community online can benefit.

The Power of Outcome Harvesting: A Paradigm Shift in Monitoring & Evaluation?

What will you learn from this article?

  1. Outcome harvesting is an evaluation approach that documents the outcomes of a program strategy by embracing complexity.
  2. The core principles of outcome harvesting include a contribution analysis, continuous iteration, engagement with key actors, and quantitative data collection.
  3. These approaches are well suited to reveal unexpected, more nuanced outcomes, particularly in complex environments.

In the realm of project management and evaluation, the quest for effective methodologies to capture and assess outcomes is ever-present. Among various approaches, outcome harvesting has emerged as a dynamic and innovative method that places emphasis on learning and adaptation. Our recent work with Agenda for Change allowed us to explore how outcome harvesting can serve as the core foundation for a results framework.

What is outcome harvesting?

Outcome harvesting is an evaluation approach that focuses on identifying and documenting the outcomes of a program strategy or strategic intervention without necessarily knowing what these are in advance. It embraces complexity, encourages actor engagement, and aims to capture unintended and unexpected changes that have occurred. Outcome harvesting provides a flexible, realistic approach to measuring progress.

Figure 1: Outcome harvesting as an evaluation approach

The core principles of outcome harvesting

  1. Contribution Analysis: Outcome harvesting prioritizes understanding the contribution of the project to observed outcomes. It acknowledges that multiple factors influence outcomes and seeks to uncover the initiative’s role within the broader context.
  2. Iterative and Adaptive: Outcome harvesting acknowledges that outcomes can change and evolve over time, which allows for ongoing learning and adaptation.
  3. Engagement: This approach actively involves stakeholders throughout the process, recognizing their unique perspectives and knowledge. Stakeholders play a vital role in identifying outcomes and planning responses, resulting in a more inclusive and comprehensive evaluation.
  4. Qualitative Data Complements Quantitative Data : Unlike traditional evaluation methods that heavily rely on quantitative data, outcome harvesting emphasizes the collection of qualitative data. This includes narratives and stories that provide rich insights into the outcomes and the project’s contribution. Meanwhile, as shown in Figure 1, qualitative stories are complemented with quantitative data (i.e. parameters) in order to affect and evaluate progress on the basis of an organization’s Theory of Change. This process, in turn, enriches the narrative with quantitative information.

Outcome harvesting in practice

An example use case of outcome harvesting could potentially be to capture stories about systems strengthening. In this case, systems strengthening could be measured through stories being classified against building blocks through tags like #financing, #servicedelivery. The stories could be further contextualized with quantitative indicators, such as JMP service level information. In this way, outcome harvesting could be used to holistically examine how an organization’s work is impacting both systems and services.

Benefits of outcome harvesting

Outcome harvesting is particularly well-suited and useful in operations in complex environments with many actors. The approach enables a more nuanced understanding of outcomes which may not be apparent in traditional approaches. Furthermore, by adopting an iterative and adaptive approach, outcome harvesting promotes continuous learning and improvement. One of the key benefits of outcome harvesting is that it is excels at revealing unexpected outcomes, which may include both positive and negative effects. By actively involving stakeholders throughout the process, outcome harvesting fosters ownership and enhances the credibility and usefulness of evaluation results. It builds trust and encourages collaboration among diverse actors.

Figure 2: How are stories are linked to organizational vision and priority areas?

Outcome harvesting offers a new perspective on results frameworks, shifting the focus from rigid indicators to a more adaptive and learning-oriented approach. By embracing complexity, engaging stakeholders, and capturing unexpected outcomes, this methodology unlocks insights that traditional evaluation methods may overlook.

We’re curious to know, have you used outcome harvesting in any of your projects? If so, how? What were the key learnings, challenges, surprises?