• Systems Approach for Lasting Change: Tanager focuses on addressing the underlying causes of global poverty through market systems development, exemplified by our SELEVER 2 project in Burkina Faso, which aims to transform the poultry sector inclusively.
• Challenges in Measuring Outcomes: Market systems projects like SELEVER 2 face difficulties in capturing and attributing changes directly to interventions due to their indirect impact through various market actors. This necessitates sophisticated monitoring and evaluation (MEL) strategies.
• Synchronizing Methodologies for Consistency: SELEVER 2 synchronizes sampling methodologies across baseline, midline, and annual surveys to ensure consistent measurement of progress. This approach helps identify trends, refine strategies, and understand the project’s broader impact on the community.

By Dave Shaw, Director, Data & Learning 

As a leading proponent of market systems work, Tanager implements projects at a systems level wherever possible. This approach identifies and addresses the underlying causes of global poverty to enact lasting, large-scale change, rather than piecemeal solutions at a single point of intervention. In Burkina Faso, for example, Tanager is using a systems approach to inclusively transform the country’s poultry sector. (The project, Soutenir l’Exploitation Famaliales pour Lancer l’Elevage des Volailles et Valoriser l’Economie Rurale, or SELEVER 2, is slated to run through 2026.) 

Market systems development projects like SELEVER 2 face unique challenges in measuring outcomes, however. Unlike direct-intervention projects, where the impact on beneficiaries can be more easily assessed, projects that take a systems approach operate through various market actors. This makes it difficult to capture and attribute changes directly to project interventions. These projects therefore require sophisticated and innovative monitoring and evaluation (MEL) strategies to effectively measure progress and impact. 

Periodic Evaluations vs Annual Surveys

One common approach for market systems projects is to implement methodologically rigorous baseline, midline, and endline evaluations, often using quasi-experimental designs with control groups. This approach helps isolate the project’s impact by tracking changes over time in both participant and non-participant groups, providing a clearer picture of the project’s effectiveness and the broader systemic changes it seeks to foster. SELEVER 2 conducted its midline survey in May 2023, allowing for robust comparisons of outcomes between participants and non-participants.  

In contrast, annual outcome surveys (which are conducted in the interim periods between baseline, midline, and endline evaluations) often use simpler designs. These typically focus on direct project participants without including control groups.  

While this approach is more streamlined, it can create challenges when comparing results across different evaluation periods: The detailed data from baseline and endline evaluations may capture broad systemic changes, while annual surveys might only reflect immediate outcomes for direct participants. This methodological disparity can lead to inconsistencies in the data. It may also overlook the project’s indirect effects on the wider community, resulting in a fragmented understanding of overall success.  

Synchronizing Methodologies

To address these challenges, SELEVER 2 adopted a MEL approach that synchronizes the sampling methodologies used in baseline, midline, and endline evaluations with those of the annual outcome surveys. During both the 2023 midline study and the 2024 annual outcome survey, for example, SELEVER 2 randomly sampled both participants and non-participants within intervention communities. The key difference between the 2023 evaluation and the 2024 outcome survey lies in the scale: The annual survey targeted a smaller sample size and measured fewer indicators than the midline evaluation.   

The project is currently analyzing the results from the 2024 annual survey, comparing them with midline outcomes. By aligning the sampling methodologies, SELEVER 2 ensures that progress within the broader market system can be measured consistently every year. Such an approach also accelerates the learning cycle, enabling the project to rapidly identify emerging trends, refine strategies to address evolving challenges, and develop a more nuanced understanding of the project’s indirect effects on the wider community.  

For Tanager, this synchronized methodology is also generating benefits beyond the SELEVER 2 project. In fact, our adaptive MEL methods enhance our organization’s capabilities in a few ways. By establishing a clear and consistent framework for measuring outcomes under SELEVER 2, we are reinforcing our commitment to transparency and accountability. We continue to strengthen our experience in market systems development through a robust methodology that more accurately tracks our impact. Perhaps most importantly, this streamlined approach to data collection and analysis helps us realign resources, if needed, to benefit the people that matter most: our project participants.