Assessing the Policy Impact of ‘Indicators’: A Process-Tracing Study of the Hunger And Nutrition Commitment Index

This report applies a process-tracing approach to understand the policy impact of indicators and contributes to debates about assessing the impact of development research. It focuses on the case of the Hunger And Nutrition Commitment Index (HANCI), which publishes annual indices of countries’ political commitment to reduce hunger and undernutrition, as well as complementary knowledge products.

"2.2 Process-tracing methodology

The HANCI is an intervention with ‘complicated aspects’ (Forss, Marra and Schwartz 2011). The HANCI has multiple components (e.g. rankings, scorecards, expert survey evidence), multiple implementing agencies (within country and international), incorporates multiple causal processes, and works differently in different contexts. In order to assess HANCI uptake and policy impact, the study adopts a process-tracing (PT) methodology as advocated by Beach and Pedersen (2013). PT is a systematic qualitative method of enquiry that can be applied to complex contexts with competing causal explanations. PT methods posit a causal mechanism that connects an independent variable X and a dependent variable Y, and employ inferential logic to enhance confidence in the validity of the mechanism. A causal mechanism can be defined as ‘a complex system which produces an 12 CSOs can devise policy framings targeting policy elites, with high levels of specificity, and more generic framings for general audiences (Shiffman 2007). 17 outcome by the interaction of a number of parts’ (Glennan 1996: 52, cited in Beach and Pedersen 2013). 

Beach and Pedersen identify three types of PT: case-explanation, theory building and theory testing. We employ a theory testing PT approach within a single-case research design. Drawing on the literature review discussed in Section 1, we postulate the general causal mechanism for how indicators (independent variable X) influence policy elites (dependent variable Y). We propose that the causal mechanism is broadly applicable across a range of policy contexts and a range of indicators. We then apply and test the theory, using the case of the HANCI, assessing whether the theorised mechanism is actually present, and whether it functions as expected."

Assessing the Policy Impact of ‘Indicators’: A Process-Tracing Study of the Hunger And Nutrition Commitment Index (HANCI) (2016, p.16-17)