ALNAP guide: Evaluation of protection in humanitarian action

This companion to the ALNAP EHA Guide offers protection-specific insights for evaluations and evaluation commissioners across the humanitarian sector.

Challenges addressed include those arising from the multi-faceted nature of protection activities, the difficulty of understanding cause-effect relationships underlying protection risks, and the challenges of accessing and managing very sensitive data.

This resource and the following information was contributed by Amelie Sundberg, Neil Dillon, and Maria Gill.

Authors and their affiliation

• Ian Christoplos, Director at Glemminge Development Research 
• Neil Dillon, Research Fellow at ALNAP
• Francesca Bonino, at the time of writing she was a Research Fellow at ALNAP. She now works as Senior Evaluation Officer with UNHCR.

Key features

The Guide is useful as it covers the full evaluation cycle, from planning through data management to analysis and use. It also includes a set of 6 toolkits for commissioners and evaluators alike, that can be tried and used in practice. It also includes a range of considerations and insights from other evaluations in this area.

The Guide is designed to be accessible to even staff without an evaluation background, as agency staff in protection programming advisory and support roles are often called upon to contribute to the evaluation cycle. The Guide can be read from start to finish, but we recommend users focus on different modules depending which stage of the evaluation process is most relevant to them.

Module A: Initiating the evaluation of protection in humanitarian action

Module B: Data management: legal and ethical implications

Module C: Analysis

How have you used or intend on using this resource?

I found the module on initiating the evaluation particularly useful. It clarifies the protection-specific evaluability conditions and opportunities to promote utility. This was helpful to ringfence the expectations that senior management had of an evaluation we conducted of social cohesion programming – a complex concept often hard to pin down. 

Why would you recommend it to other people?

Often in humanitarian contexts we can rush into planning for an evaluation, without truly considering if it is appropriate and relevant to undertake an evaluation in the first place. This is particularly important for protection programming, due to the sensitivity of the topic and the addition logistical and ethical challenges to data collection that this brings with it. Moreover, protection programming can often be designed with objectives that are difficult to define and measure (for example, women feeling more secure in their communities – which can mean different things to different people). An evaluator will need to check if the programme’s results framework is coherently articulated and that expectations from the evaluation are realistic.