Measuring the Un-Measurable: Solutions to Measurement Challenges in Fragile and Conflict-affected Environments

This guide, written by Vanessa Corlazzoli and Jonathan White for the UK Department for International Development (DFID), was developed to improve the measurement of programmes in fragile and conflict environments. The guide provides a detailed overview of tools, methodologies and approaches that can be used in these situations and includes examples of different ways the tool can be integrated into programme evaluation.   

Excerpt

"To improve overall societal stability and resilience in situations of conflict and fragility, interventions will need to include a range of interconnected sectors, including peacebuilding, justice, security, and crime. In order to make a contribution towards development, these programmes will have to be able to monitor their contributions, measure their impacts, and learn from their results.

While nuances exist between the types of interventions, accurate and rigorous measurement of desired change is a common challenge for any sector working in conflict and fragile environments. The changes sought in knowledge, attitude, behaviour and perceptions are not easily grasped, observed or measured; they are inherently intangible. This is compounded by obstacles that prevent or hinder effective and rigorous data collection techniques and evaluation methods in conflict and fragile situations (FCAS)." (Corlazzoli & White, 2013)

Contents

• Challenges to Measuring Impact in Peace & Conflict, Security & Justice 9
• Social science methodology for outcome measurements 15
  • Likert scales 15
  • Community Score Cards 17
  • Indices 19
  • Proxy Indicators 20
  • Information and Communications Technologies 22
  • Incident Log 24
  • Media Content Analysis and Discourse Analysis 24
  • Participant Diaries and Video Logs 26
  • Rapid Assessment Procedures 28
  • Remote Monitoring 29
  • Social Network Analysis 30
  • Stakeholder Analysis 31
  • Outcome Mapping 32
• Social science methodology for impact-level and long term measurement 35
  • Impact Evaluation, Quasi- and Experimental Methods 35
  • Natural Experiments 36
  • List Experiments 37
  • Most Significant Change 39
  • Global Giving Storytelling Methodology 39
  • Systems analysis 41
• Social science methodology for measuring contribution to peace writ large 43
  • Longitudinal and Cohort Studies 43
  • Meta-Analysis 44