Innovations in monitoring & evaluating results

This discussion paper, written by Thomas Winderl and edited by Jennifer Colville for the United Nations Development Program (UNDP), provides a detailed inventory of innovative practices used for monitoring and evaluation in a development context.

Excerpt

"Eleven innovations have been identified based on extensive research and analysis. Increased frequency of input and broader citizen participation are key features in most of the innovations presented in this paper; in addition, many present cost-conscious and flexible approaches to managing and assuring quality of policies, programmes and service delivery. The first eight innovations promote citizen engagement, with the first five requiring active participation of citizens and the next three reflecting more passive engagement. The ninth is designed to enhance the usefulness and accessibility of the information collected, and the final two present progressive methodologies for more credibly measuring and interpreting results. Most of the innovations are not mutually exclusive; for example, mobile data collection can be used with micro-narratives to provide different perspectives on a particular initiative."

Contents

• Crowdsourcing:  A large number of people actively report on a situation around them, often using mobile phone technology and open source software platforms
• Real-Time, Simple Reporting:  A means to reduce to a minimum the formal reporting requirements for programme and project managers and free up their time to provide more frequent, real-time updates, which may include text, pictures, videos that can be made by computer or mobile devices
• Participatory Statistics: An approach in which local people themselves generate statistics; participatory techniques are replicated with a large number of groups to produce robust quantitative data
• Mobile Data Collection: The targeted gathering of structured information using mobile phones, tablets or PDAs using a special software application
• The Micro-Narrative: The collection and aggregation of thousands of short stories from citizens using special algorithms to gain insight into real-time issues and changes in society
• Data Exhaust: Massive and passive collection of transactional data from people’s use of digital services like mobile phones and web content such as news media and social media interactions
• Intelligent Infrastructure: Equipping all – or a sample of – infrastructure or items, such as roads, bridges, buildings, water treatment systems, handwashing stations, latrines, cookstoves, etc., with low-cost, remotely accessible electronic sensors
• Remote Sensing: Observing and analyzing a distant target using information from the electromagnetic spectrum of satellites, aircrafts or other airborne devices
• Data Visualization: Representation of data graphically and interactively, often in the form of videos, interactive websites, infographs, timelines, data dashboards, maps, etc.
• Multi-level Mixed Evaluation Method: This approach includes the deliberate, massive and creative use of mixed (quantitative and qualitative) methods on multiple levels for complex evaluations, particularly for service delivery systems
• Outcome Harvesting: An evaluation approach that does not measure progress towards predetermined outcomes, but rather collects evidence of what has been achieved, and works backward to determine whether and how the project or intervention contributed to the change