What is it
Unintended results may be positive or negative. Unintended results should be considered as part of the development of a program theory (see Develop program theory or logic model). This means that possible unintended outcomes and impacts, especially negative impacts (that make things worse, not better) can be investigated and tracked. Negative program theory involves identifying ways in which program activities might produce negative impacts rather than their intended impacts, and this can be done at the same time as the standard program theory. In addition, the data collection in the evaluation should remain open to finding unanticipated unintended results by including some open-ended questions and methods that might uncover this (such as interviews or by encouraging reporting of unexpected results).
The main BetterEvaluation site points to a range of options for identifying potential unintended results, both before implementation to build on work to Develop program theory or logic model, and as part of data collection and monitoring systems. Consider these additional tasks and the general information pages before considering options to apply to C4D. Key informant interviews involve asking experienced people to identify possible negative impacts, based on their experience with similar programs. Program critics can be especially useful. Risk assessment identifies the potential negative impacts, their likelihood of occurring and how they might be avoided. Six Hats Thinking about unintended results: might be a process that can be used to encourage people to consider possible negative impacts and how they might come about. Unusual events reporting could be a part of the open-ended data collection, ensuring that unforeseen events, incidents or outcomes are recorded.
Identifying potential unintended results and C4D
Applying the C4D Evaluation Principles
It is not possible to predict all the impacts that might emerge from an intervention with complex aspects. These impacts can be positive or negative, and once identified responses can be developed. Therefore R,M&E plans need to have some way of looking backwards to identify and document these (such as through open-ended questions in interviews).
|Unintended results can be both positive and negative. As part of being accountable it is important to minimise any harm from unintended results from C4D. We need to use tools to help us predict (as far as possible) unintended outcomes, together with monitoring processes to identify and respond to unpredictable and negative unintended impacts as quickly as possible.|
|Unintended results may not affect everybody, and adverse outcomes for minority groups may not be obvious in the data. A critical approach and an equity lens to the identification of unintended results with contributions from local groups is important for understanding how C4D initiatives are affecting the least powerful.|
|Holistic, open and interpretive approaches to data collection are important for identifying unintended results. Ethnographic and Ethnographic Action Research approaches are particularly strong in this way.|
|Involving different stakeholders in the task can draw on their unique knowledge and perspectives about an initiative, and reveal new information.|
Recommended options and adaptations for C4D
The EAR Toolbox provides guidance on using Ethnographic Action Research approaches (Click here to read a summary of how it is consistent with the C4D Evaluation Framework). This toolbox and the approach support participatory and holistic R,M&E approaches that are particularly good for identifying unintended results. It is consistent with the C4D Evaluation Framework in the following ways:
- learning-based: the action research approach means that the emphasis on continual learning and evaluation towards improvement. Unintended results should become evident throughout the implementation process.
- complex: because of the learning-based approach, the unpredictable unintended results should become evident throughout implementation.
- holistic: the open nature of the ethnographic approach means that the approach is particularly useful for uncovering unintended results.
Cholera outbreak in Kenya
A cholera outbreak in Kenya in Kenya highlights the need to be monitoring for unintended outcomes of communication. In this case, a health program implemented in 3 Counties had been very successful in introducing zinc tablets, Oral Rehydration Salts, and Amoxicillin antibiotics, dispensed by frontline workers, for the prevention and treatment of diarrhea. Many lives were saved.
One of the Counties in which the health program had been implemented got hit with a cholera outbreak. Reports from the outbreak location indicated that some families were not taking their sick family members to hospital and were instead treating them at home. By the time they did come to hospital the cases were critical. It was recognised that this was the result of unintended consequences of the Zinc/ORS campaign which fuelled the misconception that cholera characterised by watery stool can be treated at home with ORS Zinc just like diarrhea.
"Now they have zinc, they have ORS, and they have seen community health volunteer, who is just their neighbour, treat their child who was has Pneumonia very effectively. So, the unintended communication is that you can actually manage some of these things at home. So as a result they were trying to take zinc and ORS. So we’ve got to go back and tell them there is a difference between this diarrhea and the other one we told you about. If we are not able to monitor that, we are not doing good practice." (interview, C4D UNICEF Kenya)
This exemplifies the following principles:
- complex: the health promotion program was working as intended, but the introduction of a new factor (cholera) changes the interaction of causes and outcomes for diarrhea. This case shows the importance of staying attuned to the situation in complicated and changing environments and adaptive messages and programming in responsive ways to avoid doing harm.
Retrospective Analysis study of Open Defecation in Nadia District, India:
The UNICEF India Office commissioned a retrospective analysis of a successful campaign and social mobilisation effort towards Open Defecation Free status. This initiative exemplifies the C4D Evaluation Framework in relation to this task in the following ways:
holistic: Ethnographic approaches were chosen in order to provide a holistic on how the campaign had worked (and perhaps, not worked) and what the unintended results had been.
critical: Ethnographic approaches and particularly fieldsites were chosen in order to provide a critical perspective on how the campaign and related initiatives had been experienced by particular sub-groups (caste, ethnicity, gender, wealth, geographical location).
More information about how this study exemplifies the approaches advocated in the C4D Evaluation Framework will be available soon.