Complexity

An issue of increasing interest in evaluation, especially development evaluation, is whether and how we might apply ideas and methods from complexity science to evaluation.

Complexity ideas and methods have important applications for how we think about programs and policies, how we collect and analyse data, and how we report findings and support their use.

In 2014 this is one of the priority themes that BetterEvaluation will be focusing on, with events, activities and resources planned throughout the year. We’ll be updating this page throughout the year.  You can also follow this discussion, and add to it, on Twitter using the hash tags #complexity #eval.

Complexity is sometimes dismissed as a ‘trendy’ term that is used to avoid accountability and planning.  But there are two important ideas that it raises:

  • Multiple components (sometimes labelled ‘complicated’
  • Emergence

Interventions can have some simple aspects, some complicated aspects and some complex aspects, and it is more useful to identify these than to classify a whole intervention as complex.

Multiple components

Many evaluations have to deal with programs with multiple components, multiple levels of implementation, multiple implementing agencies with multiple agendas, and long causal chains with many intermediate outcomes, or outcomes that can only be achieved through a 'causal package' involving multiple interventions or favourable contexts.

In these situations, evaluations have to be based on a logic model and data collection and analysis plan that provides information about the different components which all need to work effectively and together, or processes that work differently in different contexts, or which only work in combination with other programs or favourable environments. It is essential to report on these in terms of ‘what works for whom in what contexts’.

In some frameworks (especially two classic papers by Glouberman and Zimmerman (2002)  (see http://albordedelcaos.com/2011/09/27/conociendo-el-blog/   for a Spanish language version of these ideas)and Kurtz and Snowden (2003) , this aspect is referred to as “complicated’ to distinguish it from emergence.

Emergence

Many evaluations have to deal with programs that involve emergent and responsive strategies and causal processes which cannot be completely controlled or predicted in advance. While there is an overall goal in mind, the details of the program will unfold and change over time as different people become engaged and as it responds to new challenges and opportunities.Projects that focus on community development or leadership development are particularly likely to have these features.

In these situations, evaluations have to be able to identify and document emergent partners, strategies and outcomes, rather than only paying attention to the objectives and targets identified at the beginning. Real-time evaluation will be needed to answer the question “What is working?” and to inform ongoing adaptation and learning. Effective evaluation will not involve building a detailed model of how the intervention works and calculating the optimal mix of implementation activities - because what is needed, what is possible, and what will be optimal will be always changing.

Resources

Exploring the science of complexity: Ideas and implications for development and humanitarian efforts

The paper details each of the 10 concepts of complexity science, using real world examples where possible. It then examines the implications of each concept for those working in the aid world. Here, we list the 10 concepts for reference, using the next section of this summary to suggest some overall implications of using the concepts for work in international development and humanitarian spheres.

View resource

Discussion Note: Complexity Aware Monitoring

USAID’s Office of Learning, Evaluation and Research (LER) has produced a Discussion Note: Complexity-Aware Monitoring, intended for those seeking cutting-edge solutions to monitoring complex aspects of strategies and projects.

View resource

Complex adaptive systems: A different way of thin

king about ​Health care systems

Looking at how complexity science could be used in health systems which are characterised by nonlinear dynamics and emergent properties arising from diverse populations of individuals interacting with each other and which are capable of undergoing spontaneous self-organisation..

View resource

 

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.