Lost causal: Debunking myths about causal analysis in philanthropy

This 2021 paper, updated in 2024, advocates for more causal analysis in philanthropic evaluation - not just describing actions taken and changes observed, but also learning how and why change occurred.

It explores ways to make change pathways more visible, test assumptions and hypotheses about how change happens, and generate new insights based on what happened in the black box of a systems-change strategy.  

In particular, it advocates for evaluations that can investigate cause-effect relationships in dynamic and emergent strategies including network building, field building, advocacy, organizing, and movements. This includes understanding that organizations’ strategies are part of a ‘causal package’ - a broad mix of complementary interventions, actors, events, and contextual factors that work together to produce a change.

The paper sets out and debunks nine common myths:

Rigor and causal analysis

• Myth 1: RCTs or quasi-experiments are the only ways to test cause-and-effect relationships.
• Myth 2: There are no rigorous nonexperimental designs for examining causality.

Usefulness of causal designs

• Myth 3: Causal designs focus on the past and do not help with future decision-making.
• Myth 4: Causal designs are summative only and cannot be used for real-time decision-making.
• Myth 5: Causal designs are not appropriate for complex settings.
• Myth 6: Causal designs are too burdensome for participants.

Causal designs and equity

• Myth 7: Causal designs cannot be implemented in ways appropriate for working with communities.
• Myth 8: Causal designs are rooted in white supremacy.
• Myth 9: Causal designs center philanthropy’s ideas about change.