Modus operandi

Interventions create distinctive/characteristic patterns of effects.

Scriven describes the modus operandi as a set of footprints:

"Each cause has a set of footprints, a short one if it’s a proximate cause, a long one if it’s a remote cause, but in general the modus operandi is a sequence of intermediate or concurrent events or a set of conditions, or a chain of events, that has to be present when the cause is effective."

An evaluator can draw on the previous experience of participants and stakeholders to determine what constellation or pattern of effects is typical for an initiative.

The evaluator can build a picture of predicted effects and supplement this picture through research. The combined picture will help the evaluator look for typical patterns of effects and contrasting patterns. The consistency of the ‘trace’ with the predicted pattern provides evidence for the program theory. Any differences from the predicted pattern helps to disprove the program theory or open a new line of questioning. 

Realist synthesis can be added to modus operandi. Realist synthesis asks: What mechanisms are known to operate with particular subpopulations?

Examples

Participants in a leadership programme in the New Zealand public service developed individual development plans. The evaluator articulated the modus operandi of the programme as:

  • Participant plans identified needed experiences
  • Candidates sought these out
  • Candidates obtained a more senior position to gain that experience

Advice for choosing this method

Provide the evaluator with the original program theory - if there was one - or relevant documents to develop a retrospective program theory.

Advice for using this method

  • Develop a list of possible alternatives, based on evidence and information gathered throughout the evaluation process. Test the program theory and counterfactual with participants.
  • Liaise with the evaluators to check the program theory is correct and the validity of possible alternative explanations and the counter-factual.

Resources

Davidson, J. (2010, May). Outcomes, impacts & causal attribution. Presented at ANZEA regional symposium, Auckland, New Zealand. Retrieved from http://realevaluation.com/pres/Causation-anzea2010.pdf

Scriven, M. (1974). Maximizing the power of causal investigations: The modus operandi method. In W. J. Popham (Ed.), Evaluation in education: Current applications (pp. 68-84). Berkeley, CA: McCutchan Publishing.

Expand to view all resources related to 'Modus operandi'

'Modus operandi' is referenced in:

Back to top