Synthesise Data Across Evaluations

These options answer questions about a type of intervention rather than about a single case – questions such as “Do these types of interventions work?” or “For whom, in what ways and under what circumstances do they work?” The task involves locating the evidence (often involving bibliographic searches of databases, with particular emphasis on finding unpublished studies), assessing its quality and relevance in order to decide whether or not to include it, extracting the relevant information, and synthesizing it.  Different options use different strategies and have different definitions of what constitutes credible evidence.

Options

  • Best evidence synthesis: a synthesis that, like a realist synthesis, draws on a wide range of evidence (including single case studies) and explores the impact of context, and also builds in an iterative, participatory approach to building and using a knowledge base.
  • Lessons learnt: Lessons learnt can develop out of the evaluation process as evaluators reflect on their experiences in undertaking the evaluation.
  • Meta-analysis:  a statistical method for combining numeric evidence from experimental (and sometimes quasi-experimental studies) to produce a weighted average effect size.
  • Meta-ethnography: a method for combining data from qualitative evaluation and research, especially ethnographic data, by translating concepts and metaphors across studies.
  • Rapid evidence assessment: a process that is faster and less rigorous than a full systematic review but more rigorous than ad hoc searching, it uses a combination of key informant interviews and targeted literature searches to produce a report in a few days or a few weeks.
  • Realist synthesis: synthesizing all relevant existing research in order to make evidence-based policy recommendations.
  • Systematic review: a synthesis that takes a systematic approach to searching, assessing, extracting and synthesizing evidence from multiple studies.  Meta-analysis, meta-ethnography and realist synthesis are different types of systematic review.
  • Textual narrative synthesis: dividing the studies into relatively homogenous groups, reporting study characteristics within each group, and articulating broader similarities and differences among the groups.
  • Vote counting: comparing the number of positive studies (studies showing benefit) with the number of negative studies (studies showing harm).

Resources

Websites

 

 

Comments

There are currently no comments. Be the first to comment on this page!

Add new comment

Login Login and comment as BetterEvaluation member or simply fill out the fields below.