Evaluation, valuation, negotiation: some reflections towards a culture of evaluation

This article, "Evaluation, valuation, negotiation: some reflections towards a culture of evaluation" explores the issues of developing standards for an evaluation, when these have not previously been agreed, in a rural development program in Argentina. 

Authors and their affiliation

Pablo Rodríguez-Bilella (CONICET / IOCE. Argentina) and Rafael Monterde-Díaz (Universidad Politécnica de Valencia. / SEE. España)

Year of publication 

2012

Type of resource

Example

Key features

One of the trickiest situations in evaluation is when, after the draft report is submitted, it becomes clear that the evaluation team and the commissioning organization have different ideas about what the evaluation should do and how it should do it - and therefore have major differences about what the evaluation report should say.  

In this interesting example, BetterEvaluation community member Pablo Rodríguez-Bilella and his colleague Rafael Monterde-Díaz discuss a case where these differences became evident very late in the process.  They describe the changes made to the first, second and third drafts of the evaluation report to address concerns raised by the commissioning agency, while trying to remain true to the evidence that had been gathered and the evaluative criteria that had been previously identified.

They recommend addressing these issues up front wherever possible - in particular agreeing on the criteria and standards for evaluating the program or policy, and being clear that the evaluation report will refer explicitly to these. While these may change over the period of the evaluation, ongoing discussions about these will help to develop a culture of evaluation within the organization.

Who is this resource useful for?

• Commissioners/managers of evaluation;
• Evaluators;

How have you used or intend on using this resource?

We've used this resource as an example to highlight the importance of discussing the standards for an evaluation with stakeholders prior to the evaluation taking place. It's also a useful tool for both evaluators and commissioners to understand the negotiation process that can take place without this understanding.

Why would you recommend it to other people?

Examples like this, which come from real-world, messy situations in evaluation practice, help to capture the process of managing stakeholders and expectations in a way that can be much more useful than a textbook or manual. They raise a vital question that is important for both self-reflection and for enhancing the field through healthy discussion - What would you have done in this situation? For those who are new to evaluation, examples such as these will draw attention to potential issues that they may come up against, and potentially help to ward these off. More experienced hands at evaluation may be able to identify with some of the elements of this story.