Supporting use of evaluation findings - examples

These examples have been contributed for discussion at the 'flipped conference' session of the American Evaluation Association to be held  Saturday (November 11, 2017) 09:15 AM - 10:00 AM in the Thurgood Marshall North Room. 

Example 1 - Using learnings about sustainability - Patricia Rogers

Some years ago I worked on an evaluation of a large, diverse program that involved many hundred of different projects funded under 7 different initiatives.  Sustainability was an important issue for these projects which only received short-term funding, and as part of the evaluation we produced an issues paper on sustainability (archived link) and also followed up 100 projects after they ended and produced a report (archived link) on the different types of sustained impact they produced and what factors predicted these.  One of the big findings was that projects which had developed a plan for sustainability early on were more likely to have sustained impacts.

Three important strategies were used to support the use of these findings. 

Firstly, the government department which had commissioned the evaluation, built the finding into the requirements for the next stage of the program.  Projects that received funding under the next stage were required to develop a sustainability plan within the first 12 months.

Secondly, ARACY (the Australian Research Alliance for Children and Youth), which brought together researchers, policymakers and practitioners as a kind of knowledge broker, commissioned the lead evaluator to write a plain language summary of the earlier reports, and to also do additional research to document the use of these ideas in other projects.

Finally, the lead evaluator was engaged to run a workshop with the leaders of projects funded in the next stage to work through the ideas and help them start drafting sustainability plans.

There were therefore four different strategies brought together – mandated implementation requirements, new knowledge products from existing material, new evaluation work based in the local context, and interactive workshops. 

Example 2 - Building multiple reporting phases into a three-year evaluation - Jade Maloney (ARTD)

What was it an evaluation of?

A new disability support program.

What was done to support people to put the findings into practice?

Over the three-year evaluation, there were three reporting phases so early findings could inform ongoing roll-out. Each reporting phase had a face-to-face discussion with managers before written reporting to support shared interpretation of findings. After the final phase, there was a discussion group with key frontline staff to reality test and identify how the recommendations could be taken up. Finally, there was a joint conference presentation on evaluator recommendations and organisation response, plus progress.

How did it work out?

The multiphase approach enabled improvements to be identified, then implemented and tracked in next phase of evaluation.

What factors helped or hindered people to apply the findings?

The systems set up wouldn't have been enough without organisational commitment to learning from the evaluation.