Un-boxing evaluation through developmental and agile approaches

Guest author Nerida Buckley discusses how un-boxing evaluation can benefit from looking at practices from developmental and agile approaches.

This year’s AES conference asks us to think outside the box, or even boxes, when it comes to delivering on the potential of evaluation in our increasingly complex and disrupted world. This includes the potential to uphold innovation, the potential to facilitate just and equitable transitions, and the potential to shape public value in a given context.

As an emerging evaluator and public sector employee, I’m interested in how tried-and-tested and emerging approaches to program design and delivery from different sectors can be blended to deliver on the promise of public value. For the last few years at the annual AES Conference we’ve been hearing more about the strengths of Developmental Evaluation (DE) in tackling uncertainty, and facilitating real-time decision making that incorporates user needs.

The parallels between DE and Agile Project Management methodology are strong – both approaches broadly seek to maintain focus on end user outcomes, to encourage innovation, and provide a framework for informed and iterative decision-making in complexity. There is also an alignment with adaptive project management which emerged from the environmental management sector and heavily influences the Agile Manifesto.

For their similarities, there are also differences between these approaches. Where Agile works within discrete project boundaries, DE enables system-level enquiry. Whilst they both lend themselves to participatory approaches, DE stresses the importance of relationships between evaluator, project team and stakeholders to build trust and underpin participation.

So how might implementing these approaches in a more traditional organisation play out in reality? There is an important distinction in the Agile Manifesto that underpins how the process of implementing inter-disciplinary approaches works in reality: it is necessary to have  a supportive environment in which motivated and self-organising teams are trusted to ‘get the job done’.  Organisations using DE and Agile approaches really need to embrace uncertainty, be comfortable with a level of risk, and empower delivery staff for DE to show its true value. Maintaining focus on outcomes over outputs is key. Some practical measures that may support more organisations to embed DE and Agile approaches include:

• Incorporating sprint debrief milestones into regular project reporting to demonstrate how the uncertain can become certain. Keep in mind key learnings you will want to communicate to the Project Sponsor to make progress tangible throughout delivery.

• If procuring a consultant, build in contingency when securing approvals at the sourcing stage and ensure this is also reflected in your program’s overall budget. This will allow you to build flexibility into your contract and allow for adaptation as you deliver.

• Developing a joint schedule, at least at a high level. DE is part of project design and delivery, not a special add-on.

• Incorporating user testing feedback (in agile terms, priorities from your product backlog) into the project’s quality assurance plan or success criteria as you go, where relevant.

• Considering how DE may facilitate effective risk management for more ambiguous or challenging programs and, where relevant, integrating with risk management plans and reporting milestones.

• Embedding DE consultants into project teams and encouraging collaboration with staff from relevant service teams, such as communications and engagement, to facilitate delivery efficiencies.

• Where possible, providing opportunities to make the DE approach and knowledge outputs visible to others, particularly decision-makers.

In today’s resource-constrained, and sometimes politically unpredictable environment, ambiguity can be the norm for decision-makers and project managers. Evaluators can play an important role in helping organisations navigate complexity by embedding approaches within existing operational and project management frameworks. This may help de-risk new and emerging approaches for decision-makers and encourage focus on public value outcomes.

I’m looking forward to this year’s AES conference and learning how other sectors are implementing new and emerging approaches, as we look beyond the traditional box of evaluation.

What does un-boxing evaluation mean to you?

Un-boxing evaluation is not a straightforward process. It raises a lot of questions about what it means to be an evaluator and what it means to do evaluation. We’re hoping that aes19 will advance these conversations and that you’ll be inspired to join in. Registrations close on Wednesday, September 4th – hopefully we'll see you there! 

This blog is the fifth in our series about un-boxing evaluation – the theme of aes19 in Sydney, Australia. The series is designed to generate a global discussion of the theme ‘un-boxing evaluation’ and what that means for our profession and practice.