Objectives-Based Evaluation (OBE) for impact investing

Bob Picciotto is a former Director General of the Independent Evaluation Group which oversees evaluation in the International Finance Corporation, an agency dedicated to the promotion of private sector development in developing countries.

In this guest blog, he argues that the ethical investment community has much to learn from the Objectives-Based Evaluation (OBE) approach used by the multilateral development banks: OBE rates social and environmental sustainability performance as well as economic and financial returns.

Currently, social impact assessors either use negative screening to exclude ethically controversial investments (tobacco, coal mining, incendiary weapons, pornography, etc.) or engage in positive screening to select corporations that aspire to achieve positive social and/or environmental outcomes as well as generate adequate financial returns to their shareholders. They usually do so by examining the intended goals and relying on a battery of key performance indicators (KPIs).

It is only a matter of time before ethical investors will insist on better assessment standards that demonstrate that their investments are contributing to social and environmental sustainability. In turn, this means that they will expect impact analysis to rely on a comprehensive, fact-based, independently verified assessment of direct and indirect impacts, both intended and unintended. To meet such expectations, social impact analysts might consider adopting Objectives-Based Evaluation (OBE) – the approach used by multilateral development banks in their evaluations of private sector interventions.

Objectives-Based Evaluation

OBE is a distinctive impact assessment approach that supports both accountability and learning. It has been codified by the Evaluation Cooperation Group (ECG), a professional network established in 1996 to promote a harmonized approach to evaluation methodology among the five major multilateral development banks. It has proved serviceable over decades of multilateral development bank experience. The OBE guidelines for private sector operations, first issued in 2001, were updated in 2003, 2006, and 2010. 

Quite apart from business management criteria, OBE assesses whether the evaluand (i.e. the corporation or the project being evaluated) achieves its major relevant objectives efficiently and sustainably, with full account taken of its side effects. It pays special attention to compliance with corporate social responsibility values and principles.

At its best, the OBE approach is highly participatory. It brings diverse perspectives and evidence together and it mobilizes mixed-methods appropriate to the context. It is used ex-ante as well as retrospectively to generate organizational learning. The following OBE characteristics are particularly relevant to the conduct of social impact assessments:

• Go beyond achievement of stated project objectives
• Use rubrics rather than KPIs (Key Performance Indicators)
• Include independent verification of ratings

Going beyond the stated project objectives

Beyond measuring the extent to which intended goals are reached, OBE uses rubrics to synthesize evidence and ascertain value across each of the following criteria, broadly aligned with the development effectiveness criteria of the Development Assistance Committee (DAC) of the Organization for Economic Cooperation and Development (first issued in 1991 and updated in 2019):  

• Relevance: Is the corporation doing the right things? Are its policies aligned with generally accepted corporate social responsibility standards and/or are its corporate objectives contributing to the Sustainable Development Goals?

• Efficacy: Is it achieving its intended objectives and doing so in compliance with sound corporate social responsibility policies and standards?

• Efficiency: Is it making efficient, responsible use of scarce resources (including natural resources)?

• Coherence: How compatible and synergistic are corporation activities relative to generally accepted international norms and local, national, and global programs addressing ‘problems without passport’? 

• Impact: Is the corporation making a significant and positive difference in terms of its potentially transformative effects on the environment, and the society, taking account of its unintended, indirect, secondary, and potential consequences on people’s well-being, social equity, and the natural environment?

• Durability: Are the net economic, social, and environmental benefits of corporate operations likely to be sustained over the medium and long-term?

A 4 or 6-point rating scale ranging from highly unsatisfactory to highly satisfactory is used, based on a combination of evidence secured from document reviews, fieldwork, stakeholder consultations, combined with judicious deployment of evaluation methods (e.g. cost-benefit analysis). Each criterion is rated individually and then combined. Based on estimated scores for each criterion, an overall development effectiveness rating is generated through thoughtful aggregation of performance ratings rather than simple averaging.

Rubrics vs. KPIs

The use of OBE does not preclude the use of Key Performance Indicators (KPIs) for performance monitoring.  Indeed, OBE complements KPIs that suffer from well – known limitations as evaluation instruments. As Mishkah Jakoet opined in a previous blog: “A single number can leave out important considerations, especially when there are multiple aspects of an impact that should be included”. By contrast, OBE is evaluative: it uses rubrics to deliver a comprehensive assessment of merit, worth and significance.

A rubric well used synthesises diverse evidence and embraces dimensions that cannot be fully captured by KPIs selected at the start. It takes explicit account of new and emerging evidence about intended and unintended impacts. The overall rubric-based rating does not privilege some criteria at the expense of others – e.g. it avoids giving excessive emphasis to corporate goal achievement (efficacy) at the expense of relevance or impact.

By way of illustration, a large dam project constructed by a private utility aimed at generating power at low cost may achieve excellent profitability while failing to resettle thousands of farmers displaced by the rising waters (impact). Or it may opt to distribute the generated power to prosperous neighbourhoods without providing access to electricity to disadvantaged communities (relevance). In both cases, OBE would generate an unsatisfactory overall rating for the intervention, despite the ‘clean’ energy claims of its corporate sponsor.

Independent verification

Finally, a critically important feature of OBE is independent verification that attests to the validity of the evidence, the rubrics process, and the resulting scores. Thus, in the multilateral development banks, the ratings are first awarded by operational staff (self-assessment) and then verified by an independent evaluation unit that reports to the Board of Directors. This is akin to an auditing function that ascertains the probity of financial statements.

Such a due diligence feature combined with thoughtful use of rubrics and objectives-based assessments going beyond achievement of intended goals would go a long way in reassuring ethical investors that their hard-won savings are being mobilized to promote the public interest.

Read more about Objectives-Based Evaluation:

Robert Picciotto (2013) The logic of development effectiveness: Is it time for the broader evaluation community to take notice? Evaluation. Vol. 9. Issue 2. April. pp. 155-170 (Journal access required)

Vinod Thomas and Namrata Chindarkar (2019) Economic Evaluation of Sustainable Development. Palgrave Macmillan, pp. 95-123 – available as free download especially the chapter: Objectives-Based Evaluation for Accountability and Learning