In the past week I’ve been invited by three different groups to give presentations on similarities and differences between Cost-Benefit Analysis (CBA), Social Return on Investment (SROI) and Value for Investment (VfI) and how to navigate decisions about which of these approaches to use. It’s great to see increasing awareness that there are multiple options for tackling value-for-money (VfM) questions and a desire to understand what each option offers.
Previously, I’ve written detailed analyses that compare these methods from different perspectives. I’ll share links to those articles at the end. The purpose of this article is to simplify the comparison, with a series of visual summaries to highlight key features of the three approaches.1
The following diagrams (or something like them) are likely to find their way into my PowerPoint presentations - so please let me know if they do help clarify things.
CBA, SROI and VfI provide different sorts of conclusions
One of the ways to contrast the three approaches is by what their conclusions look like. Each provides a different kind of answer to a VfM question. The scope of the answer expands as you progress from CBA to SROI to VfI. Here I’m just talking about the punchline, not the whole story…
CBA expresses the value of a policy or program as a number.2 This number, usually a Net Present Value (NPV) or Benefit Cost Ratio (BCR), is an aggregation of monetary valuations of resources and impacts attributed to the investment. For example, an NPV of a particular intervention might indicate that its benefits exceed its costs by $5 million. A BCR of the same intervention might indicate that for every $1 of resources invested, $2 worth of benefits are created.
SROI provides the same kind of number as CBA, plus a performance story. The number, called the SROI ratio, is equivalent to a BCR and represents the present value of benefits divided by the present value of costs. The performance story often draws on qualitative, quantitative and financial evidence to explain how the policy, programme or organisation functions and what it achieves. You might see similar information in a CBA report, too. The difference is that in a CBA this information principally plays a supporting role in providing background information and explaining rationale for modelling decisions, whereas SROI guidance is more explicit about treating the performance story as an important component of the findings in its own right. In this way, SROI is intended to provide greater breadth of information than a CBA.
VfI provides transparent evaluative judgements, based on agreed criteria (aspects of value), standards (levels of value) and multiple strands of evidence. VfI can include CBA or SROI findings within the evidence mix, but it doesn’t have to. Here’s an example of what a set of evaluative judgements can look like (the specific criteria and standards are determined contextually so the ones shown are illustrative). In this example, we could use something like CBA or SROI to provide full or partial evidence for the cost-effectiveness criterion, and various other quantitative and qualitative methods to supply evidence to address other criteria.
This comparison only highlights how the headline findings differ: a number (CBA), a number and a performance story (SROI), and a set of evaluative judgements backed up by evidence and transparent reasoning (VfI). All three approaches provide much more than just headline findings, of course. I would expect to see breakdowns of evidence and discussion of findings in all three reports.
Furthermore, any CBA, SROI or VfI report should be transparent about design and methodological decisions, data sources, assumptions, procedures, sensitivity analysis, and limitations.
CBA, SROI and VfI have different theoretical foundations
CBA is built on principles from welfare economics, a branch of economics that studies how the allocation of resources affects the overall wellbeing of society.
SROI’s home turf is social accounting. However, the SROI Guide references the same suite of methods for placing monetary values on impacts as economists use. So in the diagram below I have plotted SROI at the intersection of social accounting and economics.
VfI integrates theory and practice from evaluation and economics. VfI is explicitly an inter-disciplinary approach. Moreover, VfI draws more from economics than just CBA. It also incorporates other economic methods of evaluation (such as cost-effectiveness and cost-utility analysis, where feasible and appropriate), and uses economic concepts such as opportunity cost, efficiency, productivity, and more.
The conduct of a specific VfI, SROI or CBA may draw on other disciplines to inform study designs and methods that fit different contexts. For example, an assessment of a population health program might draw on insights from epidemiology. However, this diagram captures the disciplinary roots from which each approach developed.
CBA, SROI and VfI vary in the scope of methods used
The scope of methods expands as you progress from CBA to SROI to VfI.
CBA is a quantitative method. That doesn’t mean qualitative considerations aren’t taken into account in its design, nor that qualitative considerations aren’t acknowledged in the report. It’s just that the core of the analysis is a spreadsheet with numbers in it, supported by quantitative approaches to valuing and causal inference. And the output of the analysis is a number, like NPV.
SROI uses qualitative and quantitative methods. It includes the same kind of spreadsheet analysis as CBA. However, it also involves engagement with stakeholders. SROI guidance suggests methods such as interviews, focus groups, surveys, documents review and impact mapping to understand what changes and the value that stakeholders place on those changes. It emphasises that the number alone isn’t the story, and balances qualitative, quantitative and monetary aspects of describing value.
VfI is a mixed methods approach. It doesn’t prescribe the mix of methods; rather it can accommodate any mix. VfI contributes a framework for selecting and using a situationally-informed mix of methods and tools to answer VfM questions. The aim of combining different evidence sources is to provide a deeper understanding of the policy, program or organisation than we might get through the use of a single method.
Myth-busting: Perceived differences that aren’t really differences
It is sometimes perceived that SROI incorporates a broader range of benefits than CBA. Not true. The nature of benefits that are included (e.g., social, environmental, economic, financial) depends not on which method is used but how they are used. Deciding which benefits are in and out of scope is an important aspect of study design under either method, and either method can accommodate the same scope of benefits and costs. When deciding which benefits and costs to include in a CBA or SROI, there can be trade-offs, e.g., between comprehensiveness and reliability. Not everything of value has a precise monetary valuation. Determining what values are relevant, material, and feasible to estimate to a reasonable degree of accuracy, precision and credibility is an important judgement call whichever method is used.
It is sometimes perceived that SROI is “overly-optimistic CBA”. In my view, a good CBA and a good SROI should be conducted to the same standards of rigour and transparency, and should reach equivalent conclusions. If a CBA and SROI reach different conclusions about the same investment, it would reflect differences in how well the methods were applied. SROI has introduced systematic analysis of costs and benefits to a wider audience than those trained in economic evaluation. This is positive, but also brings risks of variability in the way it is practiced. A recent article on SROI highlighted some deficiencies in the way some SROIs have been carried out, including problems with approaches taken to causal attribution and valuing impacts. These issues are not unique to SROI (all methods are vulnerable to sub-standard use) and would be addressed if analysts follow established standards of scientific research and economic evaluation.
It is sometimes perceived that CBA is “more objective” than VfI. This is false. Both CBA and VfI (like all forms of evaluation) involve dealing with facts which are value-laden, and values that are also facts. Both approaches necessitate various kinds of judgements. The outcomes of a CBA are influenced by the analyst's decisions regarding the relevant scope of benefits and costs, the perspective of the analysis, the time frame over which benefits and costs are significant, the discount rate, the methods used for monetary valuation, and the interpretation and presentation of the results. These judgements (and more) should all be made transparent in a CBA report. In a VfI evaluation, evaluative reasoning based on explicit criteria and standards offers a transparent and systematic approach for synthesising evidence. It minimises individual bias by clearly and collaboratively defining shared values and making the evaluation process open to scrutiny and challenge. Ultimately, both CBA and VfI offer valid approaches to combining robust evidence with human judgement.
In summary
Cost-benefit analysis is primarily a quantitative method which reports the value of a policy or program based on the present value of impacts compared with the present value of costs, measured in monetary units.
Social Return on Investment provides a similar number to CBA plus a performance story. SROI is a mixed methods approach because it also involves stakeholder engagement, impact mapping, and combining qualitative, quantitative and financial data.
Value for Investment is a system, not a method; it provides a set of principles and a process to help guide contextually-responsive evaluations and select an appropriate mix of methods. Those methods can include CBA or SROI, among others. VfI isn’t a rival to CBA or SROI, but it can help you decide when and how to use them.
VfI is intended for situations when a broader perspective is needed than CBA or SROI alone provide, involving multiple criteria and evidence sources. Explicit evaluative reasoning (using criteria, standards and evidence to make transparent judgements) is the backbone of a VfI evaluation.
While SROI includes stakeholder engagement aimed at understanding what changes, VfI goes further, sharing power with stakeholders and involving them in evaluation co-design, fact-finding and sense-making.
Bottom line
All three approaches can help to answer VfM questions and inform good resource allocation decisions. Whichever method you select, they all require reliable evidence, sound reasoning, transparent reporting, and a healthy dose of evaluative thinking to ensure the conclusions reached are warranted.
To find out more
Thanks for reading!
Where’s this? The first person to correctly identify the location of this photo in the comments below wins a free 12-month subscription to the full Evaluation and Value for Investment archive.
As Sarah Hogan pointed out on LinkedIn, in practice analysts may use elements from all three approaches, so the distinctions are sometimes less clear in practice than they are in theory.
The price of trying to be concise is sometimes being too glib. I should acknowledge that there is a lot of nuance surrounding “the number”. As Aaron Schiff said on LinkedIn, “…the components of that number can be used to tell a story about sources of benefits and costs. It can do equity analysis by looking at benefits and costs for population groups. With sensitivity analysis it can show how the BCR changes in different conditions. Qualitative evidence can feed in to the assumptions, you do have to put dollar values on things but in my view that's a good thing as it forces you to think about relative values. A good CBA is a framework for communication rather than just a spreadsheet”. Thanks Aaron, I agree.
Very informative article. Thank you. I'm guessing that photo is the Sydney Opera House?