Impact valuations for CBA and SROI in the NZ Treasury CBAx tool
New guide by Aaron Schiff, PhD
A couple of months ago I shared a brief guide to data and information requirements for economic evaluation of social interventions, written by my colleague Aaron Schiff, a New Zealand-based independent consulting economist and data scientist.
Now, Aaron has released a new guide to placing monetary values on social impacts, using a database of valuation factors developed for the NZ public policy context. Though the valuation factors in this database are NZ-specific (and are not intended to be used in other countries), general considerations regarding their use apply to similar databases or ‘social value banks’ developed in other countries.
What’s it about?
Whether you’re doing a cost-benefit analysis (CBA) or social return on investment (SROI), you’ll need to identify and quantify impacts that you can attribute to the intervention, and translate those impacts into monetary values.
Monetary valuations are a way of representing the value of impacts to society. By measuring both impacts (benefits) and resource use (costs) in monetary units, the benefits and costs can be directly compared and reconciled, providing an indicator of whether an intervention creates more value than it consumes.
There are various ways of putting monetary values on things. Sometimes the impacts of interest are already measured in money (e.g., profits), making the task relatively straightforward. Other times, monetary proxy values can be based on prices of goods and services that are bought and sold in markets (e.g., market wages for an occupational group). Failing that, various strategies like surveys or experiments can be implemented to reveal what people would be willing to pay for intangible benefits (e.g., increased happiness).
Sometimes, valuations can be ‘borrowed’ (with due caution) from previous studies - and sometimes, a range of borrowed valuations are compiled into a database or ‘social value bank’, promoting ease and consistency of use.
Aaron’s new guide introduces one such value bank, a database of 274 per-unit values of various social impacts. This database is part of a CBA tool published by the NZ Treasury called CBAx, designed to assist NZ government agencies to evaluate the impacts of their initiatives. The CBAx tool helps to standardise the approach to CBA across the public sector by providing a framework for monetising and comparing the societal impacts of different policy options, driving improvements in the consistency and quality of analysis. I recommend this article for discussion of CBAx development and agencies’ practical experiences using it.
In the following brief guide, Aaron outlines the valuation task, introduces the Treasury CBAx tool and takes us through the CBAx database of impact valuations. Do check it out.
External validity of valuation factors
Aaron mentions the importance of checking where the numbers come from and deciding whether they’re reliable and relevant to your context. I’d like to add a few thoughts to this important point.
The concept of external validity is about the extent to which findings of a research study can be generalised or applied to other situations, people, places, or times beyond the specific context of the study. For various reasons, monetary valuations of costs and impacts are highly context-specific. Consequently, the external validity of monetary valuations is notoriously limited, with even small shifts in context potentially making them inappropriate to use.
Contextual differences include:
Geographical variation: Values obtained in one country or region may not be directly applicable to other places because of differences in local economic conditions (like prices of things), policy and regulatory environments, variations in cultural norms and preferences, and other social factors. For these reasons, readers conducting CBAs in countries other than NZ shouldn’t use valuations from the CBAx database. These valuations are only intended for use in NZ.
Temporal factors: Valuations from past studies may reflect conditions that existed at a different time. We can inflation-adjust estimates to bring them up to the current day. However, we may not be able to so easily adjust for changes in purchasing power, shifts in societal values and preferences, or technological developments that affect costs or benefits.
Program-specific factors: Differences in intervention design and delivery, target populations, project scale, scope, duration, and other factors can significantly affect per-unit values of costs and benefits. Policies and contexts have so many dimensions that we can’t expect any two to be equivalent.
Even when using recent valuations derived from a relevant location, it is always worth being aware of the potential for valuation factors to be used out of context. The CBAx database provides links to source documents and, as with any proxy valuations from any source, it is important to understand how specific estimates were derived before you use them.
This isn’t a reason not to use the CBAx database or other values banks. It is a reason to tread carefully and understand the data you’re using, just like any evaluation ever. If there’s some uncertainty around precise valuations, sensitivity analysis should be undertaken to see to what extent increasing or decreasing the assumed valuations affects the end result.
Bottom line
There’s a lot of interest in SROI and CBA in NZ right now. This guide is a timely contribution to help government and non-government organisations develop skills in estimating costs and benefits of their interventions.
Download a free copy
Schiff, A. (2024). Impact valuations for CBA and SROI in the NZ Treasury CBAx tool. Schiff Consulting. https://www.schiff.co.nz/en/resources/impact-valuation/
Where’s this?
If you’re the first person to correctly identify the location of this photo in the comments below, and explain how you know, I’ll shout you a free 12-month subscription to the full Evaluation and Value for Investment archive.
Nairobi National Park, Kenya.
I've guessed it based on a bit of googling.