CBA as a mixer

How to build systematic analysis of benefits and costs into mixed methods evaluation

I’ve argued that insights from social cost-benefit analysis (CBA) are important and useful but aren’t enough on their own, and should be combined with broader evidence and criteria to guide evaluation and decision-making.

Did you know that some other public policy experts and economists think so too?

Matthew Adler & Eric Posner (2006), in New Foundations for Cost-Benefit Analysis, found CBA to be an imperfect but useful decision procedure, which estimates and aggregates gains and losses in utility but doesn’t incorporate “the totality of moral considerations”. Even Cass Sunstein (2018), a strong proponent of CBA, while arguing in The Cost-Benefit Revolution that CBA provides essential information, acknowledged that it may be incomplete:

"Whether or not an analysis of costs and benefits tells us everything we need to know, at least it tells us a great deal that we need to know. We cannot safely proceed without that knowledge.” (Sunstein, 2018)

CBA is just one input into decision-making… but how to combine it with other considerations?

Bent Flyvbjerg and Dirk Bester (2021) in their article, The Cost-Benefit Fallacy, argued:

"For cost-benefit analysis to be accepted and have impact, it must be understood... in ways that fit with the messy, non-expert character of present-day democratic decision making. Here cost-benefit analysis is just one of many inputs that are amalgamated into the overall decision-making process." (Flyvbjerg and Bester, 2021)

Similarly, Adler and Posner (2006) recognised that there are considerations beyond those captured in CBA. They concluded that "CBA is not a superprocedure" for combining all relevant values and acknowledged the need for a comprehensive approach to integrate CBA outputs with wider considerations. However, they conceded:

"We suppose that that is a theoretical possibility - but we have absolutely no idea what the superprocedure would consist in." (Adler & Posner, 2011)

Evaluators know how!

The field of program evaluation offers methods and tools to address this need. The "superprocedure" that eluded Adler and Posner is explicit evaluative reasoning and is central to the theory and practice of evaluation. It involves using criteria (aspects of performance that matter to people) and standards (defined levels of performance) to make transparent judgements from evidence. It provides the means to make transparent value judgements based on multiple values and types of evidence - economic, quantitative, and/or qualitative.

We’ve unpacked it all in this handy guide

In the following guide, we unpack how to design and implement mixed methods evaluations that incorporate CBA, underpinned by explicit evaluative reasoning:

King, J. & Hurrell, A. (2024). A Guide to Evaluation of Value for Money in UK Public Services: Why cost-benefit analysis alone may be insufficient to evaluate VFM, and how to navigate a solution. Verian Group.