Maximising the future public value of government spending

Value for Investment in the UK Policy Context

Last week, Verian Group launched our new Guide to Evaluation of Value for Money in UK Public Services. Today I’m going to share and expand on a few excerpts from the guide.

The following sections include verbatim or slightly modified passages. Please cite the original source document:

King, J., Hurrell, A. (2024). A Guide to Evaluation of Value for Money in UK Public Services: Why cost-benefit analysis alone may be insufficient to evaluate VfM, and how to navigate a solution. Verian Group. 

In a nutshell

Cost-benefit analysis (CBA) contributes to value for money (VfM) assessment, providing insights that would otherwise be difficult to gain. In particular, systematic and rational analysis of costs and benefits enables the value of a policy or program to be reduced to a single indicator, communicating value in a straightforward and compelling way.

However, CBA may not suffice to fully answer VfM questions about complex public investments. For example, VfM assessment often involves balancing multiple criteria, whereas CBA assesses performance against a single criterion. The scope of a CBA may be constrained by what is measurable and monetisable, while in practice some of the most valuable social outcomes may be hardest to value monetarily to a reasonable degree of accuracy and precision. If we only did a CBA, we could overlook some important considerations. These limitations also apply to Social Return on Investment (SROI).

The Value for Investment (VfI) approach offers a way forward. By integrating CBA (and other economic methods and concepts) within a broader evaluative framework, the VfI approach offers a more comprehensive assessment of policies and programmes, drawing on the strengths of CBA without limiting the evaluation to economic metrics.

The UK policy context

This guide was written from a UK perspective, though the general principles about integrating insights from economic analysis with other evaluation criteria and evidence apply anywhere.

VfM has become an increasing focus for governments around the world. VfM assessment is important for accountability and transparency in the use of public resources as well as for learning, improvement, and communicating the value of policies and programmes. Ultimately VfM is important to inform good resource allocation decisions that maximise future public value.

There is no universal definition of VfM. However, in the UK, the National Audit Office (NAO) has defined VfM as “the optimal use of resources to achieve the intended outcomes”. The new UK government is establishing an Office for Value for Money to place VfM at the heart of spending decisions.

Green Book

CBA plays a crucial role in the UK policy context, serving as a fundamental tool for evaluating and appraising public policies, programmes, and projects. HM Treasury’s Green Book, which provides central government guidance on appraisal and evaluation, is the primary resource for conducting CBA in the UK. This comprehensive guide outlines the methodology for assessing the costs, benefits, and trade-offs of various policy options, ensuring that public resources are allocated efficiently and effectively. The Green Book emphasises the importance of considering both monetary and non-monetary impacts, including environmental and social factors, to provide a holistic view of a policy's potential outcomes.

Magenta Book

Complementing the Green Book is the Magenta Book, which focuses on evaluation methodologies and practices. While the Green Book provides guidance on ex-ante (before the event) appraisal, the Magenta Book offers insights into how to design, conduct, and interpret evaluations throughout the policy cycle with the aim of assessing whether expected impacts and benefits have materialised. Together, these two resources form a robust framework for evidence-based policymaking in the UK (and influence other countries including my home NZ), ensuring that CBA is not only used to inform initial decision-making, but also to assess the actual impacts of implemented policies. This comprehensive approach allows policymakers to refine and improve interventions based on empirical evidence, and ultimately should lead to more effective and efficient public policies.

There’s more to value for money than cost-benefit analysis

In the UK public policy context, VfM is often framed as an evaluation objective or question within a wider evaluation. For example, terms of reference might call for an evaluation of processes, outcomes, and VfM. In this setting, the term ‘value for money’ is often used almost interchangeably with economic evaluation methods, such as CBA (I see this often in Australia and New Zealand too).

However, VfM and CBA are not synonymous. VfM can be assessed in various ways including but not limited to CBA. We argue that for public investments it is often be too limiting to rely on CBA alone. For example, CBA assesses performance against a single criterion (the monetised value of impacts relative to costs) whereas VfM assessment often balances multiple criteria that can relate to different stages of a policy or program. Moreover, the scope of a CBA may be constrained by what impacts can readily be measured and monetised. While the output of a CBA can provide valuable evidence, it may not be sufficient on its own to provide a comprehensive VfM assessment.

We argue that to effectively evaluate VfM and maximise the future public value of government spending, the UK public sector should integrate insights from a range of evaluation tools in conjunction with CBA - taking a broader view of value than CBA alone can provide.

VfM is multi-dimensional

The NAO and the Foreign, Commonwealth & Development Office (FCDO) appear to agree, recommending a combination of up to five criteria (“the five Es”) to assess VfM of government spending:

• Economy (minimising input costs while having regard to quality)

• Efficiency (the relationship between outputs and the resources used to produce them)

• Effectiveness (achieving objectives and outcomes)

• Equity (the extent to which services benefit the intended people)

• Cost-effectiveness (the relationship between resource use and impacts).

Addressing these criteria involves the use of multiple methods. While CBA can sometimes help to address the cost-effectiveness criterion, this isn’t a requirement, nor is it always feasible. Moreover, the inclusion of five criteria necessitates trade-offs and judgements, making VfM a matter of context and perspective.

The five Es are often used when assessing VfM of existing policies and programmes (ex-post evaluation) but are challenging to apply in future-facing (ex-ante) assessments because there may be too many unknowns to assess economy, efficiency and effectiveness of something that doesn’t yet exist. One example of a set of future-facing criteria is the five case model for business cases:

• Strategic case (alignment with organisational priorities, goals and objectives)

• Economic case (net social value of benefits and costs, often assessed using CBA)

• Commercial case (viability of the initiative in terms of capacity, capability and feasibility for third-party suppliers to deliver)

• Financial case (financial implications including costings, funding sources and affordability)

• Management case (delivery strategy, governance, risk management and operational plans to ensure successful implementation).

The five Es and the five cases are just examples. Different criteria may apply depending on the situation.

Criteria are necessary, but not sufficient. A good evaluation framework also needs standards, defining a transparent basis for distinguishing (for example) excellent, good, adequate, and poor performance against the criteria. Our approach describes how to develop and use criteria and standards in VfM assessment.

The contribution of this guide

Verian’s new guide focuses on the interface between CBA and the full suite of evaluation methods in the context of UK public policy, including:

• How CBA contributes to VfM assessment

• Why CBA alone may not suffice

• How to design and conduct a Value for Investment (VfI) evaluation that matches methods to context, incorporating CBA where feasible and appropriate

• How to use VfI as an alternative when CBA isn’t possible.

The VfI approach is designed to bring clarity to evaluating how well resources are used, the extent to which value created justifies the resources invested, and how more value can be generated from public investments.

VfI hinges on defining agreed criteria and standards that are used to make transparent judgements from quantitative and qualitative evidence.

Criteria and standards: a logic, not a tool - and always there, whether explicit or not. In professional evaluation involving judgements by, with, or for people affected by policies and programs, conclusions are strengthened by involving stakeholders and by making the logic transparent. All explained in this article.

The four principles and eight steps of the VfI process are summarised in the new guide, and each week I explore different aspects of the approach in depth right here on Substack.

Real-life examples of VfI in action

The Value for Investment approach can be used to integrate insights from CBA with wider evidence and criteria. VfI also offers a way to answer VfM questions in situations where CBA isn’t feasible. Here’s an example of each situation. More examples are available on my VfI resources page.

VfI evaluation that included CBA within a mix of methods

This series of social and economic impact assessments for the International Atomic Energy Agency (IAEA) illustrate the use of the VfI approach including criteria and standards designed collaboratively with stakeholders.

CBA, including break-even analysis, was combined with surveys, case studies, key expert interviews, and administrative data - providing insights that would not have been gained from the use of CBA alone.

There’s a worked example on my website, providing a step-by-step demonstration of the VfI system in action.

The reports are available here:

King, J., McKegg, K., Arau, A., Schiff, A., Garcia Aisa, M. (2020). Social and Economic Impact Assessment of Mutation Breeding in Crops of the RCA Programme in Asia and the Pacific. International Atomic Energy Agency, Vienna.

King, J., Arau, A., Schiff, A., Garcia Aisa, M., McKegg, K. (2022). Social and Economic Impact Assessment of the RCA Programme: Radiotherapy Case Study. International Atomic Energy Agency, Vienna.

King, J., Arau, A., Schiff, A., Garcia Aisa, M., McKegg, K. (2022). Social and Economic Impact Assessment of the RCA Programme: Non-Destructive Testing Case Study. International Atomic Energy Agency, Vienna.

VfI evaluation that provided a viable alternative to CBA

Sometimes, for various reasons, a CBA may not be feasible. This example from New Zealand shows how the VfI approach can provide answers to VfM questions in these circumstances.

This evaluation of the national rollout of a Youth Primary Care Mental Health and Addictions Initiative was conducted early in the life of the services and before quantitative outcome measures were implemented. The mix of methods used in the evaluation included surveys, interviews, documents review, and analysis of service output data.

The evaluation report is available here:

Field, A., Crocket, A., Garden, E., King, J., Moss, M., Parslow, G., Schiff, A., Spee, K., Wehipeihana, N. (2023). Youth Primary Mental Health and Addictions Evaluation. Final Report for Te Whatu Ora – Health New Zealand. Dovetail Consulting Limited, Auckland.

In addition, the evaluation team was commissioned to prepare a VfI guide, detailing the application of the approach and a road map for future evaluations and capacity building in the sector. The guide is available here:

King, J., Crocket, A., Field, A. (2023). Value for Investment: Application and Insights. Youth Primary Mental Health and Addictions Evaluation. Exemplar report for Te Whatu Ora – Health New Zealand. Dovetail Consulting Ltd.

Bottom line

The principles and process of the Value for Investment approach build on sound theory and practice from evaluation and economics. Value for Investment isn’t another method – it’s a system to guide the selection and contextually-appropriate use of existing evaluation methods to answer value-for-money questions. For this reason, it is applicable to any setting, ex-ante or ex-post, where the objective is to address evaluative questions about resource use and value creation.

Grab your copy of the new VfI Guide here

Free download:

King, J., Hurrell, A. (2024). A Guide to Evaluation of Value for Money in UK Public Services: Why cost-benefit analysis alone may be insufficient to evaluate VfM, and how to navigate a solution. Verian Group. 

Also see:

Wate, D. (2024). Could an Office for Value for Money embrace Value for Investment?

Disclaimer

The views expressed in this article are solely my own and do not necessarily reflect the opinions or positions of any organisations or individuals mentioned.

Thanks for reading!

Where’s this? The first person to correctly identify the location of this photo in the comments below wins a free 12-month subscription to the full Evaluation and Value for Investment archive.