This post launches my new workshop, Navigating Economic Methods of Evaluation. Designed for evaluators and evaluation users who want to make better use of cost-benefit analysis, social return on investment, cost-effectiveness analysis and cost-utility analysis, it equips you to choose and discuss method options, critically review and interpret economic evidence, and work productively with economists in a wider evaluation. Available online as two 3-hour sessions or face to face as a one-day workshop. Adaptable to your sector, programmes and live decisions. Bookings are now open for 2027.
“This programme returns $6.20 for every dollar spent.”
You’re in a meeting about a programme that has just been evaluated. It supports young people’s mental health and tackles entrenched inequities in access to services.
A consultant presents their report, which features an authoritative-sounding number: “For every dollar invested, this programme generates $6.20 in social value”.
It seems impressive. But where on Earth did the number come from? How does improved mental health translate into $6.20? How can you tell if the analysis is solid?
Some of the questions I might ask next include:
From whose perspective is the study conducted? Does it represent costs and benefits to young people, their families, taxpayers, or society as a whole?
What would plausibly have happened without the programme, and how confidently can any difference be attributed to it?
What benefits have been included in the monetary valuations, and were any important effects left out?
What key assumptions and uncertainties does the $6.20 hinge on, and how would the result change if those assumptions were varied?
Is this number enough to conclude that the programme is good value for money? If not, what else would we need to know?
Questions like these, and knowing what else to look for, are crucial to determining what the $6.20 actually means and doesn’t mean.
A benefit-cost ratio can be a useful piece of evidence, helping us think systematically and rationally about resources, consequences and alternatives. But it is not, on its own, an evaluative judgement.
A programme might show benefits greater than costs in a social cost-benefit analysis and still not adequately answer questions about legitimacy, cultural fit, implementation quality, lived experience, rights, or outcomes that cannot credibly or appropriately be valued in monetary terms. The calculation may also conceal how benefits and costs are distributed, or rest on assumptions that are weaker than the headline number suggests.
To use economic evaluation methods well, we need to understand what a particular method can tell us, what it doesn’t tell us, and how to integrate the findings with other evidence and aspects of performance.
No gold standards in evaluation
I was in a packed room when an eminent Professor of Medicine proclaimed, “the only way to really know if something is good value for money is to conduct a randomised controlled trial, followed by a cost-benefit analysis”.
How would you respond? At the time, I was left a little speechless. It was one of the formative moments that led to my PhD research.
In some policy and evaluation conversations, people speak as if there’s a universal hierarchy of methods; the more experimental and monetised the analysis, the better.
A randomised controlled trial (RCT) can provide strong evidence about causal effects in some circumstances, and cost-benefit analysis (CBA) can provide useful evidence about costs, consequences and trade-offs. In some cases, used together, they can provide powerful evidence. But neither method is automatically necessary, desirable, feasible or sufficient for every evaluative question or decision.
No method is superior in all cases. Different methods (economic, quantitative, and qualitative) give us different kinds of information, and combining them can sometimes be more informative than relying on one alone. Methods should be chosen for their fitness for purpose and probative value (how well they can support the particular claim or judgement at issue), not because they sit on an abstract hierarchy of what is “best”.
Become a savvy user of economic evaluation
Economic evaluation can sometimes feel like a foreign language, and it has technical conventions of its own. But many of its central questions will be familiar to evaluators, such as: compared with what, for whom, over what period, on what evidence, under what assumptions, with what uncertainty?
There are a few new concepts to learn as well, but the basic questions and reasoning are accessible, especially when they are connected to the evaluation questions people already work with. Many of the people who need to make good use of economic evaluation are not, and do not need to become, specialist economists. That includes evaluators, commissioners, programme managers, policy advisers, funders, board members, and community representatives. Their roles differ, but all may need to decide how much confidence to place in an economic claim.
They don’t need to conduct the analysis themselves, but they do need to be able to ask good questions of it, recognise its limits, and use it alongside other evidence when making evaluative judgements.
A new workshop: Navigating Economic Methods of Evaluation
This is a workshop for people who need to commission, collaborate on, interpret or critically use economic evaluation. You will not become a specialist economic evaluator in a day, but you will be better placed to commission, interpret and challenge economic evidence, and to use it alongside other evidence when making a judgement.
We will unpack the main economic approaches, explore the reasoning behind indicators such as benefit-cost ratio (BCR), net present value (NPV) and incremental cost-effectiveness ratio (ICER), and practise interpreting and critically appraising economic claims.
The ratio isn’t the judgement
A BCR explores whether benefits included in an analysis, valued in monetary terms, plausibly exceed the costs included in that analysis, from a stated perspective and under specified assumptions. Useful as that is, it answers a narrower question than whether a programme represents good value for money.
Value for money is a balanced evaluative judgement about how well resources have been used, what value has been created, for whom, and whether the same resources could plausibly have created more value elsewhere. It takes more than a ratio alone.
Before getting from a BCR to a conclusion, I would want to know whether the analysis is transparent and fit for purpose, whether data and assumptions are credible, and what else matters that’s not in the number. A ratio reflects choices the analyst has made. To use it as evidence in an evaluative judgement, we need to understand these choices, what the analysis includes and leaves out, and whether it addresses relevant questions for the decision at hand.
Who this workshop is for
This workshop will suit people with little to moderate prior exposure to economic evaluation, and who already have some connection with evaluation, policy, programme management, commissioning, or decision-making.
The training may be for you if you have:
Been handed a cost-benefit analysis (CBA), cost-effectiveness analysis (CEA), cost-utility analysis (CUA), or social return on investment (SROI) report and wondered how to tell if it’s well designed, well conducted and relevant to your needs
Heard a ratio dominate a discussion that seemed more complex than just a number
Been asked whether a programme is good value for money and felt unsure how to approach it
Needed to advise a client, senior manager or board on which method or combination of methods would be useful
Wondered whether a proposed economic evaluation is feasible, necessary, sufficient, or appropriate
Wanted to collaborate more effectively with an economist while still leading the evaluative thinking.
The workshop uses explanations, discussion, worked examples and practical exercises. It focuses on economic methods rather than the broader Value for Investment (VfI) approach, but shows how the two fit together. There is a separate workshop available that covers the VfI approach in more depth.
I am grateful to Dr Aaron Schiff for peer reviewing the workshop, and to the ARTD Consultants team for taking part in an early pilot session in Sydney. Their questions, discussion and reflections helped me fine-tune the workshop before offering it more widely.
I’m excited to be bringing this workshop to UK-based clients in October, and look forward to scheduling more for the new year.
Bookings and delivery options
I am now taking bookings for 2027. The workshop can be delivered online or face to face, as a full day, two half-days, or another format suited to your setting. It can be commissioned as a private workshop for an organisation, team or network, offered through an evaluation association or conference, or arranged for a smaller group. I can tailor it around your sector, programmes, decisions and live examples.
If this sounds useful for your team, network or event, I would be glad to discuss what format would work best.
Explore this workshop, my other workshops and delivery options.
Get in touch to discuss a 2027 booking.
