Communicating value

The headline travels around the world while the real story is pulling its boots on

Imagine a program was evaluated by two different consultants at the same time, Their reports reached very different conclusions.

Compare these two statements:

1. Early observations from the program indicate a range of promising trends and positive results. While these results suggest potential benefits, further evaluation is needed to fully understand the sustainability and long-term impact. (Report 1)

2. For every dollar spent, the program creates $10 worth of social value.¹ (Report 2)

Now consider: which is the most memorable, and most likely to be quoted by news outlets and politicians?

Report 1’s conclusion represents the evaluator’s judgement about what can reasonably be claimed from current knowledge. But it’s not very memorable - and to decision-makers, it can seem like “sitting on the fence”.

On the other hand, Report 2’s conclusion is bolder, but it masks the reality that it’s early days and the evidence is still building. The 10:1 benefit-to-cost ratio includes estimated future value, explored through scenario analysis, based on evidence and assumptions.

Our understanding is strengthened by considering both perspectives together - and in reality, a good report would include both. For example, it might say:

• Early observations indicate promising trends and positive results. Scenario analysis suggests that for every dollar spent, the program may be able to generate $10 worth of social value. Further evaluation is needed to fully understand sustainability and long-term impact.

But something important is missing

The report lacks a final evaluative judgement - like, is this a good enough result for the circumstances? Is it more than good enough? Or less?

As I wrote here, you could present “just the facts” without making a judgement. But:

• That’s not evaluation, because like a Hawaiian burger without the pineapple, it misses the essential feature that would make it so; and

• It’s kicking the judgement can down the road - because somebody, somewhere, like a politician or bureaucrat, will still make their own judgement, but they’ll do it without the benefit of a clear answer from an evaluator who engaged with affected people and understands what matters to them.

The judgement I’m referring to is more than a mere opinion - we need a warranted judgement, made on a transparent and defensible basis. The credibility of the judgement is enhanced when it is supported by a logical chain of reasoning and reflects the values of stakeholders.

Now consider this third statement:

3. In its establishment phase, the program is performing at an excellent level, with results meeting the agreed threshold for excellence according to current evidence and projected scenarios.

This statement presents an evaluative judgement that can be communicated in:

• One word (excellent);

• One sentence (statement 3 above); or

• An entire report, structured with: the judgement at the front; transparent reasoning in the middle connecting the claim to evidence and rationale; and a detailed account of the methods and evidence used at the end.

ROI is valid but insufficient

The evaluative judgement is informed by the findings from a cost-benefit analysis (CBA) or social return on investment (SROI). But that’s more than just the headline “return on investment” (ROI) figure. The number alone isn’t enough because:

1. It only counts some benefits and costs - the ones that were feasible to value in dollars

2. It addresses one criterion, whereas we often apply multiple criteria to judge the worth of an intervention

3. It privileges quantitative measurement and monetary valuation over other forms of evidence

4. It’s an abstraction that loses any resemblance to what actually matters to people whose lives are affected.

Besides the headline number, an evaluative judgement should incorporate extra considerations like:

• Uncertainty (e.g., the “$10” figure might represent the midpoint of “somewhere in the vicinity of $5-15”)

• Risk (e.g., the estimated future value will only be realised under particular conditions that aren’t guaranteed)

• Distributional impacts

• Any significant intangible (non-monetisable) benefits and costs that aren’t reflected in the number

• As many additional criteria and evidence sources as are necessary and sufficient to represent stakeholder values and make a well-informed judgement.

These extra considerations may be included in a good CBA or SROI report, but they tend to lose visibility in the final communication of value.

Moreover, the ROI number is all some decision-makers are interested in.

How to counter the ROI fixation?

I share the frustration of my colleague who wrote:

…when I speak about MUVA I focus on our commitment to agency, especially of women’s. Creating conditions for them, for us, to have a voice that is heard, choices, and control over our lives, bodies, and income. This is what keeps us up at night. This work is intricate, messy, not linear and hardly fits into a number.

The only comment from the donor was “Interesting approach, but we’re looking for a ROI of 10”.

It hurt. I was speechless (and that’s rare!)…

What will it take to shift the conversation from ROI to something more comprehensive?

Long-term, it requires culture change around what success looks like, what counts as evidence, and whose values matter. My whole Substack series represents part of my effort to contribute toward this culture change.

In a more immediate sense, let’s get into the habit of providing a punchy evaluative claim - as sticky as the number but with broader meaning behind it - that can be communicated in one word, one sentence, or unpacked more to satisfy the questioner:

Program is [good] because [evidence] and [reasons].

This claim can include ROI, of course - and more.

And in the heat of a frustrating conversation with a funder who only wants “10”, let’s open up the conversation with a Socratic question: What might be missing if we focus only on ROI?

For the avoidance of doubt:

I’m not saying that the ROI number isn’t valid (it’s a unique and important piece of evidence). I’m saying value is more than a number alone.

I’m not saying CBA or SROI are only a number (they’re much more - a whole journey of discovery and a qualitative story). I’m saying the number travels while the full value story is pulling its boots on.

I’m not saying that you can’t interrogate the number (you can - a good CBA or SROI is transparent and shows its working). I’m saying the number travels light, leaving assumptions and caveats behind. Most readers (including policy makers) won’t interrogate very far beyond the key findings.

I’m not saying that CBA and SROI don’t provide an overall judgement (they do - or it’s widely agreed they should). I’m saying that the judgement shouldn’t just be the analyst’s opinion - it should be scaffolded by explicit criteria and standards, developed and applied with stakeholders, to provide a traceable logic from the evidence to the conclusion.

Bottom line

Value can be communicated in a way that is both direct and nuanced. It takes more than a number to do it.

CBA and evaluative claims aren’t rivals; they’re natural partners. Incorporating CBA within a mixed methods evaluation, guided by explicit evaluative reasoning and stakeholder participation, strengthens our assessment of VfM and can still communicate value in a way that gets straight to the point and fits in a headline.

For a deeper dive into the reasons why CBA alone isn’t enough and what to do about it, download our free guide.

“A lie can travel halfway around the world while the truth is putting on its boots”

Jonathan Swift wrote in 1710:

Falsehood flies, and the Truth comes limping after it … so that when Men come to be undeceiv’d, it is too late; the Jest is over, and the Tale has had its Effect.

This phrase aptly describes what can happen when “the number” (like a benefit-cost ratio) travels unburdened by the full truth of the assumptions, uncounted intangibles, uncertainties, and other caveats that sit behind it.

Ironically, the phrase itself befell the same fate: a more modern version, along the lines of, “a lie can travel halfway around the world while the truth is putting on its boots”, is widely but mistakenly attributed to figures like Mark Twain, Winston Churchill, or the Baptist preacher Charles Spurgeon. None of these attributions is substantiated by credible historical evidence.

It appears that Jonathan Swift’s quote is the earliest source of the sentiment. More than a century later, an 1820 Maine newspaper wrote, “Falsehood will fly from Maine to Georgia, while truth is pulling her boots on”. By 1859, Charles Spurgeon quoted the line as an “old proverb”, demonstrating that it was already well-known and in use before his sermon.

Acknowledgements

I’d like to thank Heidi Peterson for peer review. Errors and omissions are mine. These opinions are mine alone and don’t represent people or organisations I work with.

Also see:

Ten Commandments for Social CBA and SROI

Getting to the real value in ‘value for money’

Beyond the hierarchy

Ratios as VfM indicators

Classic 2-storey semi-detached New Zealand state house (illustration created in PowerPoint by the author). This design proliferated in the mid to late 1940s as part of NZ’s postwar state housing expansion. The multi-unit form was introduced to address housing shortages, making efficient use of land and materials.

1. This number is formally known as a benefit-cost ratio (BCR). Sometimes you’ll see it referred to as a SROI ratio, or sometimes informally as just “ROI” - though, technically, for the pedants among us, ROI refers to a different number.