Getting to the real value in ‘value for money’
A muppet fable featuring frogs, pigs, colonisation, self-determination, cost-benefit analysis, and rubrics
Once upon a time, there was a beautiful country, inhabited by beautiful people.
They were strong, healthy, they knew where they were from, and they knew what mattered to them. Let's call them the froggies.
I would have loved to include illustrations but for copyright reasons, have opted for written imagery. Here, please imagine a peaceful scene with Kermit and his family and friends sitting around on lily pads, living their best lives.
Then across the sea came a tribe of strangers with a puffed up view of their place in the world.
These piggies brought with them beads and blankets; guns, germs and steel; and high fructose corn syrup.
Imagine a Viking longship full of muppet pigs with fancy clothes and supercilious expressions.
They imposed new worldviews and ways of doing things. And all was not right with the world.
The people of the land fared poorly for the next couple of centuries.
But they were a resilient and resourceful people. They never forgot who they were, where they were from, or what mattered to them.
And in time, the tide started to turn. There was a renaissance of froggy ways of doing and being.
Imagine Kermit with a guitar, singing a traditional froggy song under a moonlit sky.
A great froggy leader saw a need for a new school.
Some of the young frogs had potential but they were at risk of squandering it, as youngsters are apt to do.
You see, the piggy school system largely assumed you would hang your froggidom on the coat hook as you came in the door. So the learning experience felt discordant and uncomfortable for the young frogs. Plus, it was boring. So they made their own fun, and chaos ensued.
So this great froggy leader came, and said to the children, "I'm going to set up a new school. And it's going to get you through your exams with flying colours. Not only that, we'll make sure you come out with a strong sense of who you are, where you are from, and what matters."
"And", she said, "you'll leave school equipped to be part of the next generation of leaders, not just making your own way in the world but working for the good of all froggidom."
And off they set, helped by a bit of seed funding from some benevolent piggies.
Imagine Miss Piggy dressed as a society lady, with pearls and an expensive handbag. She likes helping people less fortunate than herself, expects all traffic to yield to her Range Rover, and she parks anywhere she pleases.
But there was a catch.
With the funding came a requirement to evaluate the new school. The piggies wanted to know if they were getting value for money.
The benevolent piggies even said they knew just the right person to do the evaluation. Clever piggies. They sent in...
An economist.
Apologies to my economist friends for what happens next.
This is a caricature. I appreciate your work. In fact, I encourage evaluators to use economic thinking more in evaluations. Nonetheless, let’s have a bit of fun with this…
Casting this role was a conundrum because there were so many worthy candidates. The Chef would have been great because you know he’s trying to tell you something important, it’s just in another language. But there can be only one winner and in the end I chose Sam Eagle for his stern humourless vibe.
The economist met with the froggies.
Well, he Zoomed in. More efficient. Also, his tummy disagreed with froggy food. So he thought it would be best if he stayed home. And anyway, he already knew what to do. He'd spent six years earning a PhD in economics. He knew that all he needed was for the frogs to email him their data, and he could take it from there. All he needed to do was work out the value of the resources used, and the value of the outcomes. Dollars in, dollars out. The spreadsheet would do the rest. Voila.
Valuing the resources used was easy. The piggies provided a cost centre report, and that was that.
But wait a minute!
It took more than just money, surely, to get this school up and humming? The piggy donors were of course keen to monitor the use of funds. The cost centre report showed how much money was spent and maybe even a few clues about how well it was used. But there’s more to good resource use than money alone.
What other resources were invested? Well, take a moment to imagine the great leader and think about the vision, charisma, networks, knowledge, skills and sheer tenacity it took to create the school. And the amazing team she built around her, who had deep cultural roots and knew how to engage the hearts and minds of the young frogs. Are those not essential ingredients of success, limited in supply, with their own opportunity costs?
The economist reasoned that since salaries paid for all those things, their value was already represented in the cost centre report. Salaries are, after all, negotiated in a labour market. They represent a willing exchange between a buyer (the employer) and seller (the worker). Therefore, he argued, market salaries would adequately represent the agreed value of the leader and her team.
But that only gives us a number to represent how much was spent on some essential ingredients of success. Whether or not you agree with the argument that it represents their value, it tells us nothing about how well these important resources were invested. I don’t know about you, but I think that looking after the intangible resources of the team (relationships, culture, know-how, morale, etc) would be crucial to getting good value out of the investment in the new school. I would want to know how well those resources are being nurtured, coordinated, and leveraged.
Never mind, let’s move on to outcomes.
For the economist, outcomes were fairly easy to calculate too. The new school was recording student exam data. The achievement of froggy students in mainstream schools provided a ready point of comparison. Using clever statistics, the economist estimated the difference the new school was making to educational outcomes. Then he turned that difference into dollars, using published rates of return on education.
The results were impressive.
The young frogs had passed their exams with excellent grades. Many had enrolled in piggy universities, with aspirations of becoming lawyers, architects and veterinary surgeons. Think what those froggies would earn! Think of the value they would add to the economy!
There were other outcomes too.
There wasn't any real data, but the frogs spoke passionately about cultural identity and self-determination. The economist thought it sounded a bit tangential to value for money, but it seemed important to the frogs.
And he was a conscientious economist, so he went back to the piggies and negotiated a change of scope. And then, he did something amazing. He left his office and visited the frogs!
Imagine 20-30 frogs in a meeting room, bemusedly regarding an uncomfortable eagle.
And he sat with them, and he did something called a willingness to pay study. The study worked like this: It said, here's your precious outcomes: your children feeling proud of who they are, connected to their cultural roots and ready to become future leaders...
... and over here, is a bag of money.
Now, how big would I have to make the bag of money, for you to go without your children's educational outcomes?
To the frogs that question was, frankly, offensive. Like, how could you put any kind of monetary value on these priceless outcomes without cheapening them? But they were polite and went along with it.
Now, these frogs were not wealthy, and this bag of money, though modest, seemed quite big to them. To the economist, who was accustomed to working with lots of zeroes, it seemed as if the frogs didn't really value the cultural outcomes much at all. "Thought so", said the economist.
The economist's report came out.
Drum roll please…
It showed the new school was indeed value for money! This was mainly because of the high rates of return on education and the positive exam results the students had achieved.
The frogs were glad the school met with the economist's approval. But somehow they felt short-changed and a little deflated. They felt as if the study had missed the point.
Imagine many frogs looking nonplussed. You know that Kermit face when Jim Henson screws up his fist? All the frogs looking like that.
The economist had valued the froggy school using piggy values.
And some of the most valuable outcomes, like the cultural outcomes that differentiated the new school from regular ones and were most prized from a froggy perspective, had somehow been lost in translation.
But the great leader knew what to do. She made a phone call, and brought in...
An evaluator.
I was unsure who to cast in this pivotal role. My favourite muppet is Animal. He seems open to anything, which is a good trait for an evaluator. But he’s also a bit unpredictable, so perhaps we’d prefer a more calming character like Janice or Rowlf. Nominations are welcome. Which muppet best represents evaluators? Choose your favourite.
The evaluator understood the power of the economist's work.
So she kept that. But she could also see that it wasn't quite enough. She thought she could see a way to enhance it.
The evaluator knew that valuing didn't always have to be done with dollars. Using the logic of evaluation, she could work collaboratively with the froggies and the piggies to unpack what good value would look like from different perspectives, and use this information to help make sense of the evidence and understand the value of the school.
Now, this evaluator wasn't a frog.
But the frogs had invited her into their space, and she was there on the frogs' terms. And, she surrounded herself with froggy experts. She respected, and was guided by their values.
First, the froggies and the piggies sat around a table together with the evaluator. They shared food and stories, got to know each other a bit, then they got down to the serious business of working out what matters.
The evaluator facilitated a series of evaluation design workshops.
Areas of commonality were found. There were also some aspects where different groups agreed to accommodate each other's perspectives, in order to have a basis of valuing that reflected and respected the diversity around the table.
Then the hard work really started. Oh wow, this was a brain workout. But through this process, they cemented relationships, understanding and trust. The piggies and froggies came away with a new appreciation of how the other saw the world. Together, they had developed a shared understanding, neatly summarised in a rubric - an agreed definition of good value for money in the context of the school.
Imagine a rubric.
The evaluator and the frogs worked together to gather evidence.
This included the economist's findings, plus other evidence - numbers and stories. The process was guided by the content of the rubric (signposting what aspects of performance to focus on and what kinds of evidence would be necessary and credible), and froggy cultural values regarding appropriate ways of working to collect the evidence.
The evaluator facilitated a process that brought the frogs and pigs back together.
They sat around the table, shared food and stories, and sifted through the evidence together, using their co-constructed rubric as a guide to making judgements.
Findings were reported in a range of ways that honoured and validated froggy and piggy values. The students produced an artwork that depicted their journey. One of the parents wrote a song about her gratitude to the school. A short film was produced and posted online, telling the story through the eyes of two families.
Of course, there was also a Word document in 12-point Calibri, with nifty data visualisations and weighty appendices. And a short, visually appealing HTML summary of key findings.
Everybody endorsed the report.
Frogs could see their values and experiences represented in the findings. They felt validated. The evaluation process had given them the opportunity to reflect on the evidence and identify opportunities to keep improving the school. The piggies had the information they needed to allocate more funding to scale the impact and value of the school.
And they all lived happily ever after.
There's a serious side to this.
Imagine the whole cast looking straight into the camera, with folded arms and solemn faces.
Cost-benefit analysis is valid and useful, but isn’t enough on its own. It provides pieces of evidence that need to be considered alongside other pieces. Value for money is more than a benefit-cost ratio or even a social return on investment.
I've argued that value for money is an evaluative question (concerned with merit, worth and significance) about an economic problem (resource allocation).
In other words: How well are we using resources? Is the resource use justified? How can we improve resource use?
Value for money is more than just efficiency. For example, it can also be defined in terms of stewardship, relevance, equity, ethics, sustainability, and more. Balancing multiple criteria can involve trade-offs. Good resource use is a matter of context and perspective.
Determining value for money requires a judgement to be made. Not a 'subjective' judgement, but a reasoned, explicitly evaluative judgement, using evidence and explicit values. Evaluators know how!
Muppets take a bow, curtain closes.
Free Value for Investment resources
VfI resources page including theory, practice, and real-world examples: www.julianking.co.nz/vfi/resources/
King, J., Hurrell, A. (2024). A Guide to Evaluation of Value for Money in UK Public Services: Why cost-benefit analysis alone may be insufficient to evaluate VfM, and how to navigate a solution. Verian Group.
King, J., Wate, D., Namukasa, E., Hurrell, A., Hansford, F., Ward, P., Faramarzifar, S. (2023). Assessing Value for Money: the Oxford Policy Management Approach. Second Edition. Oxford Policy Management Ltd.
King, J., Crocket, A., Field, A. (2023). Value for Investment: Application and Insights. Youth Primary Mental Health and Addictions Evaluation. Exemplar report for Te Whatu Ora – Health New Zealand. Dovetail Consulting Ltd.
