Welcome to Evaluation and Value for Investment! Here's a thematic directory of posts

Nov 21, 2022

This series is about getting clear answers to “value for money” questions, so people can make good decisions with limited resources in the real world.

This page is your hub for finding posts on the topic you’re looking for.

How to use this page:

New to VfI: Start with “Value for Investment in an hour”.
Designing or delivering an evaluation: Jump to “Practical VfI guidance” and the thematic library to find posts on rubrics, program theory, methods, and cases.
Explaining VfI to others: Use the “Explaining VfI to stakeholders” section when preparing board papers, slide decks, or conversations with colleagues and decision-makers.

It’s free to subscribe. The foundational articles below are free, and new articles are free for the first two months after posting. Paid subscribers get access to the full archive with advanced deep dives, worked examples, and tools.

Value for Investment in an hour

I developed the Value for Investment (VfI) approach through doctoral research to bring greater clarity to questions about how well resources are used, whether enough value is created, and how more value may be created from investments in policies and programs. VfI is my contribution to better value-for-money (VfM) assessment.

If you want the essentials in an hour, start here:

Getting to the real value in “value for money” - start here for a playful but serious introduction to the VfI approach: a Muppet fable featuring frogs, pigs, colonisation, self-determination, and better answers to VfM questions.
Using Economic Methods Evaluatively - a plain-language summary of my 2016 journal article, marking the formal starting point for the ideas that became the VfI approach.
Better VfM assessment - a concise overview of the four key principles and eight-step process that underpin the VfI approach.
It Depends podcast - if you prefer audio, this conversation with Matt Healey and Tenille Moselen covers how I stumbled into evaluation and the path to VfI.
What’s the value proposition of the VfI approach? - applies a value-proposition lens to VfI itself: who the approach is for, how it delivers value, and what it looks like to work in fidelity with VfI principles.

Practical VfI guidance to get you started

The free posts below introduce and link to key VfI guidance documents. You can also check out the VfI Resources Page on my website, where you will find more guidance, practical examples, academic articles, and other materials; it is updated regularly and is intended for you to use and share.

Assessing Value for Money: The Oxford Policy Management Approach - introduces our guide to assessing VfM in international development programs, with a direct link to the full document.
Verian Group’s guide to VfM in UK public services - introduces VfI in domestic policy settings where cost-benefit analysis alone cannot answer the questions that matter.
Value for Investment: Application and Insights - a worked example of VfI in action in a health setting, with transferable lessons for picking up the approach.
There’s a book on the way - stay tuned for more.

Explaining VfI to stakeholders

These free posts unpack key VfI concepts and common concerns about value-for-money assessment, and will help you respond to questions, perceptions, and politics when you need to defend or “sell” VfI to your leadership team, board, or community.

Reach for this set when you’re preparing a board paper, slide deck, or conversation with sceptical colleagues:

Beyond the hierarchy - challenges the misconception that methods like cost-benefit analysis sit at the top of a value-for-money “league table”, and shows how the General Logic of Evaluation and the VfI system help you choose and combine methods to make transparent, defensible judgements.
Not just money… Investment - explains the rationale behind the “Value for Investment” name, reframing policies and programs as investments in value propositions and setting up VfI to evaluate all the resources and value at stake.
Criteria and standards: a logic, not a tool - explains evaluative reasoning as an always-present logic and shows how making that logic explicit and participatory (often via rubrics) strengthens the credibility and ethics of VfI judgements.
Scaling Value for Investment - lessons from helping organisations adopt VfI at whole-of-organisation scale, showing how it can shift culture, embed evaluative reasoning in systems, and support clearer, more credible decisions; includes a list of frequently-asked questions (FAQs) for scaling VfI.

Full thematic library of posts

This is my full directory. Free introductory content is signposted, and some posts are for paid subscribers – you can explore those with a free seven‑day trial if you’d like to sample the deep dives and tools.

Each link below takes you to a themed sub-directory, so you can browse posts on that topic on one place.

The Value for Investment approach - background, principles, and practical guidance on the VfI system.
Rubrics and evaluative reasoning - making criteria, standards, and value judgements explicit, ethical and useful.
Program theory and value propositions - connecting theories of change to “who benefits, in what ways, and is it worth it?”
Economic methods of evaluation - cost-benefit analysis (CBA), social return on investment (SROI), and other economic tools, viewed within a broader evaluative frame.
Cubist Evaluation - an avant-garde proposal for doing rigorous evaluation in complex settings by mixing evaluative reasoning styles, methods and perspectives, and challenging narrow ideas about what counts as rigour.
Causality and contribution - reflections on approaches to understanding whether and how interventions cause or contribute to outcomes.
AI tools in evaluation - using AI in ways that support good quality, ethical, human-led evaluation practices.
Other topics - more personal, playful, or experimental pieces that still circle back to value, evidence, and decisions.

1. The Value for Investment approach

Free posts (in addition to the ones above):

VfI backstory and overview - traces the origins of Value for Investment: why traditional value-for-money approaches weren’t enough, and what gap VfI sets out to fill.
Value for Money: my worldview - a quick overview of core rationale behind the VfI approach.
Evaluative reasoning and VfM - introducing the VfI system’s intuitive step-by-step process to get from questions to answers.
VfI quick-start guide - offers a one-page “plug in and go” guide, walking you through the initial steps of clarifying purpose, choosing criteria, and sketching a workable rubric you can refine later.
Principles and methods to help evaluators answer value-for-money questions - distils key principles from Gargani & King’s (2023) article.
Rapid VfM assessment - four guiding principles for a good VfM assessment, and four examples of rapid, minimalist ways to implement them.
Maximising public value through better data and evaluation - a proposal for embedding VfM thinking and evaluation across the whole policy cycle so public investments create more real value.

Practical tips on designing a VfI evaluation:

How to choose methods for value-for-money assessment - helping you decide when cost-benefit analysis (CBA) or social return on investment (SROI) is enough, when you need something more, and how to combine multiple methods and sources of evidence to answer VfM questions.
Integrating value for money within an evaluation - when VfM is just one part of a wider evaluation, here’s a practical way to integrate it with other questions and objectives in one coherent evaluation design.
Value for Money criteria - how to start from a program’s value proposition, and then define context-specific VfM criteria (using the 5Es to illustrate) so you evaluate what really counts, not just what’s easy to count.
Navigating criticisms of evaluation methods - five practical tips to boost stakeholder satisfaction with your evaluation reports, from co-designing frameworks to constructively handling late-breaking criticisms of methods.
Value for Money reporting - a simple formula for evaluation reports that put clear answers up front, show their working in the middle, and keep methods and detail at the back - resulting in reports that aren’t torture to read and can easily be adapted for multiple different reporting needs and formats.

Also see practical tips on using rubrics.

VfI principles and theoretical foundations - the what and the why:

Principles to advance value for money - a visual summary of key points from a conference presentation, providing a simple overview of the way I view VfM.
Unpacking VfI jargon - shares a plain-language “VfI thesaurus” (riffing off Michael Scriven’s Evaluation Thesaurus) so you can explain the difference between value-for-money and Value for Investment, value propositions, and VfI criteria in everyday words.
Maximising the future public value of government spending - excerpts from Verian Group’s VfM Guide (King & Hurrell, 2024) and a link to the original document.
VfI in the Garden of Eval - situating the VfI approach in The Garden of Evaluation Approaches (Montrosse-Moorhead, Schröter, & Becho, 2024) - a practitioner-oriented classification based on eight dimensions of practice.
Evaluation vs. value for money - unpacks how VfM assessment and evaluation overlap and offers principles for joining them up.

The 5Es - a series of short reads:

Value for Money and the 5Es - setting the scene.
Equity - why it should be the first E (not the last) and options for approaching it in greater depth than just “counting beneficiaries”.
Cost-effectiveness - how to evaluate whether a policy or program creates enough value to justify the investment of resources, with or without economics.
Effectiveness - tackling outcomes, impacts, causal questions, and unintended consequences in VfM assessment.
Efficiency - evaluating productivity and ways of working when your program is more complex than an output-maximising widget factory.
Economy - evaluating stewardship of resources without falling into the “spending less” trap.
Square peg, round hole - integrating top-down criteria with what actually matters to stakeholders.

Power dynamics and VfI:

Locus of power in VfM assessment - a free post introducing and adapting a model developed by Nan Wehipeihana that helps you examine where power sits in your VfM assessments, where it should sit, and how to shift the locus of power.
Towards Made-in-Africa VfM assessment - a free post sharing my presentation at the 2024 African Evaluation Association (AfrEA) Conference.
Is evaluation violent? (Part 1) - examining the concept of epistemic violence in evaluation and VfM assessment.
Is evaluation violent? (Part 2) - in which I encourage precision when we describe the harms evaluation can cause, and identify 41 alternatives to labelling it “violence”.

VfI case examples:

Navigating donor wants and needs in evaluation (Part 1) - a free post co-authored with Zack Petersen, highlighting the problem of “impact theatre” and how it can divert programs away from their purpose.
Navigating donor wants and needs in evaluation (Part 2) - a VfI-based solution to the impact theatre problem, with a worked example.
Investing in Peace - a free post co-authored with Kris Inman, explaining why CBA and SROI often miss the hardest-to-quantify value in peace building investments, and showcasing VfI as a solution.
When “parachuting in” isn’t an option - a free post co-authored with Adrian Field, discussing a VfI evaluation that was conducted almost entirely remotely, and how we managed the quality and safety of our work across continents.
VfI in a Youth Primary Care Mental Health initiative - a free case study from New Zealand, with Dovetail Consulting.
Value propositions and criteria: are free school lunches worth investing in? - a free case study from New Zealand, with the University of Auckland and Eastern Institute of Technology.

2. Rubrics and evaluative reasoning

Evaluative reasoning is the bedrock of the VfI system, using explicit criteria (aspects of value) and standards (levels of value) as a scaffold to make sense of evidence and render a warranted value judgement. A rubric - a matrix of criteria and standards - is one way of making evaluative reasoning explicit.

Free posts about evaluative reasoning and rubrics:

Criteria and standards: a logic, not a tool - explains evaluative reasoning as an always-present logic and shows how making that logic explicit and participatory (often via rubrics) strengthens the credibility and ethics of VfI judgements.
Evaluative thinking - distinguishes evaluative reasoning (how we make judgements) from methods (how we generate knowledge) and evaluative thinking (how we know the reasoning and methods are solid), and shows how curosity, criticality, and attention to assumptions improve the quality of evaluation.
Evaluative reasoning: extra for experts - shows how criteria and standards can support technocratic, deliberative, all-things-considered, and tacit reasoning in a mixed reasoning approach and why that can support better-formed judgements.

Introductory posts - the ‘what’ of evaluative reasoning and rubrics:

Evaluative reasoning (audio) - introduces evaluative reasoning as the logic that links criteria, standards, and evidence to warranted judgements, and shows why this matters for real-world evaluation practice.
Different kinds of rubrics - compares types of rubrics (e.g., holistic, analytic, generic) and how context affects each style’s fitness-for-purpose.
Rubrics from science and medicine - draws examples from outside evaluation to illustrate that rubrics are more than just tools for teachers and program evaluators.
If-then statements - shows how simple if-then structures can make evaluative logic explicit, clarifying how evidence supports claims about value.
Goldilocks and the three criteria - uses the Goldilocks story to argue for “just right” criteria that are specific enough to guide action but flexible enough for complex systems.

Philosophical and conceptual posts - the ‘why’ behind evaluative reasoning and rubrics:

Objectivity and subjectivity in evaluation - unpacks myths about “objective” evaluation, arguing that values are central and that transparency about them is what supports credibility, validity, and use of evaluation.
Rubrics and alternatives - situates rubrics among other evaluative reasoning strategies, clarifying when rubrics help and when other approaches might be a better philosophical fit.
Interpreting evidence through the prism of explicit values - uses Newton’s “backwards” prism and Pink Floyd to show how criteria and standards act as a prism that disperses and recombines values and evidence into clear evaluative judgements.
Evaluative reasoning in complexity - explores how evaluative reasoning can stay rigorous in complex, emergent contexts without abandoning criteria, standards, or participation.

Practical posts - the ‘how’ of developing and using rubrics for evaluative reasoning:

Developing rubrics - step-by-step guidance on co-developing criteria and standards with stakeholders and turning them into usable rubrics.
Surfacing values through rubric development - shows how rubric development can be designed as a process for eliciting, negotiating, and documenting what stakeholders value.
Sense-making with stakeholders and rubrics - illustrates how to use rubrics in workshops and dialogues so stakeholders co-interpret evidence and co-own judgements.
Rubric fails - candid lessons from rubric adventures that went wrong, with practical tips to avoid repeating my mistakes.
A chicken-egg problem in evaluation - reflects on what should come first (values, questions, methods, evidence), how sequencing choices influence what counts as value, and how to manage the chicken-egg problem in practice.
Making meaning through co-constructed lenses - shares reflections on co-construction processes that turn rubrics into vehicles for shared meaning.
The deliberative value formation model - introduces Kenter et al.’s model for deliberative value formation and reflects on how its ideas about collective sense-making and changing values can enrich evaluative reasoning and VfI practice.

3. Program theory and value propositions

VfM assessments are too often fixated on the money and unclear about the value. We can address this by re-framing policies and programs as investments in value propositions. That’s why I prefer the term Value for Investment. Value propositions are useful constructs because we can define them and evaluate how well they’re met.

Posts about program theory and value propositions:

Value propositions: clearing the path from theory of change to rubrics (Part 1) - introduces value propositions as a bridge between theories of change and VfM criteria, showing how “who is it valuable to, and in what ways” can sharpen rubric development.
Value propositions: clearing the path from theory of change to rubrics (Part 2) - applies the value proposition idea to a real policy initiative, turning the questions from Part 1 into a worked example that surfaces draft VfM criteria.
Value for Money criteria - how to start from a program’s value proposition, and then define context-specific VfM criteria (using the 5Es to illustrate) so you evaluate what really counts, not just what’s easy to count.

What’s the value proposition of the VfI approach? - this free post applies a value-proposition lens to VfI itself, explaining who the approach is for and how it delivers better answers to value-for-money questions.
Program theory GPT - an early experiment in the use of ChatGPT when it was fairly new, exploring how a large language model (LLM) can help unpack a theory of value creation from basic principles.

Navigating donor wants and needs in evaluation (Part 1) - a free post co-authored with Zack Petersen, highlighting the problem of “impact theatre” and how it can divert programs away from their purpose.
Navigating donor wants and needs in evaluation (Part 2) - a VfI-based solution to the impact theatre problem, with a worked example.
Investing in Peace - a free post co-authored with Kris Inman, explaining why CBA and SROI often miss the hardest-to-quantify value in peace building investments, and showcasing VfI as a solution.
Value propositions and criteria: are free school lunches worth investing in? - a free case study from New Zealand, with the University of Auckland and Eastern Institute of Technology.
Value propositions (Part 1) - introduces value propositions as an extra layer on top of a theory of change, clarifying what kinds of value a program creates, for whom, and how, to guide VfM criteria development.
Value propositions (Part 2) - applies the value proposition idea to a real VfM assessment, showing how the Part 1 ramework played out in an actual evaluation.

4. Economic methods of evaluation

VfI is an inter-disciplinary approach, because “good resource use” is a shared domain of evaluation (determining the goodness of things) and economics (studying how people choose to use resources). The VfI approach encourages the inclusion of economic concepts, models, methods and tools, where feasible and appropriate within a wider evaluative framework.

Free posts about economic methods of evaluation:

Ten commandments for social CBA and SROI - sets out ten practical principles for applying cost-benefit analysis and social return on investment in social sectors, from making a judgement to not treating them as the whole evaluation.
The stone tablets of social value (audio) - an AI-generated podcast-style exploration of the Ten commandments for social CBA and SROI post.
Cost-benefit analysis and the logic of evaluation - shows how CBA maps onto Scriven’s logic of criteria-standards-evidence-synthesis, arguing it is a form of evaluation and belongs as much to the field of evaluation as to economics.
Using Economic Methods Evaluatively - a summary of my 2016 journal article of the same name, which marked the formal starting point for the key ideas that later became the VfI approach.
Ratios as VfM indicators - why simple cost-per-output or per-outcome ratios are tempting as VfM indicators, what feasibility, quality, and interpretability hurdles they must clear, when they help, and when they mislead.
What data do we need for an economic evaluation of a social investment? - introduces a free guide by Aaron Schiff, offering practical advice on planning needs early so that CBA/SROI are feasible, credible, and not derailed by missing information.
Impact valuations for CBA and SROI in the NZ Treasury CBAx tool - highlights Aaron Schiff’s guide to using CBAx impact valuations, and adds my perspective on how to use them thoughtfully in CBA and SROI without losing sight of context and judgement.

Foundations and ‘big picture’ pieces:

Bringing costs into evaluation - three books I recommend on economic evaluation methods - and what they don’t cover, and how VfI ties it all together.
Cost-benefit analysis through an evaluative lens - re-frames CBA as one strand in a mixed-methods value story, highlighting what it captures, what it can miss, and how to combine it with broader evidence and values.
Disambiguation - clarifies how CBA, SROI, and VfI differ in purpose, logic and use, and where they overlap.
How to choose methods for VfM assessment - offers a simple decision logic for when to use CBA, SROI, VfI, or other methods (alone or in combination) to answer VfM questions in different contexts.
CBA as a mixer - argues that CBA is best used as a “mixer” in a VfI cocktail: one strong ingredient alongside rubrics and mixed-methods evidence rather than the whole drink.
Know when to hold ‘em - most of the literature on cost-benefit analysis (CBA) focuses on how to do it, with relatively little guidance on why, and next to nothing on when not to do it or when to combine it with other methods. This post is an effort to fill this gap.
SROI with a rubric? - highlights a core element of evaluation theory and practice that’s missing from standard social return on investment, and shows how rubrics and SROI can work together so the headline ratio and other pieces of evidence contribute in a balanced way within an explicit evaluative framework.
CBA in U.S. regulatory decision making - uses the U.S. regulatory context to discuss how CBA influences big policy choices, and what evaluators elsewhere can learn (and unlearn) from that history.

CBA in practice: design, data, uncertainty, equity:

Contextually-responsive CBA - argues for tailoring CBA design to context, stakeholders, questions, and constraints, rather than following a rigid template that ignores what matters locally.
Modelling costs and benefits in uncertainty - introduces scenario and sensitivity analysis, showing how to handle uncertainty honestly and informatively to enhance economic analysis.
Break-even analysis and rubrics - explains what break-even analysis is, and how it can be combined with rubrics to answer “how good is good enough?” when evidence is incomplete but decisions cannot wait.
Scaling return on investment - explores what happens to ROI when programs scale up or down, and why benefit-cost ratios from pilots do not automatically carry over to system-level decisions.
Wellbeing and CBA - shows how subjective wellbeing and related measures can help put valid numbers on intangible outcomes in social CBA.
Equity through a cost-benefit lens - examines how CBA can address distributional issues, and argues for explicit treatment of equity beyond CBA alone.
Navigating CBA’s triple tradeoff - uses four diagrams to show how CBA juggles rigour, scope, and feasibility, why it cannot answer VfM questions on its own, and how VfI-style mixed methods can incorporate CBA within a wider view.

Ethics, standards, and CBA/SROI comparisons:

With great power comes great responsibility - argues why CBA should be used more but differently, combining its strengths with those of other methods to tell a fuller value story.
Communicating value - why it takes more than a headline number, and how to combine return-on-investment with explicit evaluative judgements so value can be expressed in one word, one sentence, or a full mixed-methods value story.
How should Program Evaluation Standards inform the use of CBA in evaluation? - links program evvaluation standards to CBA practice, showing how to keep it systematic, fair, and useful.
Can we stop treating CBA as if it’s “objective”? - challenges the myth of CBA objectivity, unpacking where judgement enters and why transparency about values matters.
Spending to save: avoiding future fiscal costs - welcomes rigorous modelling of avoidable fiscal costs but argues “spend now to save later” is not the same as public value, and shows how social CBA and VfI can keep prevention policy focused on societal benefits, not just treasury savings.
SROI versus CBA - compares social return on investment and cost-benefit analysis in aims, practice, and typical strengths, and argues for mixing the best of both rather than choosing sides.
CBA and SROI in the garden of eval - uses the Garden of Evaluation Approaches framework to locate CBA and SROI philosophically and show why blending them with other approaches gives richer answers.
Transparency in SROI and CBA - sets out what methodological and value transparency should look like in SROI and CBA, and why “show your working” is non-negotiable.
Avoiding “computer says” social value assessments - warns against overly technocratic use of SROI/CBA tools and offers tips to keep human deliberation and evaluative reasoning firmly in the loop.
Causal attribution in SROI - digs into how SROI can handle counterfactuals and contribution claims credibly, and common pitfalls in over-claiming impact.

Other economic concepts and methods:

Cost-effectiveness analysis: an evaluator’s perspective - explains CEA in evaluator-friendly terms, when cost-per-outcome is a good fit, and how to interpret results.
Cost-utility analysis: an evaluator’s view - introduces CUA and quality-adjusted life years (QALYs), and discusses their strengths and limitations for social and health sector evaluations.
Realist economic evaluation - introduces the Realist Economic Evaluation Methods (REEM) project and explores how realist evaluation and economic methods can be combined to understand what works, for whom, in what contexts, and at what cost.
Markov models - demystifies Markov modelling for evaluators, and where they add value in economic evaluation and other fields.
The road not travelled - focuses on opportunity cost and why “what else we could have done” is central to economic evaluation and VfI.
Worthwhile travel time - reflects on how CBA currently values travel time savings, and suggests ways this evolving practice could better capture what makes travel time genuinely worthwhile (or not) for people’s lives.

5. Cubist Evaluation

Cubist Evaluation seeks to challenge and disrupt conceptions of rigorous evaluation in complexity by encouraging us to think beyond methods, beyond measurement, and beyond objectivity. It involves mixing methods and reasoning strategies, honouring multiple perspectives, challenging dominant narratives, contributing new meaning, and opening up possibilities. It highlights a set of dispositions that we can choose to bring to evaluation and Value for Investment.

No alternative text description for this image

Posts about Cubist Evaluation:

Mixed reasoning and Cubist Evaluation - introduces Cubist Evaluation as a way of mixing different evaluative reasoning strategies so that stakeholders can view value from multiple-angles and co-create well-formed judgements.
Mixing methods and metaphors - plays with ideas of mixed methods, bricolage, and eclectic methodological pluralism, arguing that evaluators should comfortably mix disciplines, tools, and metaphors in VfI practice.
AES23 highlights - this free post relects on the AES 2023 Conference where I first floated the Cubist Evaluation idea in public, and shares my slides that sketched out the idea, linking it to new conceptions of rigour and pluralistic evaluative reasoning in complexity.

6. Causality and contribution

Evaluation and causation are two different tasks, with different logics and methodologies. VfI contributes, in particular, to applying the valuing part of evaluation to value-for-money questions - how to define what good resource use and value creation mean in a context, how to use those definitions to identify evidence requirements and select methods, how to make and communicate evaluative judgements.

Addressing whether an intervention caused or contributed to an observed set of outcomes is usually an important task within a VfM assessment - because, before we can determine the value of an impact, we need to ascertain whether it is valid to claim impact and, if so, to what extent and in what ways. However, I have only written a few articles on causality because VfI doesn’t prescribe causal approaches. Evaluators and researchers have developed a diverse range of strategies and methods for addressing causality and contribution in different contexts, and they are all on the table as far as I’m concerned. What matters in a VfI evaluation is that the causal questions are addressed, and that the methods used to address them are selected according to context, with possibly more than one method used to support triangulation.

Here are my posts on this topic, which are by no means exhaustive. For more information on causal approaches, see the Better Evaluation website.

Gold standards and double standards - in this free post, I unpack why I think RCTs are so important for health interventions, yet insist on a broader mix of causal methods (including but not limited to experiments) for social program evaluation.
Applying Bradford Hill criteria to observational evidence - this free post uses a worked example to show how Bradford Hill’s epidemiological criteria can inform context-appropriate evidentiary standards for causal inference in evaluation, shifting focus from “best method” to quality of reasoning.
Triangulating observational and administrative data - shows how integrated data infrastructures (like NZ’s IDI) can generate statistical counterfactuals that, triangulated with service data and evidentiary criteria, can strengthen causal claims in social services.
Eco-systemic frames in medical research (and evaluation) - reviews a medical paper on “eco-systemic” research and draws parallels with evaluation, arguing for complexity-informed, mixed-method, stakeholder-engaged approaches over narrow “what works” hierarchies.
Realist economic evaluation - introduces the Realist Economic Evaluation Methods (REEM) project and explores how realist evaluation and economic methods can be combined to understand what works, for whom, in what contexts, and at what cost.
Causal attribution in SROI - digs into how SROI can handle counterfactuals and contribution claims credibly, and common pitfalls in over-claiming impact.

7. AI tools in evaluation

There’s a lot of talk about whether AI will replace evaluators. Better questions are: which parts of an evaluation can safely and ethically involve AI, which must remain human, how do we decide, and how do we design appropriate processes?

In an “evaluator‑free evaluation” experiment, an AI tool designed and conducted an evaluation in one minute that would normally take a team 40–50 days. The framework and evidence synthesis looked impressive, and it was easy to imagine commissioners feeling tempted to skip months of procurement, consultant management and big budgets in favour of a one‑minute, desk‑based evaluation.

But for consequential public decisions, that would be a mistake. The fully automated AI evaluation blurred criteria and indicators, mixed up program performance with evidence quality, and left no space for authentic stakeholder voice. With insufficient human oversight, the one‑minute evaluation was unreliable and arguably worse than no evaluation at all.

The opportunity lies in a more sensible balance: human evaluators in the lead, with AI as an assistant. Think “5‑day evaluation” that meets program evaluation standards such as utility, feasibility, propriety, accuracy, accountability and cost. If hybrid, AI‑intensive approaches can meet those standards more affordably, we may one day ask whether it’s ethical *not* to use AI.

If we can get the balance right, evaluations may become better, quicker, and more accessible - and we could find ourselves busier than ever. It may even open up new methodological possibilities, like conducting the same evaluation thousands of times to map and visualise the interpretive space, supporting better-informed human judgements.

AI-enhanced evaluation - this free post argues for “humans-first, AI-assisted” evaluation, using comparative advantage to get more-better-quicker-cheaper evaluations while keeping ethics, sense-making, and value judgements firmly human.
AI, reflexivity, and where I draw the line - outlines an ongoing academic debate about whether and how AI should be used in qualitative research, and my thoughts on its application in mixed-methods evaluation.
AI and the Gell-Mann amnesia effect - highlights how easily we can trust slick AI outputs, and offers concrete quality assurance prompts and habits so evaluators stay ahead of the unreliable side of LLMs.
Evaluator-free evaluation - test-drives fully AI-generated evaluations to explore what’s possible in a minute, why it’s not good enough for real decisions, and where human evaluators remain indispensable.
One-minute evaluation, a thousand times - explores the idea of ultra-rapid, repeated micro-evaluations to map the interpretive space and how they might complement (not replace) deeper human-led evaluations.
EvalGPT - a light-hearted exploration of a dystopian future in which Egbert the Evaluator automates his evaluations and puts his feet up.
Program theory GPT - an early experiment in the use of ChatGPT when it was fairly new, exploring how a large language model (LLM) can help unpack a theory of value creation from basic principles.

8. Other topics

Living with the uncertainty monster - exploring ambiguities, disagreements and unknowns that inhabit evaluations, how they can affect what is decided, and practical ways to design and report evaluations that surface uncertainty and support better decisions.
Evaluator sorting hat - a Hogwarts-themed quiz to discover your evaluator “house” and reflect playfully on your strengths, blind spots, and the overall magic of your team.
Beware of the Leopard - ten Douglas Adams-inspired reflections that skewer bureaucratic absurdity and suggest principles for good evaluation.
Evaluator reacts to Stranger Things - what the series teaches us about decision-making when worlds of evidence collide and the stakes are high.
Should I set up shop? - this free post takes a candid look at going independent as an evaluator; co-authored with Adrian Field.
ebikonomics - a free post about why I think e-bikes are a case study in VfM, providing both tangible and intangible returns on time, effort and money.
Beyond the numbers: growing value in NZ science - New Zealand’s research reforms promise growth, but will its metrics hit the target or miss the point?
Number Needed to Treat - a friendly of explainer of a key clinical metric and its novel application to road safety, revealing that upgrading your car can make your commute safer, but not to the extent you might have thought.
Electric vehicles, emissions and thermal runaway - a free post unpacking when EVs really cut emissions, and how to think about battery risks without panic.
Elephant - a Wellington-set retelling of the Blind Men and the Elephant fable that captures how complex systems feel from different vantage points.
My story - a personal account of how I fell into evaluation and VfI, and the thinkers and practitioners who’ve influenced my professional journey.

Evaluation and Value for Investment

Discussion about this post

Ready for more?