Developing context-specific criteria
Using value propositions to make evaluation frameworks meaningful to both funders and communities
A familiar challenge in evaluation
Off-the-shelf criteria, like efficiency and effectiveness, are often handed down as mandatory focal points for evaluations. But these terms can feel generic and disconnected from what stakeholders actually value. This leaves us trying to force-fit standard criteria into situations that demand nuance - a square peg, round hole problem.
This post shares a practical strategy for bridging that gap: how to identify and define evaluation criteria that are meaningful in context while still respecting top-down frameworks and expectations. By bringing together formal criteria with stakeholders’ perspectives on what value looks like, I’ll show how starting with a clear value proposition can translate top-down requirements into context-specific terms, and provide you with prompts to make it actionable.
Value propositions pave the way to clear criteria
The key insight is that by defining the value proposition (what makes an intervention valuable, to whom, and how), we create a bridge from the context to a set of evaluation criteria. By answering a set of value proposition questions, we are, in effect, focusing on the program or policy rather than the criteria. Then, when we turn our attention back to the criteria, we have primed ourselves to define what each criterion means in context-specific terms. This helps keep the criteria connected with explicit stakeholder values.
For example, contrast the instruction, “define efficiency” in a value for money (VfM) framework, with “what ways of working will help maximise value from this program?”. The first is abstract and criterion-focused, the second is more concrete and context-focused.
Below, I will illustrate this principle at work by offering some conversation starters for: 1) the 5Es of VfM; 2) the OECD DAC evaluation criteria; and 3) a few bonus criteria that often turn out to be important. This isn’t an exhaustive list of criteria, but it serves to illustrate the underlying principle.
Bear in mind that the value proposition questions can be asked in different ways depending on the setting and the stakeholders involved. The aim is to make the questions as intuitive as possible for stakeholders to engage with - so they should be adjusted as needed.
1. The 5Es of Value for Money
The 5Es are a good doorway into a conversation about VfM criteria because they systematically address an investment’s value chain. The following questions are aimed at moving from generic to context-specific definitions for each E-word. As a reminder, these are orienting questions, not evaluation questions; their purpose is to help surface criteria rather than directly evaluate the policy or program.
Economy
What resources are invested, by whom? (monetary and non-monetary). What does good stewardship of resources look like? For example:
Monetary: How do we secure the highest quality inputs (staff, materials, technology, partnerships) at the best value? What procurement, recruitment, and resource allocation decisions optimise our starting position?
Non-monetary: How do we identify, attract, and nurture the highest quality intangible resources - such as knowledge, relationships, organisational culture, reputation, trust - at the greatest long-term value?
Efficiency
What ways of working will help maximise value from the investment in this program? For example:
How do we ensure we’re investing in the right mix of activities - prioritising objectives, balancing our portfolio, and allocating resources to deliver across all outputs?
How do we organise workflows, manage processes, and coordinate activities to maximise productivity? What management approaches eliminate bottlenecks and redundancies?
How should the program adapt and evolve - using monitoring, evaluation, feedback, new technologies, and risk management - to become more efficient over time? What processes support reflection, learning, and rapid response to shifts in context or emerging opportunities?
How do we build and maintain relationships, trust, and effective communication to ensure smooth operations and good resource use? What explicit investments are made in relationship-building as a foundation for future efficiency and effectiveness, and how is progress monitored and valued?
Effectiveness
What changes in people, groups, or places - or in some cases, protections against things getting worse - should the investment bring about? How will we know if we’re making progress toward them?
What lasting, transformative changes should this investment create in the broader system? What ripple effects - positive and negative - might we expect?
Cost-effectiveness
To whom is the investment valuable, and how is it valuable to them? What evidence and rationale would support the claim that it is worth investing in?
Equity
Who needs this investment and why? What inequities does the investment tackle, and how?
Who should benefit most? How should resources, actions, impacts, and value be allocated to achieve this?
The point is:
These questions help us pin down criteria that are specific to a setting and what stakeholders care about. Our criteria will be aligned with generic definitions of the 5Es, and at the same time tailored so that they get to the heart of what value creation looks like in a particular setting.
For example, if we apply the 5Es-specific value proposition questions to an education program aimed at enhancing parental engagement in children’s education, supporting literacy and numeracy development in early years of school, it can help us define context-specific VfM criteria.
The following criteria are illustrative only. In practice, evaluation criteria would be developed in collaboration with program stakeholders to ensure relevance, inclusivity, and alignment with school and community priorities. As this example illustrates, it is best to develop these criteria early so that the design of the criteria and the design of the program are part of the same conversation.
Economy
The program leverages both financial and non-financial resources from schools, families, and communities, ensuring money and effort are directed toward outcomes that matter for young learners. The highest quality staff, learning materials, and digital tools are sourced at appropriate prices through transparent procurement and strong partnerships. The program recognises and actively cultivates intangible assets that are critical to success - including trusted relationships with families and a positive school culture.1
Efficiency
Workflows are designed so each participant understands their role and time is used wisely, maximising quality engagement and learning support. Objectives are prioritised clearly so resources and efforts are balanced across key program activities. Regular reviews ensure activities are aligned with changing needs. Communication is open, coordination is strong, and management systems are in place to prevent delays and duplication. The program adapts over time through continuous feedback and the adoption of fresh ideas, using monitoring and evaluation insights to respond quickly to new opportunities or challenges. Building trust and maintaining strong links - between staff, parents, and the wider school community - are treated as essential foundations for consistent program performance.
Effectiveness
The program contributes to positive changes in children’s early literacy and numeracy skills by equipping parents to support home learning with confidence. Measurable improvements are seen in student achievement, parental participation rates, and enthusiasm for learning at home and school. Over time, the program nurtures a culture where collaborative learning between families and educators is the norm, leading to broader, sustained gains for all involved.
Cost-effectiveness
The investment brings clear value to students, parents, teachers, and the community by producing achievement gains and closing learning gaps more efficiently than isolated classroom or home efforts alone. Its effectiveness is demonstrated by a strong evidence base of improved outcomes relative to cost, stakeholder satisfaction, and increased family-school engagement - making a strong case for ongoing support.
Equity
The program targets families and communities who are historically less engaged or face greater barriers to participation, ensuring resources and supports reach those with the highest need. It actively works to close literacy and numeracy gaps for diverse student populations. Efforts are made so that every child, regardless of background, benefits equally, and allocation of resources and attention is guided by principles of fairness and identified need.
2. The OECD DAC criteria
The OECD Development Assistance Committee (DAC) criteria are Relevance, Coherence, Effectiveness, Efficiency, Impact, and Sustainability. These terms already have generic definitions within the framework - and the challenge we face as evaluators is to define and manifest them in ways that get to the heart of the intervention we’re evaluating.2 The following questions are designed to help with this task:
Relevance
What real needs, problems and priorities does this investment (e.g., policy, program, service, etc) address, and for whom? How will we know if these are the right things to focus on right now? How will we know if this investment is an appropriate intervention?
Coherence
What other efforts are happening around the same issues, both within this investment (internal coherence) and in the wider system (external coherence)? How should this investment complement those efforts, avoiding duplication or harmful disruption, and where appropriate strengthen them? What partnerships and alignments would create synergies and multiply its impact?
Effectiveness
What changes in people, groups, or places - or in some cases, protections against things getting worse - will the investment bring about? How will we know if we’re making progress toward them? Who should benefit most?
Efficiency
How do we optimise the pathway from resource investment through to productive delivery and lasting results?
Resource investment: What resources are invested, by whom? How do we secure the highest quality inputs (staff, materials, technology, partnerships) at the best value? What procurement, recruitment, and resource allocation decisions optimise our starting position?
Productive delivery: How do we organise workflows, manage processes, and coordinate activities to maximise productivity? What management approaches eliminate bottlenecks and redundancies?
Lasting results: How do we design and adjust our delivery methods so that our efforts lead to the intended changes in people’s lives?
Impact
What lasting or transformative changes should this investment create in the broader system? What ripple effects - positive and negative - might we expect?
Sustainability
What capabilities, systems, and conditions need to be in place for the benefits to continue long after the initial investment? Which of these markers of sustainability should already be visible at this point? How do we build actions and impact that last?
Again, the point is:
These questions guide us toward criteria that are grounded in how a particular context works, and what people value in that context, while staying true to the underlying generic definitions of the OECD-DAC criteria. Here’s how those questions might translate into actual criteria for one concrete case…
For example, if we apply the OECD DAC-specific value proposition questions to a multicultural food festival, it can help us define context-specific evaluation criteria.
The following criteria are illustrative only. In practice, evaluation criteria would be developed in collaboration with community members and other key stakeholders (such as funding organisations) to ensure cultural relevance, inclusivity, and alignment with community priorities. As this example illustrates, the best time to develop these criteria is right at the outset of festival planning, so that the design of the criteria and the design of the festival are part of the same conversation.
Relevance
The multicultural food festival responds to local aspirations to celebrate cultural diversity, strengthen social connection, and make the city’s public spaces feel welcoming to a wide range of communities. It addresses priorities around inclusion, recognition of migrant and indigenous cultures, and creating family-friendly spaces where people can share food and stories. It has the right focus for this time and place, given demographic change, past experiences of exclusion, and recent calls for more visible recognition of cultural communities in civic life.
Coherence
The festival fits alongside and complements existing city strategies and initiatives for social cohesion, economic development, arts and culture, and tourism without duplication. It connects with community organisations, cultural associations, schools, hospitality businesses, and local producers, so that activities at the festival reinforce other efforts to promote participation, language and culture, and small business growth. Partnerships with organisations already working on inclusion and wellbeing help the festival to pull in the same direction as related policies and programs, rather than sitting as a one-off event.
Effectiveness
The multicultural food festival contributes to increased positive cross-cultural interaction and a stronger sense of belonging for underrepresented communities. Progress can be seen in who attends and participates as stallholders, the quality of experiences reported by different groups, repeat attendance, and evidence of new relationships or collaborations formed through the event. Those who benefit include migrant and indigenous communities, local food entrepreneurs, nearby residents and families.
Efficiency
Resources such as city funding, stall fees, sponsorship, volunteer time, and in-kind contributions from community organisations are invested by both public agencies and local partners. High-quality inputs are secured by transparent procurement, fair stallholder selection processes, and accessible pathways for smaller or newer businesses to participate. Planning and logistics aim to minimise waste, make good use of shared infrastructure, and manage noise, traffic, and crowding, so that the festival delivers a high-quality experience relative to the resources used and the temporary disruption to public spaces.
Impact
The multicultural food festival can contribute to longer-term shifts in how people view cultural diversity, local food systems, and public spaces. It may help to normalise cultural expression in the city centre, encourage more diverse representation in other events, and support ongoing demand for culturally diverse food businesses beyond the festival day. Potential ripple effects include stronger networks among cultural groups, increased confidence for community-led initiatives, and a richer public conversation about identity and belonging, alongside a need to monitor unintended consequences such as new tensions between groups or the commercialisation of cultural practices.
Sustainability
To keep benefits going, the festival invests in relationships and capabilities, not just a single annual event. This includes supporting community committees or advisory groups, succession planning for volunteer leadership, and opportunities for young people to take on roles in organising and hosting. Securing a mix of funding sources, embedding the festival within longer-term city strategies, and strengthening community ownership all increase the chances that the celebration of diverse food and culture will be maintained and adapted over time, and early signs of these dynamics can be watched for during each festival cycle.
3. Other common criteria
Beyond the 5Es and the OECD-DAC criteria, here are some additional criteria that often come up in evaluations, with value proposition questions for each. This list is not exhaustive.
Affordability
What financial commitments can stakeholders realistically sustain without compromising their other essential needs? What conditions need to be met for the investment to be affordable? How do we ensure access isn’t limited by ability to pay?
Proportionality
How do we match the scale and intensity of our response to the level of resources allocated, and to the size and urgency of the problem we’re addressing?
Scalability
What would need to be true for this approach to work effectively at a much larger scale? What are the requirements and potential barriers to scaling (up, out, or deep)?
Acceptability
What cultural, social, and organisational factors will determine whether people embrace or resist this approach? How do we design with user experience in mind?
Ethics
What values and principles guide our decisions about who gets what, when, and how? How do we ensure fairness and avoid unintended harm?
Accountability
How will we demonstrate that we’re using resources responsibly and achieving what we promised? What governance and reporting relationships keep us on track?
Transparency
What information needs to be visible to whom, and when, for trust and informed decision-making? How do we balance openness with privacy and strategic considerations?
Adaptability
How will we recognise when circumstances change, and adjust our approach accordingly? What feedback loops and decision points enable course correction? How does our system need to be designed to support appropriate flexibility and adaptation, and to learn from adaptations already made?
Innovation
What new approaches, technologies, or partnerships could substantially improve outcomes? How do we balance methods that are known to be effective with those that have breakthrough potential? What is our risk appetite and how does this influence the extent to which we are willing to innovate in this context?
Conclusion
Evaluation frameworks and criteria aren’t ends in themselves - they are tools to help make sense of value. By starting with value propositions, we can translate abstract criteria with generic definitions into terms that reflect reality and resonate with stakeholders, while still aligning with top-down expectations. This helps avoid the “square peg, round hole” trap and creates criteria that are both credible and usable. The prize is a sharper picture of what matters, why it matters, and how we know we’re making progress.
Thanks for reading!
And many thanks to Niki Wood for helpful peer review. Errors and omissions all mine.
Also check out these related topics:
Each of these criteria contain multiple sub-criteria. While I have presented the criteria in this post as unified statements, it can be very helpful to use a parsed structure when developing rubrics or checklists.
The 5Es framework has two criteria that overlap with the OECD DAC criteria: efficiency and effectiveness. However, they are defined differently in the two frameworks, therefore the value proposition questions are different.