Is evaluation violent?
Is humanity served if we view it as such?
Over a month after Dr. Luke Roberts’ keynote at the UK Evaluation Society Conference, I’m still ruminating on his provocative presentation: The Violence of Evaluation: Reclaiming Evaluation for Complexity.
While some of us might equate violence with physical harm, according to Roberts:
“Violence is any intentional or avoidable non-consensual act that inflicts harm or significantly impedes the fulfilment of an individual’s, group’s or system’s basic needs”.
This definition broadens violence beyond the physical, to include systemic, psychological, and structural harms, such as those perpetuated by institutional practices - including evaluation.
Could it be that when we engage in a social and political practice that examines policies, programs, and interventions, to judge their value and guide resource allocation decisions, that we sometimes commit acts of violence?
Three ways evaluation can be “violent”
Roberts gave three examples of ways evaluation can become violent – extractive practices, reductionist approaches, and positionality. He also mentioned a fourth kind of violence, inflicted on evaluators.
Extractive practices
Evaluation has on occasion “mined” knowledge from people and groups, sometimes without regard for the impact on participants. Asking traumatic questions or harvesting data without due regard for the rights of its guardians or owners can be exploitative and dehumanising.
Reductionist approaches
When evaluation reduces complex systems to simple parts, it can miss their interconnections and emergent properties. Imposing linear models like theories of change, or using randomised controlled trials (RCTs) to address causal questions, can obscure nuance, ignore local wisdom, and promote one-size-fits-all approaches that may not always be effective or appropriate.
Positionality
When we engage with a complex system, we become part of that system - and then we may fail to recognise systemic violence already present in the environments being assessed. For example, introducing synthetic pepper spray in youth custody may be driven by institutional demands rather than evidence. Evaluating such interventions without questioning the underlying culture or power dynamics can perpetuate harm.
Violence toward evaluators
Roberts argued that violence can be done to evaluators themselves, when their work is ignored, suppressed, or misrepresented by those in power.
A transformative future for evaluation
Roberts called for a radical shift in how we approach evaluation. Rather than seeing it as a mechanistic, extractive, or reductionist process, we should embrace its autotelic value such as the inherent worth of curiosity and collaborative learning. By trusting in the power of emergence and complex systems, evaluation can become a tool for positive, organic change.
He challenged us to envision an evaluation practice that is peaceful, playful, relational, and ecological. Such an approach fosters wisdom, nurtures innovation, and empowers sustainable change. In this future, evaluation is not just about answering “what works?” but about creating conditions where new possibilities can emerge – where the process itself is as valuable as the outcomes it seeks to measure.
I (and many others) have written about similar possibilities, so this resonates.
However, it leaves me feeling a little conflicted
While I find much to admire in Roberts’ vision, I’m left with lingering questions about the implications of labelling evaluation as “violent”.
As someone who has spent years designing, conducting and advising on evaluations, and wrestling with the limitations of evidence and engagement, I’ve seen how evaluation can unintentionally cause harm. I’ve witnessed the hesitancy of indigenous leaders to allow yet another group of evaluators to “study” their community. I’ve felt stakeholders’ frustration when a survey analysis reduced their voices to data points, or an outcomes framework failed to capture the nuances of people’s lived realities. These experiences chime with Roberts’ critique of extractive and reductionist practices.
The frogs were glad the school met with the economist's approval. But somehow they felt short-changed and a little deflated. They felt as if the study had missed the point. The economist had valued the froggy school using piggy values. And some of the most valuable outcomes, like the cultural outcomes that differentiated the new school from regular ones and were most prized from a froggy perspective, had somehow been lost in translation.
From: Getting to the Real Value in Value for Money - a Muppet fable
Yet, I also continue to see evaluation as a force for positive change - a way to amplify marginalised voices, promote innovation, accountability and learning, and guide resources to where they are needed most. I’ve seen evaluations spark important conversations, challenge assumptions, and help to strengthen relationships and trust.
I struggle with the core message of the presentation: that evaluation with negative consequences is “violent”. The term jars me. I question whether Roberts’ expanded definition of violence is reasonable - though I agree that even the most well-intended work can perpetuate harm, especially when it is uncritical or inattentive to context and power.
I also wonder about the implications of this framing.
When evaluators use theories of change, experimental study designs, or any approach that carries both benefits and risks (i.e., all of them), are they truly being violent?
When evaluators extract evidence from indigenous groups in culturally insensitive ways that do not respect their sovereignty over their own data and stories, that’s unethical and harmful. But is it violent?
Pepper-spraying young people is violent, but is an evaluation of a youth custody facility violent if it fails to put an end to pepper-spraying?
If a government minister ignores my evaluation report, does that make me or program stakeholders victims of violence?
If we label all flawed or imperfect evaluation “violent”, does the term lose its power to describe the truly egregious violence that occurs every day, at every scale from interpersonal to international?
Does “violentising” evaluation risk paralysing practitioners who are striving to do good in complex, messy systems? I wonder if this language might alienate those who most need to reflect on their practice, or discourage evaluators from engaging at all.
Potentially, by this definition, all evaluation is violent. Perhaps we should stop evaluating altogether. But if we don’t evaluate anything ever, is the omission also an act of violence? Is nixing evaluation more violent or less violent than, say, co-creating a theory of change with stakeholders?
At the same time, I can’t deny the impact of the keynote.
Provocative language can force us to confront uncomfortable truths about the unintended consequences of our work. Roberts made important points, but in the end he lost me in the stretch to labelling it “violence”.
Or did he? Maybe the fact I’m still thinking about it, weeks later, reveals the success of his presentation.
What do you think?
Have you encountered examples of “violent” evaluation in your work? How do you think we can ensure our evaluations are more ethical and less harmful?
Acknowledgement
In the interests of transparency and in keeping with the “nothing about us without us” principle, I invited Dr Luke Roberts to review a draft of this article. He was happy for me to post it and interested to see your reactions. The views expressed here are my own.
Julian, I wasn’t at the presentation but in my view epistemological violence is very real for those on the receiving end of it. I have witnessed the harm caused by knowledge practices like evaluation, particularly dominant knowledge practices marginalising and/or invalidating the knowing of other groups. When we devalue or invalidate indigenous knowledge, we deny agency and voice, and lived realities, and perpetuate the felt violence of dominant and colonising systems.
This is not physical violence, but it’s just egregious, and has real and traumatic effects like poverty, ill health, higher death rates, etc.
Of course, a violent result may not have violent intent, but the point is, evaluators must be particularly sensitive to the potential for unintended violence.