Mixing methods and metaphors
Bricolage and eclectic methodological pluralism: Similar or different? Does it matter? I turned to AI for semantic assistance.
I often talk about mixed methods in evaluation. In my view it’s helpful to combine quantitative and qualitative evidence to understand things. However, the Value for Investment approach blends more than just numbers and stories. It integrates disciplines (principally evaluation and economics, while accommodating others) and provides an organised process for making sense of value in complexity which can include carefully selecting and mixing different:
Evaluation designs (from experimental to experiential and many other possibilities)
Ways of knowing (from Western science to indigenous science such as mātauranga Māori - but not astrology or memetic dogma; warrant claims are mandatory)
Ways of valuing (for example, criterial, monetary, individual, collective, utilitarian, deontological, and more)
Approaches to synthesis (e.g., rubric-based, algorithmic, deliberative, etc)
Approaches to reasoning (e.g., deductive, inductive, abductive, probative, etc)
Perspectives (e.g., subjective, objective, inter-subjective, inter-objective)
Lifeworlds (e.g., experiences, beliefs, traditions, values…)
… and so on.1
Hence the (fun, yet serious) Cubist Evaluation proposal, asserting that rigour in complexity requires thinking beyond methods - and you’d better bring your biggest palette and all your paints, because it’s going to involve a lot of mixing.
A fundamental principle that justifies and demands this mixing is that evaluative reasoning (making sense of facts through the lens of values) is inextricably connected to power (which values and whose values count, what counts as credible evidence, who gets to have a voice and who decides). This reality pretty much curtails approaching evaluation from a single viewpoint, and it means that participatory, power-sharing evaluation can’t be an optional extra - it’s core business.
I’ve been guilty of using the term ‘mixed methods’ imprecisely as a shorthand for something bigger than quant+qual. Terms like bricolage or eclectic methodological pluralism may be closer, but I’m a bit vague on how they differ. They mainly strike me as overlapping concepts. From recent correspondence on LinkedIn, it appears I’m not the only one who doesn’t feel entirely clear about the distinctions between them.
Perplexity AI
I have an app on my phone called Perplexity, which is great for quick answers to questions like this. It has a few advantages over Chat GPT:
Perplexity searches the web in real time, whereas Chat GPT relies on its training data which gradually becomes outdated.
Perplexity cites the specific sources it used to generate its responses, providing a transparency that enhances its credibility. Chat GPT doesn’t systematically cite source information (and has been known to fabricate references).
Perplexity is precise with information lookup and synthesis. While Chat GPT is great at open-ended, brainstorming-oriented content generation, Perplexity is more grounded in the real world (though in my experience still capable of hallucinating, so be careful).
Perplexity can read PDFs! You can give it the URL for a document and then ask it questions about that specific document. Or it can find and read PDFs on the internet to address your queries. Chat GPT can’t.
So I prompted Perplexity:
Define mixed methods, bricolage, and eclectic methodological pluralism
Perplexity replied:
Mixed methods refers to an approach that integrates both qualitative and quantitative methods within a single study to provide a more comprehensive understanding of the research problem. It involves gathering and analyzing both open-ended qualitative data and closed-ended quantitative data, and drawing interpretations based on the combined strengths of both datasets.
Bricolage is a methodological approach that involves piecing together various research methods, techniques, and perspectives to address complex research questions or phenomena. It is a form of methodological eclecticism that embraces the use of multiple, even conflicting, theories and perspectives to gain a deeper understanding of the research topic. Bricolage encourages researchers to move beyond disciplinary norms and combine diverse methods in innovative ways.
Eclectic methodological pluralism is one of Robert Chambers' seven canons for inclusive rigour in research, which advocates for the combination and recombination of diverse methodological approaches, theories, and perspectives to embrace complexity and generate rich insights. It endorses the idea that different, even conflicting, theories and methods can be useful when studying complex phenomena, and encourages researchers to be open to various ways of gaining knowledge, such as observation, experience, and experimentation.
Just me, or do these all seem like fancy speak for ‘bring your whole toolbox’?
I get that there are differences. But are these hairs worth splitting? Do I have to pick a tribe, like ‘Team Bricolage’ or something? Would it even be feasible and defensible to support just one of these philosophies without supporting the others? They all seem valid to me and broadly on the same team.
Perhaps it’ll help if we turn up the contrast? I asked a follow-up question:
What are the key similarities and differences between the three?
Perplexity replied:
Based on the search results, here are the key similarities and differences between mixed methods, bricolage, and eclectic methodological pluralism:
Similarities
All three involve combining multiple research methods, methodologies, theories, and perspectives within a single study.
They aim to provide a more comprehensive and in-depth understanding of complex research problems by drawing on the strengths of different approaches.
They reject the incompatibility thesis and embrace the idea that different, even conflicting, theories and methods can be productively combined.
Differences
Mixed methods specifically refers to integrating qualitative and quantitative methods within a study, while bricolage and eclectic pluralism are broader concepts that can involve combining various qualitative approaches or theories.
Bricolage emphasizes piecing together diverse methods, techniques, and perspectives in innovative ways, even moving beyond disciplinary norms. It is a form of methodological eclecticism.
Eclectic methodological pluralism, as defined by Robert Chambers, is one of the canons for inclusive rigour in research. It advocates for the combination and recombination of diverse approaches to embrace complexity.
Mixed methods tends to foreground the qualitative-quantitative divide, while bricolage and eclectic pluralism focus more on working across different methodological traditions, theories, and philosophical perspectives.
Bricolage is often associated with a pragmatist philosophy and a critical, social justice-oriented approach.
In summary, while all three involve methodological pluralism, mixed methods has a more specific focus on qualitative-quantitative integration, bricolage emphasizes innovative combinations beyond disciplines, and eclectic pluralism is a broader canon advocating diverse approaches to capture complexity.
Would an example help? I prompted Perplexity:
Use a scenario to contrast the three approaches
Perplexity replied:
Scenario: Studying the impacts of climate change on a coastal community.
Mixed Methods approach:
Quantitative component: Analyze meteorological data, sea level rise measurements, and economic indicators to assess the extent of climate impacts
Qualitative component: Conduct interviews and focus groups with community members to understand their lived experiences, perceptions, and adaptation strategies
Integration: Use qualitative data to contextualize and explain the quantitative patterns, and vice versa, to provide a more complete picture of climate change's multifaceted impacts
Bricolage approach:
Ethnographic immersion in the community to observe and experience their daily lives and cultural practices related to the coastal environment
Incorporate participatory mapping exercises to document local ecological knowledge and identify vulnerable areas
Analyze historical records, oral histories, and cultural artifacts to understand the community's relationship with the coastal landscape over time
Creatively synthesize these diverse methods and data sources to gain a rich, contextualized understanding of climate change's socio-cultural impacts
Eclectic Methodological Pluralism approach:
Engage indigenous epistemologies and traditional ecological knowledge through collaborative research with local knowledge holders
Utilize scientific modeling and GIS mapping to project future climate scenarios and their potential physical impacts
Draw on political ecology theory to examine power dynamics, policies, and decision-making processes around climate adaptation
Flexibly combine these varied approaches, embracing tensions and contradictions, to holistically study climate change's complex human-environment interactions
In this example, mixed methods integrates qualitative and quantitative data, bricolage creatively combines ethnographic, participatory, and cultural methods, while eclectic pluralism flexibly combines indigenous knowledge, scientific modeling, and critical theory to study the multifaceted issue of climate change impacts.
And the winning mixology?
Well, the good news is that I don’t think we need to split hairs about different mixological philosophies. I’ll take the works to go, thanks. Mixed mixologies, if you will.
My philosophy:
Bring all your tools and tricks, use them in contextually responsive ways, and keep people at the centre.
What do you think? Did Perplexity get the nuances right? Are you on Team Mixed Methods, Team Bricolage, Team Eclectic Methodological Pluralism, or Team Works Burger?
References provided by Perplexity
Brown, L. (2021). Bringing pragmatism and mixed methods research to transformative non-binary focused sociophonetic research. Toronto Working Papers in Linguistics, 43.
Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage Publications.
Creswell, J. W., & Plano Clark, V. L. (2018). Designing and conducting mixed methods research. Sage publications.
Dalland, C. (2022). The eclectic use of theory in educational research doctoral theses.
Denzin, N. K. (2012). Triangulation 2.0. Journal of Mixed Methods Research, 6(2), 80-88.
Haskins, N. H. (2016). Intersectionality: Engaging the epistemology of leadership theory.
Kincheloe, J. L. (2001). Describing the bricolage: Conceptualizing a new rigor in qualitative research. Qualitative inquiry, 7(6), 679-692.
Mäsch, N. (2021). Fostering autonomy in language learning.
Rogers, M. (2012). Contextualizing theories and practices of bricolage research. The Qualitative Report, 17(48), 1-17.
Tashakkori, A., & Teddlie, C. (Eds.). (2010). Sage handbook of mixed methods in social & behavioral research. Sage.
Vandenbussche, L., Ó Nualláin, S., & Timonen, V. (2023). Making sense of making sense of time: Longitudinal narrative research on the experience of aging. Qualitative Research, 23(1), 16-34.
Additional references I recommend
Aston, T. and Apgar, M. (2022) The Art and Craft of Bricolage in Evaluation, CDI Practice Paper 24, Brighton: Institute of Development Studies, DOI: 10.19088/IDS.2022.068
Chambers, R. (2015). Inclusive rigour for complexity. Journal of Development Effectiveness, 7(3), 327–335. https://doi.org/10.1080/19439342.2015.1068356
Is it feasible to combine so many methods, disciplines, designs, epistemologies, axiologies, approaches to synthesis, approaches to reasoning, perspectives, and lifeworlds? Yes! Keep calm and follow the 8 steps. www.julianking.co.nz/vfi/
Perplexity is a wonderful tip - thank you!
And an extra reference (thanks Thomas Aston!) https://journals.sagepub.com/doi/10.1177/13563890241232405