This post is an update to a post I did some time ago. I am adding to it based on some conversations I recently had at the annual meeting of the Canadian Evaluation Society. The topic I’m dealing with is the development of “Evaluation” through the lens of evolutionary biology. There are two related issues: 1) intellectual traditions, and 2) people’s affiliation as “evaluators”.
Here I am looking at the development of intellectual streams in evaluation For simplicity I’m assuming only two times in what is a long continuous process, and only two intellectual traditions. Also, I’m ignoring the possibility of linkages and networks among multiple traditions, and the multitude of reasons why the market for evaluation may change other than the behavior of evaluators. With those gross simplifications, my notions goes like this.
- There is a group of social scientists that self identify as “evaluators”, whatever that means to them.
- Evaluators live in an ecosystem of evaluation users, i.e. people and groups that may be interested in what evaluators have to say, or at least, to buy their services.
- Evaluators are influenced by various intellectual traditions, aka the roots, branches, limbs and twigs of the evaluation tree.
Over time the intellectual traditions “speak” to evaluators in such a manner that some traditions thrive and some do not. (All may thrive in various niches, but let’s leave that aside for the moment.)
- As evaluators are drawn from or withdraw from different traditions, the purveyors of those traditions themselves change because they are drawing inspiration from their audiences, learning from them, and thinking about how to cater to them. In my example this means a shrinking of tradition 1 and an expansion of tradition 2.
- The evaluator/tradition interaction changes the nature of evaluation as the evolving situation continues to draw in new people (and new kinds of people) who self identify as evaluators, while others leave.
- All the while all the evaluator ßà tradition interaction is influencing a set of relationships between evaluation and its stakeholders. This is because there are changes in what evaluation does, and what it can do. This changes the market for evaluation services.
- Needless to say, all these various changes and feedback loops can play themselves out in ever more interesting and complicated ways over time, but simplicity has its advantages, so I’ll stop now.
Affiliation as Evaluators
In this section I am thinking of the organization I know best — the American Evaluation Association. Up until the present day, the AEA has been characterized by two defining characteristics:
- Participation costs are low, both in terms of membership and the cost of annual conferences.
- Requirements for membership are minimal in terms of education, intellectual background, work experience, and so on.
Essentially, anyone who wants to join, can. One consequence of these conditions is that it is easy for AEA to grow. Another consequence is that the nature of the field is defined by its members, and not by the setting from which those members came.
To contrast this with a personal example. I am a PhD psychologist, and as such, I’m qualified to join the American Psychological Association. They would let me in. The reason they would let me in is because I would knock on their door already inculcated in the ways of professional and academic psychology. APA could rest assured that I had the necessary pedigree because they trust the Psychology Department of Northwestern University to make sure that I did. Of course I’m not claiming that APA is all that monolithic an organization. There is plenty of diversity there, and I’m sure the neighbors do not always get along. Still, anyone who comes to APA brings a reasonable degree of common educational and intellectual history into the organization. That commonality must certainly have its consequences for the directions in which APA can develop over time. AEA does not have these constraints.
Both AEA’s costs, and its membership requirements, give the organization a particularly high potential to evolve in many different ways.
Intellectual Tradition + Affiliation as Evaluators
In the first section I talked about how the interaction of intellectual tradition, needs of the membership, and the demand for evaluation, co-evolve to drive the development of the field. In the second section I tried to show that as the field develops, it can draw in a large and diverse membership. The larger the organization, the greater the effect it can have on both:
- its environment, and
- the “market value”of contributing the the development of the field.
My notion is that looking at things from this point of view might be a neat way of explaining the development of the field by looking at the feedback loops across three entities:
- People who practice evaluation
- Evaluation theory (and related intellectual endeavor)
- The ecosystem/market in which evaluation exists.
6 thoughts on “Using an evolutionary biology view to connect the intellectual development of evaluation and the development of the evaluation community”
A rich literature exists regarding the history and sociology of professions. It shows how knowledge disciplines compete to rise to the top of the occupation ladder. It turns out that the disciplines that rigorously control access to their designation make it – while those that don’t fail to make it. This is akin to a Darwinian process where only the fittest survive. As things now stand evaluation is not very fit (low remuneration; small numbers, etc.) but given their ignorance of this particular segment of the social science literature, most evaluation practitioners resist credentialing.
‘re “What then is the organisational ecology of evaluation?”
(my 2 cents worth)
Ecologies are defined by energy flows. The nearest equivalent when looking at organizations would probably be flows of money between organisations. E.g. consulting firms funded by bilateral and multilateral aid agencies, and consultants and others contracted by those consulting firms. As in biological systems, defining boundaries between one ecology and another is not straightforward But a social network analysis perspective would suggest looking for clusters of organisations whose members are more densely connected to each other by funding flows that they are to others.
My gut-level feeling is that there probably has not been too much investigation of these clusters, but I am standing by for immediate correction :-))
(I think I mentioned this before, if so sorry)…in my PhD fieldwork in Bangladesh in the 90s one part of that work looked at the population of NGOs receiving foreign funding. The question of how to sample the NGO population immediately hit the boundary definition problem, among others… One thing I remember well was how incredibly skewed the size distribution was. Which reminds me of the “O-ring theory of development” mentioned in a recent column by Tim Harford, and the conjectured role of associative sorting in the developments of inequalities between organisations. Don’t ask me too much more, I only have a cursory knowledge of this theory (see https://en.wikipedia.org/wiki/O-ring_theory_of_economic_development
Correction to last para above:.
Actually, looking for clusters does not seem to make much sense.
Better to look for ego-networks / supply chains