Why go to the trouble of applying concepts from Ecology and Evolutionary Biology to Evaluation? Concepts from these fields are alien to Evaluation, so incorporating them into our work would be no small task. Much of my writing of late has been to make the case for that “no small task”. I came to realize, however, that what I have been writing concerns targets of opportunity, i.e. evaluation scenarios where this or that concept from Ecology or Evolutionary Biology would come in handy. That is a far cry from making the case that Ecology or Evolutionary Biology can make foundational contributions to Evaluation. This blog post is my initial shot at making that case. (“Ecology and Evolutionary Biology” is a mouthful, so I tend to fall back on just using the term “Ecology”.)
Traditional evaluation focuses on a program. Two fundamental questions are:
- How does the program work?
- What consequences does the program produce?
Based on the answers to these questions, two other questions become relevant.
- Will the program work in other settings?
- What is needed to make it sustainable?
What are the implications of taking an ecological perspective?
There may be value in looking at programs in an entirely different way that shifts the focus from the program itself, up a level to the environment in which the program operates, and down a level to the entities the program is trying to change.
From program to classes of programs: The focus moves from the entity that is being evaluated, to the class of programs to which that entity belongs. The question is what consequences the existence of that class of programs has, irrespective of the operation or effects of any single member of the class. For instance, a traditional evaluation would focus on a particular community development program (or model of community development with multiple implementations) and inquire as to the consequences of that effort for communities. The ecological lens, however, does not “care” about the single organisms that make up a species. It only “cares” about the health of the species and its consequences for the ecosystem in which that species resides. Compared to a single-program focus, the ecological view would drive an entirely different set of models, methodologies, and data interpretations.
Objects of a program’s efforts: Here the shift is a focus from the program, to the entities the program is trying to change, e.g. people, schools, communities, or whatever. Part of this is our old friend outcome measurement. But another element is the collective behavior of those elements, over and above the means and standard deviations of change. One could look at this scenario as if the entities served constituted a “species” inhabiting their own ecosystem. As above, here too the ecological view would drive an entirely different set of models, methodologies, and data interpretations.
Why bother?
Evaluation and Ecology are separate disciplines, each with its own favored approaches to theory, models, methodologies, the human capital makeup of teams, and the characteristics of acceptable answers. It is not a trivial exercise to bring the world view of one discipline into another. Why make the effort? The effort is worthwhile because ecology provides a framework to understand the systems in which our programs are embedded. For instance, consider a few of the concepts that are routinely considered in ecological research: 1) birth and death rates of existing species, 2) shape of the fitness landscapes that describe vulnerability to change, resistance to change, and sustainability, 3) symbiotic relationships, and 4) competition. All of these can be seen as system characteristics, and all are subject to forms of quantitative and qualitative understanding that are lacking in Evaluation.
Evaluation Uncertainty 