Why should evaluators care about emergence? Part 7 of a 10-part series on how complexity can produce better insight on what programs do, and why

Common Introduction to all sections

This is part 7 of 10 blog posts I’m writing to convey the information that I present in various workshops and lectures that I deliver about complexity. I’m an evaluator so I think in terms of evaluation, but I’m convinced that what I’m saying is equally applicable for planning.

I wrote each post to stand on its own, but I designed the collection to provide a wide-ranging view of how research and theory in the domain of “complexity” can contribute to the ability of evaluators to show stakeholders what their programs are producing, and why. I’m going to try to produce a YouTube video on each section. When (if?) I do, I’ll edit the post to include the YT URL.

Why should evaluators care about emergence?

What’s the difference between an automobile engine (Figure 1 and a beehive (Figure 2)? After all, in each the whole is larger than the sum of its parts.

The answer is that for the engine, it is possible to explain what each part is and what role that part plays in the functioning of the engine.  I can tell you shape of a cylinder, how it is constructed, why it is needed to contain combustible material, how it moves up and down and is attached to the crankshaft, and so on. When I finished, you would know how an internal combustion engine worked and how a cylinder contributes to the overall functioning of the engine.

I could not give you such an explanation for how any single bee contributes to the construction or functioning of a beehive. The beehive materia

lizes when all those bees interact with each other as they do their simple bee things. That is emergence.

Change happens when the parts of an engine are assembled. Change happens when bees do bee things. But the type of change is different. Only the latter is an emergent phenomenon

Bees are bees, but what of emergence at the human scale of people, organizations, social groups, and political entities? It’s easy to find many examples. Some of the ones I like are: 1) The fractal nature of market fluctuations cannot be explained by the behavior of individual buyers and sellers. 2) The number of patents per capita in a city (when plotted logarithmically) increases more than the increase in a city’s population. 3) Traffic jams move in a direction opposite the direction of the flow of traffic. 4) the collective consequences of people, policy, business and infrastructure yields specialized districts in cities. In each of these cases, the behavior of the larger unit cannot be explained by breaking it down into its constituent parts.

Emergence matters in evaluation because it implies program theory that acknowledges that phenomena cannot be understood in terms of constituent parts. To take the example of specialized districts in cities. Any effort to understand the consequences of such districts for the city’s appeal to outsiders needs to be understood in terms of the impact of the district on city life. It would not help to do such an analysis by researching the individual people, policies, businesses and infrastructure that comprise the district. Those do not, and cannot, “add up” to “district appeal”. To extend the example, the appeal of the city to outsiders probably has to do with the entire group of specialized districts, and how those districts affect each other. Districts might be a meaningful unit of analysis, but the constituent parts of districts would not.

Why not try to do the analysis in terms of the constituent parts? The reason has nothing to do with our analytical capabilities or our access to data. The reason is that because these are emergent phenomena, it is no more possible to understand behavior in terms of its parts than it is to understand a beehive in terms of individual bees.

The challenge for evaluation centers on program theory, as illustrated in Figure 3. The difference between the model at the top and the model at the bottom does not seem all that dramatic. In fact, the difference is profound. The model at the top states that it is possible to understand “appeal” in terms of the individual contributions of people, policy, business, and infrastructure. The model at the bottom acknowledges that however one might understand “appeal”, it is not in terms of the unique contributions of people, policy, business, and infrastructure. The difference between the two models has very different consequences for

• Methodology
• Data requirements
• Stakeholder expectations, and
• What we can say about impact.