Common Introduction to all sections
This is part 2 of 10 blog posts I’m writing to convey the information that I present in various workshops and lectures that I deliver about complexity. I’m an evaluator so I think in terms of evaluation, but I’m convinced that what I’m saying is equally applicable for planning.
I wrote each post to stand on its own, but I designed the collection to provide a wide-ranging view of how research and theory in the domain of “complexity” can contribute to the ability of evaluators to show stakeholders what their programs are producing, and why. I’m going to try to produce a YouTube video on each section. When (if?) I do, I’ll edit the post to include the YT URL.
Complexity has awkward implications for evaluators and stakeholders
Complex behavior is problematic because it has implications for program outcomes that either do not conform to common sense, or which challenge accepted processes of program design, or both. Table 1 shows some examples. Table 2 outlines the complex behaviors that explain the outcome patterns.
Table 1: Examples of complexity-driven program behaviors that have awkward implications for program designers
| Program Behavior | Implications for Common Sense and/or Design and Funding Logic | |
| 1 | Benefits are highly skewed toward a small number of service recipients. | This pattern may not be an aberration or a fault of the program design. Rather, it may be fundamental to the program and the conditions in which it is operating. Despite this inevitability, both politics and ideology favor a reasonably balanced distribution of program effects. It’s not pleasant to contemplate that the fundamental nature of an innovation, no matter how valuable that innovation may be, will be distributed in a highly unequal manner. |
| 2 | Understanding program effects defies understanding in terms of outcome chains identified in the program theory.
|
Psychology: As humans, we have a natural desire to take things apart and see how the pieces fit back together. It’s unsettling to think that looking at the pieces will not help us understand what we have taken apart.
Political, economic, and social realities: We live in a world where incremental efforts are needed when a long-term objective is pursued. That reality makes it difficult to admit that we can’t explain how incremental change adds up. |
| 3 | A program can be relied upon to produce long term outcomes, but a chain of intermediate outcomes cannot be identified in advance. | The difficulty here is like the one above. It does not make sense that we can reliably predict where a program will end up, but we cannot identify the intermediate steps. And selling that assertion to funders can be no small effort. |
| 4 | Achieving program goals induces dysfunctional change in related programs. | Our funding mechanisms are “stovepiped”, and thus optimized to achieve a single outcome, or at least, a set of highly correlated outcomes. It is disconcerting to contemplate the possibility that success within those stovepipes will of necessity, breed undesirable change in other activities that we care about. |
| Table 2: Complex behaviors that explain the program performance | |
| 1 | Benefits are highly skewed toward a small number of service recipients.
Complex behavior = preferential attachment Preferential attachment refers to a process by which one “entity” connects with another based on “size”. One frequently cited example is the Internet, with its set of larger hubs connecting to smaller ones. Snowflakes also follow this pattern, as does wealth. What do these seemingly different constructs have in common? They can all be thought of as random processes. If you are going to link to a URL, is there a greater chance that you will know about larger or smaller URL possibilities? If you are an ice particle are you more or less likely to find and bind to a larger or smaller collection of particles? If you are a business opportunity, are you more or less likely to seek larger or smaller centers of partnering potential? The direction in these examples can be turned around. If you are a large URL, is there a higher or lower chance that you will attract potential connections? If you are a snowflake, is there a higher or lower chance that you will bump into ice particles? If you are known to possesses resources, is here a higher or lower probability that you will attract partners? Other characteristics of these patterns is that they are fractal and the sizes of their connections are power law distributed. |
| 2 | Understanding program effects defies understanding in terms of outcomes identified in the program theory.
Complex behavior = emergence Emergence is a phenomenon in which the functioning of an entire unit cannot be explained in terms of how individual parts interact. Contrast an automobile engine with a beehive, a traffic jam, or an economy. I could identify each part of the engine, explain its construction, discuss how an internal combustion engine works, and what role that part plays in the operation of the engine. The whole engine may be greater than the sum of its parts, but the unique role of each part remains. The contribution of each individual part does not exist with beehives, traffic jams, or economies. With these, it may be possible to identify the rules of interaction that have to be in place for emergence to manifest, but it would still be impossible to identify the unique contribution of each part. |
| 3 | A program can be relied upon to produce long term outcomes, but a chain of intermediate outcomes cannot be identified in advance.
Complex behaviors = sensitive dependence and attractors “Sensitive dependence” refers to a phenomenon in which small perturbations can result in a radical change in the trajectory of the system. “Attractor” is a set of conditions that constrain the states in which a system can exist. It is well within the bounds of possibility that sensitive dependence precludes identifying how an entity will move within its attractor, but that the attractor will constrain the conditions in which the system can find itself. |
| 4 | Achieving program goals induces dysfunctional change in related programs.
Complex behaviors = evolution / adaptation within an ecosystem One can think of programs as organisms that are attempting to maximize their viability on a fitness landscape. In that sense the program can be thought of as competing with the other programs with which it shares an environment. In situations like this, anything that changes the allocation of resources among the organisms will result in adaptations to new realities. Put in a language of more resonance to planners and evaluators, there will be unexpected (and probably undesirable) consequences to implementing an effort to maximize only one outcome. As an example, consider a suite of health care services – AIDS, prenatal care, women’s’ health, tertiary care, and so on. What would happen to other elements of that health care suite if money, the most interesting jobs, planners’ intellectual effort, and networks of informal relationships all flowed into AIDS efforts? |
From the point of view of doing evaluation, none of the complex behaviors, or their consequences for programs, are difficult to address. (See Part 4 Complex behavior can be evaluated using comfortable, familiar methodologies.) Getting program designers on board, however, is a different matter. One of my intentions in this blog series is to convince evaluators that program designers are acting rationally when they ignore complexity, but that productive dialogue about the complex can still be had. (See Part 3 Ignoring complexity can make sense.) And in any case, it’s a possible and worthwhile to evaluate based on complexity even when programs are not designed that way. (See Part 10 Evaluating for complexity when programs are not designed that way.)
Evaluation Uncertainty 
Hey Jonny, what’s going on here. There’s no blog posts under the headings (although I have an idea that I’ve read this series before – or at least an earlier version)
>
Look at the schedule in the table. The first part is published. The others will show up as indicated.