Common Introduction to all sections

This is part 6 of 10 blog posts I’m writing to convey the information that I present in various workshops and lectures that I deliver about complexity. I’m an evaluator so I think in terms of evaluation, but I’m convinced that what I’m saying is equally applicable for planning.

I wrote each post to stand on its own, but I designed the collection to provide a wide-ranging view of how research and theory in the domain of “complexity” can contribute to the ability of evaluators to show stakeholders what their programs are producing, and why. I’m going to try to produce a YouTube video on each section. When (if?) I do, I’ll edit the post to include the YT URL.

Part Title Approximate post date
1 Complex systems or complex behavior? up
2 Complexity has awkward implications for program designers and evaluators up
3 Ignoring complexity can make sense up
4 Complex behavior can be evaluated using comfortable, familiar methodologies up
5 A pitch for sparse models up
6 Joint optimization of unrelated outcomes up
7 Why should evaluators care about emergence? 7/16
8 Why might it be useful to think of programs and their outcomes in terms of attractors? 7/19
9 A few very successful programs, or many, connected, somewhat successful programs? 7/24
10 Evaluating for complexity when programs are not designed that way 7/31

Joint optimization of unrelated outcomes

This blog is one of two in the series that discusses the possible advantages of having a less successful program than having a more successful program. The other is Part 9: Very successful programs, or many, connected, somewhat successful programs?

Figure 1: Typical Model for an AIDS Program

Figure 1 is a nice traditional model of an AIDS prevention/treatment program. The program is implemented, and services are provided. Because of careful planning, the quality of service is high. The combined amount and quality of service decreases the incidence and prevalence of AIDS. Decreased incidence and prevalence lead to improvements in quality of life and other similar measures. Because incidence and prevalence decrease, the amount of service provided goes down. However, at whatever level, the quality of service remains high. All these changes can be measured quantitatively. Change in the outcomes also affects the activities of the program, but for the most part, understanding those changes requires qualitative analysis.

There is nothing wrong with this model and this evaluation. I would dearly love to have a chance to do a piece of work like that. Note, however, an aspect of this program that characterizes every program I have ever seen. All the outcomes are highly correlated with each other. Because of the ways in which change happens, this can have some unpleasant consequences.

The unpleasant consequences can be seen by casting the AIDS program within a model that recognize that the AIDS program is but one of many organisms in a diverse ecosystem of health activities (Figure 2). (For a really good look at the subject of diversity and change, see Scott Page’s Diversity and Complexity.)

Figure 2: Casting the AIDS Model into an Ecosystem Based Health Care Model

Looked at in those terms, the way change happens can have widespread, and probably not very desirable consequences. Table 1 explains the details in the model shown in Figure 2.

Table 1: Explanation of Figure 2-
Upper right
  • The ecosystem of health services is arranged in a radar chart to show how much each service contributes to overall health in the community. There is no scale on the chart, but the absolute value of each service quality does not matter. All that matters is that under the circumstances, each service is about as good as it can be.
Upper left
  • This is the model of our very successful AIDS prevention and treatment program, as shown in Figure 1.
Lower left
  • This is a chart of what can happen to health system resources when an overriding priority is put on AIDS, to the exclusion of everything else. Resources flow from the rest of the system to AIDS. When I say “resources” I do not mean just money. I mean everything about a health care system that is needed for the system to function well.
Lower right
  • This radar chart shows the status of the system after the AIDS effort has been in operation for a while. Indeed, the AIDS measures improve. But what of the other services? How do they accommodate to nurses choosing to move to AIDS care, or policy makers time and intellectual efforts pointed in the AIDS direction, and so on? It seems reasonable to posit that whatever happens to those other services, it will not be to their advantage. Their environment has become resource poor.

This is what happens when a single objective is pursued when a system is comprised of diverse entities with diverse goals. You will get what you worked for, but the system as a whole may be the worse off for it. What is the solution? The solution is to work at jointly optimizing multiple somewhat unrelated outcomes. “Somewhat” is an important qualifier because the range of objectives cannot be too diverse. In the AIDS example, all health care objectives certainly have some overlap and relationships to each other. It’s not as if the goals to be jointly optimized were as far apart as AIDS and girls’ schooling. Some coherence of focus is needed.

The above advice can be excruciatingly difficult to follow. One problem is that there is nothing obvious about what “joint optimization” means. AIDS prevention, tertiary care, and women’s health – imagine drawing a logic model for the goals of each of these programs. Then imagine the interesting conversations that would ensue on the topic of how much achievement of each goal was appropriate.

Indeed, one way to look at the simple model depicted by Figure 1 is that it is a program operating within an organizational silo. And as I tried to show in Part 3 (Ignoring complexity can make sense), operating within silos is rational and functional. I am by no means arguing that the model in figure 2 is in any way better than the model in figure 1, or that programs must be designed and evaluated with respect to one or the other. My only point in this blog post is to show that there is complex system behavior in the form of evolutionary adaptation that is likely to cause unintended undesirable consequences when efforts are made to pursue a set of highly correlated outcomes.

Finally, I know many people take a dim view of the dark scenario I painted above, namely, that the most likely unintended consequences of pursuing a single objective are negative. But I think I’m right. For an explanation, see the section “Why are Unintended Consequences Likely to be Undesirable?” in From Firefighting to Systematic Action: Toward A Research Agenda for Better Evaluation of Unintended Consequences

One thought on “Joint optimization of unrelated outcomes – Part 6 of a 10-part series on how complexity can produce better insight on what programs do, and why

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s