Joint optimization of unrelated outcomes – Part 6 of a 10-part series on how complexity can produce better insight on what programs do, and why

Common Introduction to all sections

This is part 6 of 10 blog posts I’m writing to convey the information that I present in various workshops and lectures that I deliver about complexity. I’m an evaluator so I think in terms of evaluation, but I’m convinced that what I’m saying is equally applicable for planning.

I wrote each post to stand on its own, but I designed the collection to provide a wide-ranging view of how research and theory in the domain of “complexity” can contribute to the ability of evaluators to show stakeholders what their programs are producing, and why. I’m going to try to produce a YouTube video on each section. When (if?) I do, I’ll edit the post to include the YT URL.

Joint optimization of unrelated outcomes

This blog is one of two in the series that discusses the possible advantages of having a less successful program than having a more successful program. The other is Part 9: Very successful programs, or many, connected, somewhat successful programs?

Figure 1 is a nice traditional model of an AIDS prevention/treatment program. The program is implemented, and services are provided. Because of careful planning, the quality of service is high. The combined amount and quality of service decreases the incidence and prevalence of AIDS. Decreased incidence and prevalence lead to improvements in quality of life and other similar measures. Because incidence and prevalence decrease, the amount of service provided goes down. However, at whatever level, the quality of service remains high. All these changes can be measured quantitatively. Change in the outcomes also affects the activities of the program, but for the most part, understanding those changes requires qualitative analysis.

There is nothing wrong with this model and this evaluation. I would dearly love to have a chance to do a piece of work like that. Note, however, an aspect of this program that characterizes every program I have ever seen. All the outcomes are highly correlated with each other. Because of the ways in which change happens, this can have some unpleasant consequences.

The unpleasant consequences can be seen by casting the AIDS program within a model that recognize that the AIDS program is but one of many organisms in a diverse ecosystem of health activities (Figure 2). (For a really good look at the subject of diversity and change, see Scott Page’s Diversity and Complexity.)

Looked at in those terms, the way change happens can have widespread, and probably not very desirable consequences. Table 1 explains the details in the model shown in Figure 2.

This is what happens when a single objective is pursued when a system is comprised of diverse entities with diverse goals. You will get what you worked for, but the system as a whole may be the worse off for it. What is the solution? The solution is to work at jointly optimizing multiple somewhat unrelated outcomes. “Somewhat” is an important qualifier because the range of objectives cannot be too diverse. In the AIDS example, all health care objectives certainly have some overlap and relationships to each other. It’s not as if the goals to be jointly optimized were as far apart as AIDS and girls’ schooling. Some coherence of focus is needed.

The above advice can be excruciatingly difficult to follow. One problem is that there is nothing obvious about what “joint optimization” means. AIDS prevention, tertiary care, and women’s health – imagine drawing a logic model for the goals of each of these programs. Then imagine the interesting conversations that would ensue on the topic of how much achievement of each goal was appropriate.

Indeed, one way to look at the simple model depicted by Figure 1 is that it is a program operating within an organizational silo. And as I tried to show in Part 3 (Ignoring complexity can make sense), operating within silos is rational and functional. I am by no means arguing that the model in figure 2 is in any way better than the model in figure 1, or that programs must be designed and evaluated with respect to one or the other. My only point in this blog post is to show that there is complex system behavior in the form of evolutionary adaptation that is likely to cause unintended undesirable consequences when efforts are made to pursue a set of highly correlated outcomes.

Finally, I know many people take a dim view of the dark scenario I painted above, namely, that the most likely unintended consequences of pursuing a single objective are negative. But I think I’m right. For an explanation, see the section “Why are Unintended Consequences Likely to be Undesirable?” in From Firefighting to Systematic Action: Toward A Research Agenda for Better Evaluation of Unintended Consequences