A few very successful programs, or many, connected, somewhat successful programs? Part 9 of a 10-part series on how complexity can produce better insight on what programs do, and why

Common Introduction to all sections

This is part 9 of 10 blog posts I’m writing to convey the information that I present in various workshops and lectures that I deliver about complexity. I’m an evaluator so I think in terms of evaluation, but I’m convinced that what I’m saying is equally applicable for planning.

I wrote each post to stand on its own, but I designed the collection to provide a wide-ranging view of how research and theory in the domain of “complexity” can contribute to the ability of evaluators to show stakeholders what their programs are producing, and why. I’m going to try to produce a YouTube video on each section. When (if?) I do, I’ll edit the post to include the YT URL.

Very successful programs, or many, connected, somewhat successful programs?

This is the third blog post in this series that touches on the on the notion that having a less successful program may be better than having a more successful program. (The first was Part 4 Complex behavior can be evaluated using comfortable, familiar methodologies. The second was Part 6: Joint optimization of unrelated outcomes.) A common theme in these posts is that unpredictable changes are likely when multiple isolated changes connect.

If these kinds of effects can result from networking, one could argue that resources are better invested in multiple programs, each of which is somewhat successful, than in a few programs, each of which is highly successful. (Of course, there is no guarantee whatsoever that these changes will be desirable, nor is it certain that the networking effects will result in greater total of positive change. But for now, I’m looking on the bright side – both the direction and amount of change will be desirable.)

To illustrate, consider the two scenarios depicted in Figure 1. In both, red arrows depict the short-term success of the program on a ten-point scale.  The scenario on the left shows four very successful programs, each of which works in isolation from the others. The scenario on the right shows six somewhat successful programs, each of which has dense connections with the others.

In the short term, the scenario on the left is the most beneficial. After all, just counting outcome points shows a total of 35, versus 19 on the right. But what might happen over time? I have no idea, but I’ll spin three not too farfetched possibilities.

Changes specified in the model:  Note that in Figure 2 there are three direct connections with girls’ education: 1) SME capacity, 2) civic skills, and 3) crop yield. The SME and crop yield relationships are plausible because of two constructs not shown in the model, namely family income and affordability of school fees. The model is: crop yield and/or SME capacity –> family income –> affordability of school fees.  The civic skills relationship makes sense because as people learn to participate in civic life, the quality of education is likely to get better.

The direct relationships in the previous paragraph are augmented by an indirect relationship: civic skills –> SME capacity. Why might this relationship make a difference? Because SME capacity leads to family income, so all programs that affect SME capacity will influence girls’ education.

Changes not specified in the model:    It is not hard to imagine that SME owners will be among those who participate in civic skills training programs, and that such participation will bring business owners together who did not previously know each other.  Between the new connections and the new skills, is it too hard to imagine that novel business opportunities will develop, or that creative ways to solve community problems will be revealed?

Sustainability consequences of collective change across multiple programs   All the changes described above can be thought of as generalized improvements in a higher-level construct called “community functioning”. Over time, “community functioning” may circle back to the original programs, and thus improve their outcomes as well.

To take just one small possibility as an example. Imagine that the highly competent administrator of the malaria prevention program left for another job. What is the likelihood of an equally competent administrator being available to take his or her place? Those odds are better if the networking effects shown on the right in figure 1 have been operating. Why? Because increased wealth in the community may result in a salary that would make the job desirable, and because the overall quality of life in the community may make it a desirable place to live.

Figure 2 is a graphical view of a possible set of consequences of implementing the models shown in Figure 1.

• Initial outcomes in the non-networked scenario exceed the initial outcomes for the same programs in the networked scenario. (Black stripes for initial non-networked, blue stripes for initial networked.)
• In the non-networked scenario, the level of program outcomes remain constant over time. (Black stripes versus black solid.)
• In the networked scenario outcomes grow so that over time, in five of the six cases, the networked outcomes exceed their initial values. (Blue stripped versus blue solid.)
• In three of the six cases, networked values grow to exceed non-networked values. (Blue solid versus black solid.)
• A, B, and C represent desirable changes that resulted from network effects that could not have been envisioned for each of the programs in isolation. These cases do not have accompanying black lines because they were never part of the outcome models for the original programs.
• D is there to remind us that we should never assume that all program outcomes (especially when not part of the program model) will be desirable. If I wanted to be pessimistic, but maybe not too farfetched, I could have included E, F, G, and H, all of which I would have depicted as negative.

An implicit assumption in the above is that networking relationships have either been designed into the programs or implemented in such a way that they evolve naturally. (For example, all the programs are in the same geographical area or administrative unit.) It is by no means certain that networking relationships will develop. Whether they do, and what they do, are empirical questions that need to be addressed as part of the evaluation.

The above paragraph implies that program designers have considered the possibility of connections among their programs. Part 3 (Ignoring complexity can make sense) makes the point that designers may not be doing their jobs well if they did try to build those connections. But even if they don’t, evaluation can still provide valuable insight about the complex behavior that the program designers are ignoring. That is the subject of the next (and thankfully final) section of this series of blogs (Part 10 Evaluating for complexity when programs are not designed that way).