Joint Optimization of Uncorrelated Outcomes as a Method for Minimizing Undesirable Consequences of Program Action

This blog post is a pitch for a different way to identify desired program outcomes.

Program Theories as they are Presently Constructed

Go into your archives and pull out your favorite logic models. Or dip into the evaluation literature and find models you like. You will find lots of variability among them in terms of:

• Richness of feedback loops
• Numbers of intermediate steps
• Time frames for chains of outcomes
• Numbers of intermediate outcomes
• Recognition of environmental influences
• Models nested within higher level models
• Specificity – lines and arrows, or just “bunch to bunch”
• Types of relationships – 1:1, 1:many, many:1,many:many

Outcomes Expressed in the Models

All the models will represent the values and priorities of the program designers. That’s the strength of the models. The exercise of constructing them makes program designers identify what outcomes they desire.

Where do those desires come from? They come from the missions and objectives of the organizations in which those program designers reside. Here are some illustrative examples.

• Diabetes Group within the Division of Prevention in the Department of Health
  The programs will pursue objectives to help diabetics and incipient diabetics to control their disease.

• Inventory Support Office within the Department of Small Business Assistance in the Department of Economic Development
  The programs will try to help small business  adopt the principles of good inventory control.

• Distracted Driving Initiative within the Division of Safety within the Department of Transportation
  The programs will try to minimize device distraction.

• And so on.

A goal setting exercise may identify many outcomes, some of which will be antecedent to the end of an outcome chain. Also, there may be more than one outcome in the terminal position. But one thing is almost certain. All of the outcomes will be highly correlated. Whatever the desired directions of change, if one outcome gets better as a result of program action, so too will the others.

Another way of stating this is that all the outcomes are manifestations of a single construct. That construct may not even be visible in the model, but if the visible outcomes are correlated, the underlying outcome must be there.

I understand why programs are like this.

• Reward systems differ.
• Political and mission priorities differ
• Funds are dedicated for specific purposes.
• Disciplinary paradigms are difficult to bridge.
• Organizational boundaries are hard to transcend.
• Coordination becomes more difficult as scope widens.
• Coalitions in favor of implementing a program grow weaker as design compromises pile up.
• If you want an in-depth explanation, see my YouTube video Why Use Complexity When Programs Do Not?

The Dark Side

On the one hand, process that gets program designers to appreciate their constraints, assure alignment between action and mission, and identify outcomes, is a very good thing. But there is a dark side that touches both outcomes, and the workings of the programs that are designed to effect those outcome.

Outcomes

Pick anything that might be the target of an intervention to make things better – people, communities, school systems, county governments, neighborhoods, universities, wetlands, road systems, farms, export policies, democracy promotion programs, or anything else that piques your interest. I think of these as organisms working at climbing their fitness landscapes, but I know that many people do not like that way of thinking. So here I’ll just call them “boxes”. Whichever box you pick, I bet that a close look will show that for that box to thrive over the long term, it will need to meet a variety of needs. Whatever these needs may be, I’m pretty sure that:

• Their time-frames will vary.
• They will not all be equally important.
• They will not have be achieved at the same level success.
• Some will be more robust in the face of challenge than others.
• Some may spring into existence where they did not exist before.
• The same need may carry different salience under different conditions.
• Some may be latent and only become manifest under particular conditions.
• Some will become important because of workings inside the box, and some because the box needs to negotiate its relationship with its surrounds.
• But whatever their specifics, there are sure to be more than one.

While there may be times when a single acute need takes precedence over all the others, eventually more than one will need to be pursued. So what will happen if a program comes along that pushes a box to meet only one of its goals to the exclusion of the others? I don’t know, but whatever it is, it won’t be good. Eventually the box will have to do something to thrive. (I am speaking as if the box  has volition, which in many cases it will not. But for explanatory purposes, speaking in terms of volition has its advantages.) The box has two choices. It may:

• Ignore the objective the program is promoting, or
• pervert the pursuit of the program’s goal to serve other needs.

Whatever the choice, I am sure it will result in what program designers would almost certainly consider unintended, and probably undesirable, outcomes.  I have no idea what would happen to the box as a program moves away from an emphasis on helping the box achieve only one need. Nor do I have a clue as to how many objectives should be pursued in order to help  the box achieve  “healthy” outcome. What I am sure of is this.

• One alone is problematic.
• Too many will strain the resource and capabilities of the program.
• At some point multiple goals will probably become fairly correlated anyway, so there is no point in chasing too many.

So my advice? Pick two and see what  happens. But whatever the choice, don’t pick only one.

Programs

The most optimistic possible scenario is that a program could, if it wanted to, implement initiatives that would jointly optimize multiple goals. In such a scenario, all program designers would have to do is to repent the error of their ways.

What I believe though, is that once programs commit to a goal, it becomes very difficult for them to adapt to new needs. In essence, the choice to help boxes meet particular objectives drives a course of program development that makes it difficult for a program to do anything else. And as they keep  pushing boxes toward a single goal, the programs generate more of the unintended consequences that they would prefer to avoid. Why does this vicious cycle exist?

Because programs can be thought of as systems, and systems do not thrive over the long term if all of their resources, and all of their actions, and their entire structure, is dedicated to only a single outcome. Those kinds of systems:

• Are rigid, and break easily.
• Have limited capacity to monitor their environments.
• Have minimal internal capacity to adopt to new circumstances.
• Ignore the reality that even if clients have a single overriding need over the short term,  they are sure to have varying needs over the long term.

The last is most important. Pursuit of a single goal, or multiple goals that are highly related, forces systems to follow one or more paths:

• Incent boxes to opt out of the program.
• Ignore other needs that boxes are sure to have.
• Find ways to pervert themselves in the service of those other needs.
• Oscillate among some of the above options.

Whichever path is  pursued, the result will not be good.

• Incent boxes to opt out of the program
  I suppose that avoiding an otherwise beneficial program is one sure way to avoid the unintended consequences of having that program in operation.

• Ignore other needs that boxes are sure to have.
  Sooner or later all the problems I mentioned will rear their ugly heads.

• Find ways to pervert the program in the service of those other needs
  Sure, if  you want to miss the chance of meeting intended program goals, not do a very good job of meeting all the other needs, and loose control over the program.

• Oscillate among some of the above options.
  A guaranteed way to waste resources and accomplish nothing.

Doing Things Differently

A big reason for my writing this blog post is to begin a conversation about changing program design to a mode in which program success is routinely defined as the

• joint optimization of outcomes that are as
• uncorrelated as possible.

I don’t think that program objectives can or should be too uncorrelated. After all, widespread coordination is difficult, and organizations do have their own,  unique, legitimate missions. Still, it is possible to make choices. Some examples:

• A language arts program that
  pursues reading achievement on test scores
  and
  incenting students to read for pleasure.

• A family support program that pursues
  work ethic
  and
  giving more attention to children.

• A STEM program for girls that teaches
  coding skills
  and
  high level conceptual elements of application architecture.

In all of these examples it would be easy to drop one goal and to pursue the other. But it would be very difficult to keep both goals and to design a program that deliberately chose not to maximize one goal because doing so would ignore the other.

I have no illusions that my idea would be easy. Here are but a few of the difficult questions that would need to be addressed.

• What happens if clients demand a single outcome solution?
• What does “joint optimization” mean, i.e. how much of each goal represents the best joint outcome?
• What to do about the fact that entire bureaucratic and funding structures drive programming toward single-outcome solutions?

Inching Toward a Solution

I have no answers to these questions, but I do have some ideas about how to go about addressing them.

My Proposed Innovation

• Begin program design by legitimizing the question of joint goal optimization during early planning stages.

• Get the question into the dialogue stream, and see what happens from there.

• Once planning begins, recognize the possibility of joint optimization in program models. The content of models makes people pay attention. Attention must be paid. Make them pay attention, and let the planning process play out.

Evaluating the Innovation

Questions
If I were to evaluate an experiment that attempted these tactics, what questions should the methodology address? Here is my list.

• Are needs assessments conducted differently?
• Is there a change in the content of budget requests?
• Is there a change in what funders ask for in their RFPs?
• Is there a change in reward systems for program designers?
• Is there a more diverse mix of human capital expertise in the planning process?
• Do internal planning processes become more accommodating to multiple-goal planning?
• Is there a change in how program personnel negotiate with higher levels of their organization?
• Are there changes in how planners interact with stakeholders, or in which stakeholders they interact with?
• Most important, does the aggregate effect of all those changes result in an organization that was more responsive to clients’ needs, and more capable of meeting those needs over the long term?

Methodology
As for methodology, I’d like to see multiple comparative case studies unfold over a few years’ time. My thought is to begin to recruit funding organizations to try to implement the changes I am dreaming about. For each one that makes the effort, pick a similar one that has not. Now that is an idea for a multiple comparative case study design that will keep a few graduate students busy for a while.

Next steps

What I’d really like is to find some funding to do the research I described in the previous section. But first things first. What’s needed is to coalesce interested people to explore the idea of joint optimization in program design.

• The easy (aka cheap) step is to set up a discussion forum in cyberspace.
• The next step would be to find some funding to convene a small group to outline:
  • critical issues
  • theoretical underpinnings of change, and
  • a research agenda.

Who to invite?

• as diverse group of knowledgeable participants as possible, to
• deliberate about joint outcome optimization in a particular domain, e.g. health, or economic development.

Why diverse participants?

• Because what I’m proposing is will require creative thinking about difficult change.

Why a single domain?

• Because I think that context-specific dynamics may be at play.