PDF version: Schedules_logic_models_complexity_Morell_2_18_22

In 2018 I published an article making the case for using project schedules (in the form of PERT and Gantt charts) as logic models (Morell, 2018).[2] I offered two reasons for doing so.

• Using a project schedule as a logic model can reveal unique insight.
• Because planners and managers pay more attention to program schedules than to evaluation logic models, casting an evaluation in a planning framework will improve evaluation use.

I won’t go into details here, but in broad overview, the article went through a five-step sequence: 1) How do models function in the research enterprise? 2) What makes a project schedule a model? 3) How can a project schedule serve as an evaluation logic model? 4) What knowledge can be gained by using a project schedule as an evaluation logic model? 5) Case example involving evaluation of a peer-to-peer intervention to decrease electronic device distraction in a large transportation setting. (For a blog post version of this argument see: (Morell, 2014)).

The purpose of this blog post is to begin to extend the above argument by folding in a third realm of knowledge to the schedule/logic model combination. “Begin” is the operative word. This post is nothing more than my impressions about how two constructs from Complexity Science can enhance the value of employing a schedule/logic model analysis. I would like to develop an overall framework, but for now the best I can do is to offer a few discrete examples.

In the discussion that follows I will speak in terms of “complex behaviors” rather than “complex systems”. I do this because I know how to incorporate the behavior of complex systems into my work, but I do not know how to change my practice based on the knowledge that I am dealing with a complex system.

Because the focus of this post is on the behavior of complex systems, I will not address the question of whether understanding program behavior in terms of program schedules should be embedded in a general context of systems thinking, although doing that would be a very good idea (Vidal, 2009). After all, the sharp boundaries of a program schedule bleed into a broader set of environmental factors. It is true that systems thinking requires drawing boundaries to identify what should and should not be included, but it seems overly restricted to draw those boundaries as contemporaneous with the tasks, milestones, and deliverables in PERT or Gantt charts.

Because project schedules usually deal with program design and implementation, most of the value of using schedules as logic models falls into the “formative/process” territory in the evaluation landscape. There is also relevance for developmental evaluation because schedules include milestones and deliverables, both of which relate to the reasons a program is being implemented. There might also be relevance for evaluating outcomes to the extent that milestones constitute outcomes or can be seen as “outcome adjacent”. The two complex system behavior examples I will use are:

• Stigmergy
• Magnitude and rate of change

In each case there are implications for understanding some combination of three themes: 1) pattern, 2 predictability, and 3) how change happens. Table 1 shows the possible relationships.

Table 1: Relationships Between Complex Behaviors and Complexity Themes
Complex behavior	Theme in complexity science
	Pattern	Predictability	How change happens
Stigmergy
Magnitude and rate of change

Stigmergy

Project management is by its nature a well ordered, controlled, hierarchical process. That is its strength. But project execution is done by the people who work in a social process that is characterized by personal choice in the face of circumstance. No matter how many layers of detail in a work breakdown structure, there is always choice with respect to how and when things should be done. That is why beyond the formalities, just as in the military, there is a necessity for choice based on knowledge of “commanders’ intent”, i.e., the objectives being pursued.

The above discussion spoke in terms of “people” because that is the easiest way to explain stigmergy. However, the “deciding unit” can be any entity that is capable of sensing its environment and making decisions based on the acquired knowledge – “agents” in the argot of complexity. For instance, an elaborate project plan may require multiple small groups doing different parts of a task. In a setting like that a decision to act might best be attributed to a group, not to the people who make up that group.

As an example, imagine an agricultural support program whose aim is to improve crop yields by promoting a new seed variety, accompanied by training in the knowledge and farming activities needed to exploit the value of the new crop variety. There may be nothing in the project plan that specifies discussion of marketing tactics for deriving value from crop yields, but project personnel may perceive a need for that knowledge and include it in their training efforts. Once that unit on marketing has been included, a path has been set that may influence further decisions about training content that both elaborates on the new training content and builds on it with respect to related topics. Looking back, an evaluator might observe the appearance of an entire new dimension to the training, and moreover, may discover that the business focus of the training had a discernable impact on farmers’ behavior and may even be a key factor in explaining the success of the entire program. It is not inconceivable that the formal activities, milestones, and deliverables for similar programs will contain marketing training as a key feature. What happened? What happened was that an independent decision affected choices that influenced the design of the program.[3]

Now consider a sociologist from Mars who was able to observe the structure and activities of the program but did not have the data or the methodology to know how decisions in the program were arrived at. Would that sociologist be able to know whether the structure of the program derived from a central planning process or from the cumulative effect of independent action? No, because the structure would be the same whether central planning or stigmergent behavior generated that structure.

In a stigmergic process, agents have no direct knowledge of what other agents have done. All they know is the situation that history has presented them with. There is no overall plan. There is only agents’ perception of their current situation and a set of decision rules. The “plan” is an expression of cumulative consequence of independent decisions. (Stigmergy was developed during the study of insect behavior but has been generalized to human-scale activities (Parunak, 2006; Theraulaz & Bonabeau, 1999)).

Implications for Evaluation

The implications touch on both theory of change and methodology.

Theory of Change

It might be a bit of a stretch, but it is not too unreasonable to think of a project plan as a theory of change with respect to going from not having a program to having a program, or from program state A to program state B. If stigmergent behavior plays a role in organizing that change, then the theory of change must:

• conceptualize success in terms of an interplay of two different drivers of change, and
• be sensitive to the contextual factors and decision rules that affect the decisions made by individual actors.

It is not unusual for evaluators to refer to change processes that are comprised of a combination of top-down and bottom-up activity. The added value of invoking stigmergy is that the concept provides a theoretical understanding of how some bottom-up change may come about. I do not want to leave the impression that all bottom-up change is a stigmergent process. I do want evaluators to consider bottom-up change as a phenomenon worthy of methodological effort, and I do want them to consider “environment sensing/independent choice behavior” as an explanatory factor in that bottom-up change.

Finally, while the question of long-term coordination is beyond the scope of a discussion of PERT and Gantt charts, it is worth considering how engineering stigmergic behavior can facilitate coherent action among stovepiped activities.(For an explanation of why this is so, see (Morell, 2021c), or page 360 in (Morell, 2021a)).

Methodology

In order to understand why a program is implemented as it is, and why it takes the that form it does, it is necessary to include a methodology that can detect meaningful stigmergic behavior. “Meaningful” is an important word because most independent action will not have long term consequences. Thus, there is a need to detect independent action, determine whether that action may be consequential, and assess the implications of those consequential actions.

Evaluation methodology must be sensitive to the consequences of stigmergic behavior for formal planning. It is important to know if, how, and when the stigmergic evolution of a program is recognized in the processes that capture timelines, budgets, and deliverables. The only way to understand a program’s development is to know the continual interaction of planned and stigmergic behavior.

Finally, unplanned action may yield unexpected results, and evaluation must be capable  of assessing those results (Morell, 2010).

Themes

Developing and attending to implementation models (Gantt and PERT charts) implies a belief that if successful change is to take place, if a program implementation is to succeed, a plan must be developed and applied. I believe that is true. Coordinated action is, in fact, an important way in which change takes place. However, the possibility of stigmergic behavior tells us that there is also another way for change to take place, Table 1 begins to fill.

Table 2: Stigmergy Relationship to General Themes in Complexity
Complex behavior	Theme in complexity science
	Pattern	Predictability	How change happens
Stigmergy			X
Magnitude and rate of change

Magnitude and Rate of Change

How often, and by how much, do timelines and budgets change? We have all witnessed these phenomena, but how many of us have taken them seriously – as indicators worthy of systematic attention? Before I started thinking of schedules as models, I never did. Upon reflection though, changes like these can be plotted in terms of “size x frequency”, an analysis that has been enlightening across a very wide range of phenomena in many scientific disciplines (Andriani & McKelvey, 2005). A few of numerous possible examples include earthquakes, income distribution, city size, firm size, aggressive behavior, and co-authorships. In all these cases there is a small number of high magnitude cases followed by a quickly decreasing number of lower and lower magnitude cases. The precise shape of the distributions, however, vary with the phenomenon being observed. (Typically, the distribution shapes give the impression of power laws, but  the question of whether they are true power laws is the subject of much research and debate (Dept. of Geography University of Kentucky, 2015; Klarreich, 2018; Smolyarenko Artico I, Vinciotti V, & Wit EC, 2020)).

Implications for Evaluation

As with stigmergy, here too there are implications for both theory of change and methodology.

Theory of Change

One perspective on the discussion above is tactical. It is a good idea to give decision makers knowledge about the magnitudes and frequencies of change in time and budget. Those magnitudes and frequencies, however, reflect deeper realities about regularity and uniqueness in program implementation. Planners can anticipate the general shape of the distribution of deviations from plan, but the specifics of those deviations are context dependent.

Another theoretical consideration is the matter of the existence of tasks, milestones, and deliverables. As we know from a developmental evaluation perspective, what programs do, what they want to do, and how they go about doing it, can evolve over time. One could cast evaluation of these kinds of changes in terms of Evolutionary Biology and Ecology (Morell, 2020, 2021b, 2021d). In those terms, tasks, milestones, and deliverables can be viewed as organisms dwelling in an ecosystem, subject to birth and death. Such a framework conjures several worthwhile questions. What are the characteristics of the organisms (tasks, milestones, deliverables) that appear and disappear? What are the rates of change? When in the lifecycle of the program do these changes take place? What else is happening during program implementation, or in the program’s environment, that may affect these changes? Over and above any specific change, what is the collective effect of the changes on expected outcomes?

Methodology

Considering the  ubiquity of power law-like distributions in so many “size x frequency” phenomena, a methodology is needed to determine the specifics of that distribution for the project plan that is being executed. Over and above the capacity to detect change is the question of units of analysis. As an example, it is not hard to imagine that cost overruns would be distributed differently for parts of a project that involved routine well-known processes, compared to activities for which there was little prior experience. Knowing these distributions could provide valuable guidance with respect to risk in a project’s implementation. Such knowledge would be particularly useful for effective planning in cases of scale-up and replication.

It is especially important to include this kind of hard data in an evaluation because estimates in project planning are distorted in powerful ways by political and economic forces, and also, are subject to a host of cognitive biases. The politics, the economics and the biases combine to yield considerable underestimation of what is needed to successfully conclude a project (Flyvbjerg, 2021; Flyvbjerg, Brurzelius, & Rothengatter, 2003).

Themes

Much of the above discussion is about how power law and power law-like distributions of deviations in project schedules is like the general pattern of such distributions that are found across a wide range of physical and social phenomena. Another element of the discussion was an argument that change in a schedule’s parameters can be seen in terms of evolutionary biological and ecological constructs. Table 1 fills  out further.

Table 3: Size and Rate Change Relationship to General Themes in Complexity
Complex behavior	Theme in complexity science
	Pattern	Predictability	How change happens
Stigmergy			X
Magnitude rate of change vertical-align: middle;	X	X	X

Summary

I based this blog post on the premise that evaluators should apply specific constructs from Complexity Science to their work. Taking statistics as an example, it is the difference between “I analyzed the data with statistics” to “I analyzed the effectiveness of my safety program by using logistic regression to measure mean time between accidents”. I am not an advocate of “complexity thinking”. I am an advocate of applying identifiable behaviors of complex systems to meet particular evaluation needs. Taking that stance, I attempted to show how specific constructs from Complexity Science can enrich the knowledge that derives from using program schedules, in the form of Gantt or PERT charts, as evaluation logic models. Two constructs were discussed – 1) stigmergy and 2) magnitude and frequency distributions of deviations from plan. In both cases there are implications for program theory and for methodology.

References

Andriani, P., & McKelvey, B. (2005). Beyond Gaussian averages: redirecting organization science toward extreme events and power laws.

Dept. of Geography University of Kentucky. (2015). The Dubious Power of Power Laws.  Retrieved from https://geography.as.uky.edu/blogs/jdp/dubious-power-power-laws

Flyvbjerg, B. (2021). Top Ten Behavioral Biases in Project Management: An Overview. Project Management Journal, 52(6), 531-546. doi:10.1177/87569728211049046

Flyvbjerg, B., Brurzelius, N., & Rothengatter, W. (2003). Megaprojects and Risk: An Anatomy of Ambition: Cambridge University Press.

Hennigan, L., & Bottorff, C. (2021). PERT Chart Vs. Gantt Chart: Which System Is Best For Project Management? Forbes Addvisor. Retrieved from https://www.forbes.com/advisor/business/pert-chart-vs-gantt-chart/

Klarreich, E. (2018). Scant Evidence of Power Laws Found in Real-World Networks. Quanta.

Morell, J. A. (2010). Evaluation in the Face of Uncertainty: Anticipating Surprise and Responding to the Inevitable. New York: Guilford.

Morell, J. A. (2014). Timelines, Critical Incidents and Systems: A Nice Way to Understand Programs.  Retrieved from https://evaluationuncertainty.com/2014/12/08/timelines-critical-incidents-and-systems-a-nice-way-to-understand-programs/

Morell, J. A. (2018). Linking Management and Evaluation: Project Schedules as Program Models. American Journal of Evaluation, 1 – 18. doi:https://doi.org/10.1177/1098214018784198

Morell, J. A. (Writer). (2020). Using Ecology to Evaluate Policy. In E. a. Planning (Producer): South African Monitoring and Evaluation Association.

Morell, J. A. (2021a). A Complexity-based Metatheory of Action for Transformation to a Green Energy Future. In R. D. Van den Berg, C. Maro, & M.-H. Adrien (Eds.), Transformational Evaluation for the Global Crises of our Times . (pp. 345 – 370). Exeter, UK: IDEAS.

Morell, J. A. (2021b). Evolutionary and Ecological Constructs that may be Useful in Evaluation.  Retrieved from https://evaluationuncertainty.com/2021/02/27/evolutionary_ecological_constructs_useful_evaluation/

Morell, J. A. (2021c). System Perspectives by Donors, Not Why, but How.  Retrieved from https://evaluationuncertainty.com/2021/11/05/system-perspectives-by-donors-not-why-but-how/

Morell, J. A. (Writer) & J. A. Morell (Director). (2021d). Using Evolutionary Biology  and Ecology to Think About Evaluation. In: 4.669… Evsaluation and Planning.

Parunak, H. V. D. (2006). A Survey of Environments and Mechanisms for Human-Human Stigmergy. In D. Weyns, F. a. Michel, & H. V. D. Parunak (Eds.), Proceedings of E4MAS 2005 (Vol. LNAI 3830, Lecture Notes on AI, pp. 163  186): Springer.

Smolyarenko Artico I, Vinciotti V, & Wit EC. (2020). How rare are power-law networks really? Proc. R. Soc. A.

Theraulaz, G., & Bonabeau, E. (1999). A Brief History of Stigmergy. Artificial life, 5, 97 – 116.

Vidal, L.-A. (2009). Thinking project management in the age of complexity : particular implications on project risk management. HAL Open Access. Retrieved from https://tel.archives-ouvertes.fr/tel-00546695/document

[1] Blog: Evaluation Uncertainty: Surprises in Programs ant their Evaluations

YouTube channel .

[2] Both PERT and Gantt are visual representations of the activities and the relationships among activities that comprise a project schedule. Planners choose which one to use, or use both, depending on circumstances (Hennigan & Bottorff, 2021).

[3] Thanks to Apollo Nkwake for helping me work out this example.