I just got back from the IDEAS global assembly, which carried the theme: Evaluation for Transformative Change: Bringing experiences of the Global South to the Global North. The trip prompted me to think about how complexity can be applied to evaluating green energy transformation efforts. I have a longish document (~2000 words) that goes into detail, but here is my quick overview.
Because transformation is a complex process, any theory of change used to understand or measure it must be steeped in the principles of complexity.
The focus must be on the behavior of complex systems, not on “complex systems”. (Complex systems or complex behavior?)
In colloquial terms, a transformation to reliance on green energy can be thought of as a “new normal”. In complexity terms, “new normal” connotes an “attractor”, i.e. an equilibrium condition where perturbations settle back to the equilibrium. (Why might it be useful to think of programs and their outcomes in terms of attractors?)
A definition of a transformation to green energy must specify four measurable elements: 1) geographical boundaries, 2) level of energy use, 3) time frame, and 4) level of precision. For instance: “We know that transformation has happened if in place X, 80% of energy use comes from green sources, and has remained at about that level for five years.”
Whether or not that definition is a good one is an empirical question for evaluators to address. What matters is whether the evaluation can provide guidance as to how to improve efforts at transformation.
Knowing if a condition obtains is different from knowing why a condition obtains. To address the “why”, evaluation must produce a program theory that recognizes three complexity behaviors – attractors, sensitive dependence, and emergence.
Because of sensitive dependence, unambiguous relationships among variables may not continue over time or across contexts. Because of emergence, transformation does not come about as a result of a fixed set of interactions among well-defined elements. The result of sensitive dependence and emergence may produce outcomes that exist within identifiable boundaries, i.e. within an attractor space. If they do, that is akin to “predicting an outcome”. If they do not, that is akin to showing that a program theory is wrong.
Models with many elements and connections cannot be used for prediction, or even, for understanding transformation as a holistic construct. Small parts of a large model, however, can be useful for designing research and for understanding the transformation process.
Six tactics can be used for evaluating progress toward transformation: 1) develop a TOC that recognizes complex behavior, 2) measure each individual factor in the model, 3) consider how much change took place in each element of the model, 4) focus on parts of the model, but not the model as a whole, 5) use computer-based modeling, 6) employ a multiple-comparative case study design.
As all the analysis takes place, interpret the data with respect to the limitations of models, and the implications of emergence, sensitive dependence, and attractor behavior.