Of late there has been a lot of discussion on Evaltalk about logic models. This reminded me of the principles I rely on when I construct models. (For a dated, but still useful slide deck of my logic model workshop, go here.)
What is a logic model good for?
Logic models can be used to explain a program, to guide an evaluation, or to advocate for a program. These should not look the same. Before constructing a model I decide what it is for.
Don’t make things look different if they are not different.
For example, if there are two outcomes, I don’t use different shapes or colors for each unless there is a qualitative difference between them that the reader should be aware of.
Don’t assume that all connections are needed.
Imagine some outcome with three inputs. Are they “and” or “or” relationships? It maters because if I really believed that all three were necessary, I’d also believe that my program is almost guaranteed to fail. If I believed that various combinations would work, I’d design my program for alternate paths to success, and I’d design the evaluation to determine which paths were activated.
Consider multiple models.
I have never understood why people insist on a single model. My bet is that if you have only one model, it’s probably a result of illusory consensus. Why can’t people disagree about how a program works, what outcomes will ensue from action, or how those outcomes affect each other? Accept the different opinions and test them.
Keep level of detail at approximately the same level.
It subverts understanding of a program if a model mixes levels of detail. For example, let’s say I have a program with two goals: 1) kids learn the math that is being taught, and 2) kids come to enjoy math as a subject. If I did that, I’d probably also have a feedback loop between the two outcomes.
I then develop a model in which “kids learn math” is detailed in terms of each math element in the curriculum, while “kids enjoy math” is left at the level of a global measure. How could my audience make sense of this model? If the reason for the model is to explain the program’s goals, why break one goal into its component parts and not the other? How does that accomplish anything but add confusion? If the reason for the model is to evaluate, would could I make sense of relationships between a global outcome and a detailed set of outcomes? If I had good reason to get detailed on only one of the outcomes, I’d make a separate model.
Jointly optimization of clarity and density.
I like to think of model building as an exercise in jointly optimizing two conflicting design constraints. 1) Make the model as dense as possible so that as many relationships as possible can be captured. 2) Make the model as visually understandable as possible.
Evaluation Uncertainty 