Evaluation is well stocked with knowledge about how to evaluate programs in terms of systems. Our stock is much less when it comes to evaluating the logic of the systems themselves. But separating the two and then bringing them back together can advance our understanding of both programs and systems. This assertion is illustrated with four examples: 1) causal chains, 2) stocks and flows, 3) network development and structure, and 4) complex systems with an attractor/equilibrium focus.
Evaluation Influence During Program Design and Funding: An Ecosystem / Environment Perspective
Lately I have been thinking about the Evaluators’ Eternal Problem, namely, how to influence decisions that are baked in during the funding and planning stages of programming. Woe unto us, we evaluators have precious little influence over what happens when those decisions get made. In this post I present an ecosystem/environment approach for addressing our EEP.
Evaluating Systems as Systems and Using that Knowledge to Inform Program Evaluation
There is much talk about how our programs are (or should be) thought of in terms of systems, and quite a bit of progress is being made toward that end. But there is a difference between: • evaluating programs in terms of systems, and • evaluating the systems themselves, i.e. abstracting the system structure from the details of a program. The purpose of this document is to take a stab at the latter. Why bother? For two reasons. First, Understanding the system that underlies a program will help us understand the program. Second, similar systems structures may indicate similarities across seemingly disparate programs.
Some Notions About the Relationship Between Systems Thinking and Complexity
Some people asked me what I thought about the relationship between systems thinking and complexity. My answer deals with complex system behaviors that explain the rules that system-thinking uses to understand how systems work.
Why Do Hospitals Coordinate Activities As They Do? Or: What I Learned From My Hip Surgery
Thanks to having my artificial hip overhauled, I became motivated to apply my interest in coordination to hospital settings.
Systems as Program Theory and as Methodology: A Hands on Approach over the Evaluation Life Cycle: Workshop at the American Evaluation Association Summer Institute
1) What do systems “look like” in terms of form and structure? 2) How do systems behave? 3) How can systems be used to develop program theory, as a methodology, and as a framework for data interpretation? 4) How should a systems approach be used along different parts of an evaluation life cycle – from initial design to reporting?
System design: Requirements, complexity, and cost
Systems that meet relatively small numbers of requirements will usually give people most of what they need. (If not most of what they want.) But people, many of whom should know better, insist on having it all, and thus doom themselves to building systems that fail.
How to Evaluate a Conference
This document provides a map of how I approach the task of evaluating conferences. It is organized around the three main stakeholders to a conference.
What complexity theory do evaluators need to know?
My last blog post dealt with why evaluators should focus on complex behavior as opposed to complex systems. Bob Williams made a comment about how the post made a lot of sense, but that it conveyed the impression that evaluators do not have to worry about complexity theory. Evaluators do need to be concerned with … Continue reading What complexity theory do evaluators need to know?
Things to think about when observing programs from a systems perspective
A friend of mine (Donna Podems) is heading up a project that involves providing a structure for a group of on-the-ground observers so they can apply a systems perspective to understanding what programs are doing and what they are accomplishing. She asked me for a brain dump, which I happily provided. What follows is by … Continue reading Things to think about when observing programs from a systems perspective
Evaluation Uncertainty 