Much of my professional work deals with coordination among trading partners in business settings. Thanks to having my artificial hip overhauled, I became motivated to apply my interest in coordination to hospital settings. After all, one needs something to occupy the mind when lying in a bed and restricted at home. Full article
Systems as Program Theory and as Methodology: A Hands on Approach over the Evaluation Life Cycle: Workshop at the American Evaluation Association Summer Institute
Description : This workshop will provide an opportunity to learn how to use a system approach when designing and conducting evaluation. The presentation will be practical. It is intended to give participants a hands-on ability to make pragmatic choices about developing and doing evaluation. Topics covered will be: 1) What do systems “look like” in terms of form and structure? 2) How do systems behave? 3) How can systems be used to develop program theory, as a methodology, and as a framework for data interpretation? 4) How should a systems approach be used along different parts of an evaluation life cycle – from initial design to reporting? The workshop will be built around real evaluation cases. Participants will be expected to work in groups to apply the material that will be presented.
System design: Requirements, complexity, and cost
Systems that meet relatively small numbers of requirements will usually give people most of what they need. (If not most of what they want.) But people, many of whom should know better, insist on having it all, and thus doom themselves to building systems that fail. Either the systems are built and do not perform to expectations, or they never get finished. Why is this so, and how can people be brought to appreciate the “requirements trap”? The problem exists because: Full article
How to Evaluate a Conference
This document provides a map of how I approach the task of evaluating conferences. It is organized around the three main stakeholders to a conference. Full article
- attendees
- exhibitors
- sponsors
What complexity theory do evaluators need to know?
My last blog post dealt with why evaluators should focus on complex behavior as opposed to complex systems. Bob Williams made a comment about how the post made a lot of sense, but that it conveyed the impression that evaluators do not have to worry about complexity theory. Evaluators do need to be concerned with theory, and Bob’s post got me to begin to crystallize some notions that have been marinating in the back of my brain for some time.
My Starting Point
Recently I have been pounding on the idea that a switch from complex systems to the behavior of complex systems would do a lot to further the abilities of evaluators to make practical, operational decisions about program theory, metrics, and methodology. And after all, that’s what it’s all about. We (I at least) get hired when someone says to me: Continue reading “What complexity theory do evaluators need to know?”
Things to think about when observing programs from a systems perspective
A friend of mine (Donna Podems) is heading up a project that involves providing a structure for a group of on-the-ground observers so they can apply a systems perspective to understanding what programs are doing and what they are accomplishing. She asked me for a brain dump, which I happily provided. What follows is by no means a systematic approach to looking at programs in terms of systems. It’s just a laundry list of ideas that popped into my head and flowed through my fingers. Below is a somewhat cleaned up version of what sent her.
Hi Donna,
What follows is not a list of independent items. In fact I guarantee there are lots of connections. For instance, “redundancy” and “multiple paths” are not the same thing, but they are related. But time is tight, and I have a Greek meatball recipe to shop for, so let’s assume they are independent. Continue reading “Things to think about when observing programs from a systems perspective”
Case Study Example for Workshop 18: Systems as Program Theory and as Methodology
Case Study Example for Workshop 18: Systems as Program Theory and as Methodology: A Hands on Approach over the Evaluation Life Cycle
This case was developed for a workshop at the American Evaluation Association’s 2015 Summer Evaluation Institute.
Construction of the Case
This is the example we will use throughout this workshop to illustrate how knowledge of system behavior can be applied in evaluation. The example is hypothetical. I made it up to resemble a plausible evaluation scenario that we may face, but which is elaborated to make sure it contains all the elements needed to explain the topics in the workshop. I am sure that none of us (me included) have ever been involved in an evaluation that is as far reaching and in-depth as the example here. But I am sure that all of us have been involved in evaluations that are similar to parts of the example, and, if you are like me, I bet you have dreamed of being involved in an evaluation of the size and scope of the example.
There are three initiatives. One aimed at adults. One aimed at mothers and young children. One aimed at teens. Each initiative has several individual programs that share some common outcomes, and which also have some unique outcomes.
All three initiatives are deliberately implemented Continue reading “Case Study Example for Workshop 18: Systems as Program Theory and as Methodology”
Matching management systems to system behavior
Matt Keene and I have been having a back and forth on the topic of how management systems are built with respect to the way in which systems actually behave. Below is a record of our conversation to date.
Jonny’s thoughts
Desirability of Narrow Rigid Planning
First, I don’t place any negative value judgments on the need for rigid planning. There are good reasons for it given the nature of the world. Second, I don’t think that environmental management is different from any other kind of management in this regard. Everyone is in the same boat. Third, the problem is not management systems. To blame the systems is like shooting the messenger. Management systems match the world in which management takes place, and that is what is giving us trouble. Continue reading “Matching management systems to system behavior”
Miscellaneous thoughts on complexity in Evaluation
My friend and colleague and partner in crime Sanjeev Sridharan sent a query to a few people asking for reaction to various ideas he has been pondering. I thought it might be a good idea to post my response here. Continue reading “Miscellaneous thoughts on complexity in Evaluation”
