jamorell Program Theory and Program Logic

Timelines, Critical Incidents and Systems: A Nice Way to Understand Programs

I have been involved in evaluating distracted driving programs for transportation workers. While working on the evaluations I developed an interesting way to understand how programs are acting and what they are doing. The method is based on a few principles, one set focusing on the nature of timelines and schedules, and the other on data collection.

Timelines and schedules
Timelines and schedules matter. These documents are constructed to meet a few objectives. They have to:

  • Provide a reasonable plan whose tasks can be executed.
  • Represent a reasonable correspondence between budget and work.
  • Satisfy customers desires for when the work will be completed, and for how much.

In a sense there is a conflict between the first two objectives and the third, resulting in overly optimistic assessments of budgets and timelines. That’s the direction of the bias in our estimates. Continue reading “Timelines, Critical Incidents and Systems: A Nice Way to Understand Programs” →

jamorell Practical Ways to use Complexity to do Evaluation, Program Theory and Program Logic

Power Law Versus Symmetrical Distributions — Implications for Policy Recommendations

I could use some help muddling through a question that has been rattling around in the inner recesses of my brain. I have no data to support this, but I’d bet that when people look at the results of an evaluation they think in terms of normal, or at least symmetrical distributions. Even if the evaluation were  qualitative, the thinking is along the lines of: “The program seems to have done X, and Y and Z. If I do it again it will do something close to X and Y and Z. If I get lucky it will do a lot better. If I get unlucky it will do a lot worse, but X and Y and  Z seems like a good bet”. Continue reading “Power Law Versus Symmetrical Distributions — Implications for Policy Recommendations” →

jamorell Program Theory and Program Logic

AEA 2012 Workshop on logic models – Pre-conference email trail with participants.

Prior to presenting my workshop this year I conducted an email conversation with participates. The technology was crude. I sent out email and asked people to hit “reply all”. Below are the (mostly unedited) comments. My responses to the participates are in CAPS. The workshop slides can be downloaded from my digital scrapbook, aka http://www.jamorell.com/. Continue reading “AEA 2012 Workshop on logic models – Pre-conference email trail with participants.” →

jamorell Program Theory and Program Logic

Preparation for AEA workshop: Logic Models – Beyond the Traditional View: Metrics, Methods, Expected and Unexpected Change

This post is to allow participants in the AEA workshop to state their preferences about workshop content, to let others offer an opinion about content for a workshop like this, and for participants to get to know each other.

Please post some information about your professional background.

Slides from a previous version of this workshop can be found at: Workshop slides. Please post your opinion about which sections are most important to you:  1- models, metrics and methodology, 2- graphic design in support of information density, 3- methods of working with stakeholders. Also, are there any particular topics you want covered?

jamorell Program Theory and Program Logic

Evaluating the relationship between development and democracy. Comments sought on draft of a presentation

I’m doing a presentation at the U.S. Department of State’s Fourth Annual Conference on Program Evaluation: “Diplomacy, Development, and Defense – Working Together to Achieve Foreign Policy Goals” June 7-8, 2011

I’m looking for comment and critique. Draft of presentation slides Continue reading “Evaluating the relationship between development and democracy. Comments sought on draft of a presentation” →

jamorell Evaluation as a Discipline and a Profession, Program Theory and Program Logic

AEA Session 2011 Development of a session on how ideology affects evaluation design

I’m sending this email to people who either expressed interest in participating the session or in helping to design it. Welcome fellow partners in crime and trouble makers! The purpose of this message is to get some discussion going to help set up the session. Here is what I have in mind. We can change it and shape it as the discussion evolves.

Theme of Panel
The basic them is how political ideology affects metrics and methodology. To give a simple example of both.

Continue reading “AEA Session 2011 Development of a session on how ideology affects evaluation design” →