This post is to allow participants in the AEA workshop to state their preferences about workshop content, to let others offer an opinion about content for a workshop like this, and for participants to get to know each other.
Please post some information about your professional background.
Slides from a previous version of this workshop can be found at: Workshop slides. Please post your opinion about which sections are most important to you: 1- models, metrics and methodology, 2- graphic design in support of information density, 3- methods of working with stakeholders. Also, are there any particular topics you want covered?
Evaluation Uncertainty 
From Anne Gadomski
I am a pediatrician and health researcher. I am currently in charge of evaluating one of the NIOSH Ag Centers, i.e. the Northeast Center (NEC) as of 9/1/11. I have attached a powerpoint slide of the current NEC logic model to this email for your review and for use in the class if you would think it would be helpful. My primary interest is in Part3, then Part 2, but after seeing my logic model, you may think I need Part 1! Thank you for canvassing the group regarding our preferences in this course. I look forward to the meeting on November 2.
I am an analyst at an evaluation agency within the Catalan government. We usually describe program theory in a narrative way, usually based on academic literature and our own intuition after a few interviews.
I guess out situation is not very original: We find it quite difficult to have program officers to describe program expectations and theory in a meaningful way (even though several programs we’ve been working on make sense, at least theoretically). Too open questions (on program objectives, mission…) lead to vague statements, whereas too detailed questions set the track for the answers. What we usually end up doing is show them what we have written, and then they usually agree and seem happy about having an explicit theory. But we are not sure this is the best way to proceed.
Order of interest: part 3, part 1, part 2
Thanks for asking!
As we will see in the workshop, sometimes it works to “show them what we have written”. The question is what to do then to get them to engage. We will talk about that in the workshop.
Hello,
My name is Gregg Friedman and I am a Monitoring & Evaluation Advisor working in the International Response and Programs Department of the American Red Cross. I focus primarily on our programs in Haiti, and I have a particular expertise in survey design and data analysis. My primary responsibility is to provide our field teams with technical assistance in Monitoring and Evaluation, with most of time spent on project/program evaluations.
A particular challenge we face is adapting and using our logframes across the entire project cycle. Often much time and thought is put into their development during the initial project development phase, and then they are often set aside until final evaluation.
My preferences for the workshop are: 3, 1, 2
Looking forward to it,
Gregg
Logic models have different uses at different times in the life cycle. I’ll have at least one slide to show and quite a bit to say about it.
My name is Jodi Jaffray and I am a Research Associate with the Ontario Centre of Excellence for Child and Youth Mental Health in Canada. My role is to provide consultative support to staff in child and youth mental health programs across Ontario who are undertaking a program evaluation, often with little or no experience in the area of evaluation.
My preferences for the workshop are 3, 1, 2.
Looking forward to meeting everyone and learning more about logic models and their use beyond the typical initial program/evaluation planning stage.
Jodi
We should give some thought as to why they are undertaking a program evaluation, what they need out of it, and their capacity (time, money, human capital) to do evaluation. Once that is known the scope of the evaluation can be determined, and the nature of a useful logic model derives from that.
Hi, I’m Stuart Henderson. I’m the associate director of evaluation for the Clinical and Translational Science Center at the University of California, Davis. We use the standard input-activity-output-outcome/impact logic model for a variety of traning programs we evaluate. I’m hoping to hear about alternatives to this traditional model as well as new ways to present logic models. I think it is important to present logic models in a way that the intended audience’s eyes don’t glass over. My preference for topics are 2, 1, and 3.
There is nothing wrong with the trusty old “standard input-activity-output-outcome/impact logic” format. But as we shall see, one can have many different forms of the same model for the same evaluation, each useful for a different purpose.
Hello
My name is Nooshin K Rotondi. I am an epidemiologist in the Health Systems and Health Equity Research group at CAMH (Toronto, ON, Canada). My research thus far has been focused on the health issues of marginalized communities, using novel research methods for sampling. I am now working on a system-level project that includes a major evaluation component, however I am new to the field of evaluation research. While I will not be conducting the evaluation, I am in the process of moving towards research related to health policy and program planning. I view this workshop and the entire AEA conference as a great learning opportunity and an important guide to my current and future work. So I look forward to learning from all of you who have experience conducting evaluation projects and working with logic models. My preference for topics are 1,3,2.
See you all next week!
~Nooshin
Hi All,
My name is Erin Watson, I’m a graduate student (defending my dissertation in a few months – yikes!) in community psychology at Michigan State University. My primary interest is in the development and evaluation of community-based interventions aimed at shifting systems and environmental outcomes. A challenge in this work is to identify, make sense of, and take advantage of emerging patterns and opportunities over time – I am interested in how evaluators can re-imagine logic models (often seen as static documents) within these emergent situations. I am excited to learn from our group and to engage in some great discussions. My preference for topics is 3, 1, 2.
~Erin
There are a few issues at play here. Of course logic models may change over the course of an evaluation. The question is why they are changing. One possibility is that the program does not change but the evaluation shows that the program theory on which the program was built was wrong. Or, maybe the program theory has changed as the program developed over time. Or both may be true. Or there may be some strong interaction between the evaluation driven change and the program driven change. And so on and so forth, the possibilities are numerous and delicious.
As for taking advantage of emerging patterns over time, that is one of my favorite topics because it touches on the ability to increase the lead time between when a change in an evaluation design must be operational, and when that need is first detected. I write about this a lot in my book because lead time speaks to the ability to change an evaluation to deal with unexpected program consequences. We won’t talk about this in the workshop because it will lead the conversation too far afield. But anyone who is interested will not have much trouble getting me to expound on the subject outside the workshop.
As for models being static after they are first developed, this is because the vast majority of the use for a logic model is to develop the evaluation prior to starting work. But there are many other uses for models along the entire evaluation life cycle. We will talk quite a bit about this when we meet.
As for finishing your dissertation, don’t tell your adviser I said this, but “better done than good.” Get the degree. Then go on and do better work.
Hello! I’m Jon Vencil, an evaluation consultant from Kema, Inc. We are a global firm that specializes in all types of evaluation related to the energy industry. Because we cover a wide variety of topics but work with technically oriented clients we try to use logic models for a variety of purposes – the main one being communication of course. While I’m looking forward to the entire workshop, I’m really interested to learn how to go about choosing the appropriate model, the connections between models and data, etc. and best ways to use them throughout the project cycle. Thanks for taking the time to put this together!
“Technically oriented clients”. I assume this means a group of people on good terms with PERT charts and other planning tools. Those models and evaluation logic models are not the same, but there are some important relationships among them. This is not a topic I usually spend much time on in this workshop, but remind me and I will at least touch on it. If it is important to you we can spend some off-line time talking about it.
Hello, I’m Angela Alexander. I’m the evaluation manager for the Clinical & Translational Research Institute at the University of California, San Diego. We use a combination of the Logic Model and the Tyler Method to capture the goals, objectives, activities, outputs and outcome/impact for a variety of programs. I look forward to learning more about the traditional logic model and how best to present and describe them to non-evaluators. My topic preferences are 3, 1, 2.
Presenting models to non-evaluators is easy. People are smart and will understand what you tell them. What’s not so easy is making sure the presentation does what needs to be done, both for the evaluator and the audience.
Hello to each of you. My name is Mary Halbleib and I am a program evaluator at Oregon State University. In my work at the Integrated Plant Protection Center my main effort involves working with a group of integrated pest management experts to deliver educational programs and support resources. The target audneinces are farm advisers and farmers, the desired outcome is less pesticides use or the selection of less toxic options when possible. In my work understanding how and why one changes behavior is important as well as the impacts of the change. I look forward to this session. Thank you Jonny for getting the sharing and thinking started before the course. Topics are all good and almost a three-way tie: 3, 2, 1
My first inclination is dip into the research literature in innovation adoption and social psychology to see what is known about effecting this kind of change. (Quite a lot actually.) Then, get program planners to base their interventions on that knowledge. That done you will end up with a logic model that can provide strong guidance for all.
Thank you for a thoughtful and helpful response. I will put this on the list of things to do when I return — suspect more will emerge from the workshop as well. And this is a GOOD thing.
Hello, my name is Sharon Turlington and I am new to evaluation (from a more scientific background). I work at a small evaluation and development company in Buffalo, NY that works primarily with school districts, CBOs, and foundations serving students in the community. My experience and exposure to logic models is limited and I am looking forward to learning a great deal in this course.
Hi all,
I am excited to join such a diverse group in discussing appropriate, innovative, and compelling logic models. I come from a science museum background, having done evaluation on exhibits and programs for the past four years. I am now part of the group working to generate grant proposals, and getting the team to think through the goals and project plan enough ahead of time is often difficult. I expect this group and class will be able to help me in a variety of ways as I enter this new stage!
As for the sections up for discussion, 1, 3, 2.
I’m sure we can help. In my experience it is easy to get people to articulate program goals. What’s hard is to get them to come up with goals that are realistic and relevant. We will have much to say on that topic. But let’s not assume that the only outcomes worth measuring are ones the model developers come up with. We will have much to say about that as well.
Hello all,
I look forward to meeting everyone this week. I am a chemist by training and typically do evaluation work in the context of National Science Foundation projects to promote changes in the teaching of undergraduate chemistry. I have used logic models primarily as a way to be sure that I have understood my clients’ descriptions of their intentions and to create a measurement plan. I would like to be able to use them in a more robust way. However, I am also very interested in ways to present logic models more effectively. The ordering I give to the sections is 2, 1, 3.
Hello,
My name is Rick Groves and I am consultant at a small management consulting firm in Chicago that works with any type of organization that seeks to create social impact. My background is in analysis, but in my current work is focused largely on helping organizations develop internal capacity to conduct ongoing M&E. Our methodology includes developing what one might call a Theory of Change or Logic Model, but does not necessarily leverage best practices from the broader evaluation field. We find that creating a model for an organization to use for many purposes is a much bigger challenge that one which simply facilitates a specific line of research or evaluation. I’m looking to develop a much richer understanding of the various types of models, what content they include and how & when they are used.
We will talk about this. LMs can be used for planning, evaluation, and advocacy. All are legitimate but one can get into trouble by confounding them. The workshop will focus on evaluation, but we will touch on the others as well. If I don’t touch heavily enough, remind me to push harder.
Hello,
I look forward to this workshop. I, Laran Despain, have graduate training in psychology and have moved into evaluation as a career since begiing work in the field in 2007. Althought I’ve learned a lto by doing and taken the one evaluation course offerred at my university, I still feel like I’m catching up when I try to explain logic models to other people. I’m hoping to develop greater mastery of the topic so I can help others learn.
Hi Everyone – I am a Psychologist by training with an extensive background in assessment and evaluation. My official title is Assistant Dean for Educational Development (think faculty development) and Evaluation (think course, faculty, student, program). I also serve as the Director of Evaluation for our CTSA grant. We use Logic Models extensively but can always learn more…especially about how to make them more than the sum of their parts 🙂 Look forward to working with all of you!