AEA’s Potential to Serve the Public Good

What you see here is a brief overview of my recent thoughts about the field of evaluation and the future of AEA. See the PDF for a more fleshed out explanation. Fair warning, it runs to 2,500 words. I am not advocating change, I’m only advocating that we recognize where we are and where we are going, and that we contemplate the consequences. AEA_Evaluation_Evolutionary_Path_Long

The challenge of contending points of view in a representative democracy goes back to the founding of the Republic. James Madison saw the polity as a collection of “factions” and believed that a stable democracy required a diversity of contending factions.

One way to think about the purpose of evaluation is to see it as an honest broker to which those factions can turn. This is not to say that any given evaluation can be, or should be, “objective”. Supporters of one or another point of view will inevitably find the results of any single evaluation wanting.

The question is whether over time, and across evaluations, it matters whether we are seen as taking sides. That perception has consequences for evaluation use, and I can see how outsiders would get the impression that we do take sides. If they do, they will shop elsewhere for evaluation information.

I’m not sure if we can or should change. But I do not think we should continue on our path without awareness of where that path is taking us.

 

 

Evaluation as Social Technology

Evaluation as Social Technology
Chapter 5 from: Program evaluation In social research Full chapter

• Introduction
• Why Is Social Research Not Relevant
• A Framework for Solutions
• Differences between Science and Technology
• Advantages of Evaluation as Social Technology
• Interrelationships between Science and Technology

So far I have argued that the value of outcome evaluation can be increased if type, validity, and usefulness are considered separately as distinct aspects of any given evaluation plan. Each of these three elements contains implications for the other two, but those implications will not be clear unless each component part is analyzed separately. There is another aspect of the problem which must also be considered, namely, the basic philosophical model of knowledge seeking upon which outcome evaluation is based. This is a consideration which cuts across elements of type, validity and usefulness, and deals with basic approaches to social research. How are questions formulated? How are variables chosen? What decision rules are used to weigh evidence, draw conclusions, and make recommendations? The answers to these questions reflect a philosophical model of research, and the model chosen can have far-reaching effects on the ultimate value of any research project. There are three main aspects to the argument about to be developed. First, there are crucial differences between scientific and technological models of knowledge development. Second, these differences have profound implications for the practical value of research. Third, evaluation is far more of a technological than a scientific pursuit.

Part 1 of a 3 Part Series on how to Make AEA, and Evaluation, Relevant in the Future: What is Diversity?

Common Introduction to all Three Parts

I have been thinking about what will happen to AEA, and to evaluation, in the future. I can conjure scenarios where AEA and evaluation thrive, and I can imagine scenarios where they whither. What I cannot envision is a future in which AEA and evaluation, as we know them now, stay the same. What I want to do is to start a conversation about preparing for the future. AEA is already active in efforts to envision its future: What will AEA be in 2020? My intent is to inject another perspective into that discussion.

What I’m about to say draws on some thinking I have been doing on two subjects – 1) AEA’s development in terms of evolutionary biology (Ideological Diversity in Evaluation. We Don’t Have it, and We Do Need It, and 2) Using an evolutionary biology view to connect the intellectual development of evaluation and the development of the evaluation community); and the nature of diversity in complex systems. (If you have not read Scott Page’s Diversity and Complexity, I recommend it.).

Part 1: What do I mean by diversity?

There are two reasons for AEA to build diversity. One is to pursue the social good. The other is to maximize the likelihood that we can thrive as circumstances change. Diversity Continue reading “Part 1 of a 3 Part Series on how to Make AEA, and Evaluation, Relevant in the Future: What is Diversity?”

Part 2 of a 3 Part Series on how to Make AEA, and Evaluation, Relevant in the Future: AEA as an Evolving Organism

Common Introduction to all Three Parts

I have been thinking about what will happen to AEA, and to evaluation, in the future. I can conjure scenarios where AEA and evaluation thrive, and I can imagine scenarios where they whither. What I cannot envision is a future in which AEA and evaluation, as we know them now, stay the same. What I want to do is to start a conversation about preparing for the future. AEA is already active in efforts to envision its future: What will AEA be in 2020? My intent is to inject another perspective into that discussion.

What I’m about to say draws on some thinking I have been doing on two subjects – 1) AEA’s development in terms of evolutionary biology (Ideological Diversity in Evaluation. We Don’t Have it, and We Do Need It, and 2) Using an evolutionary biology view to connect the intellectual development of evaluation and the development of the evaluation community); and the nature of diversity in complex systems. (If you have not read Scott Page’s Diversity and Complexity, I recommend it.).

Part 1: What do I mean by diversity?

There are two reasons for AEA to build diversity. One is to pursue the social good. The Continue reading “Part 2 of a 3 Part Series on how to Make AEA, and Evaluation, Relevant in the Future: AEA as an Evolving Organism”

Part 3 of a 3 Part Series on how to Make AEA, and Evaluation, Relevant in the Future: Evolution, Diversity and Change from the Middle

Common Introduction to all Three Parts

I have been thinking about what will happen to AEA, and to evaluation, in the future. I can conjure scenarios where AEA and evaluation thrive, and I can imagine scenarios where they whither. What I cannot envision is a future in which AEA and evaluation, as we know them now, stay the same. What I want to do is to start a conversation about preparing for the future. AEA is already active in efforts to envision its future: What will AEA be in 2020? My intent is to inject another perspective into that discussion.

What I’m about to say draws on some thinking I have been doing on two subjects – 1) AEA’s development in terms of evolutionary biology (Ideological Diversity in Evaluation. We Don’t Have it, and We Do Need It, and 2) Using an evolutionary biology view to connect the intellectual development of evaluation and the development of the evaluation community); and the nature of diversity in complex systems. (If you have not read Scott Page’s Diversity and Complexity, I recommend it.)

Part 1: What do I mean by diversity?

Continue reading “Part 3 of a 3 Part Series on how to Make AEA, and Evaluation, Relevant in the Future: Evolution, Diversity and Change from the Middle”