I’m sending this email to people who either expressed interest in participating the session or in helping to design it. Welcome fellow partners in crime and trouble makers! The purpose of this message is to get some discussion going to help set up the session. Here is what I have in mind. We can change it and shape it as the discussion evolves.
Theme of Panel
The basic them is how political ideology affects metrics and methodology. To give a simple example of both.
Metrics: Imagine a program that taught literacy to immigrants. “First order” impacts are pretty straightforward, e.g. ability to read, quality of life, employability, and so on. But what about the “second order” impacts, i.e. things touching on the overall social condition – impact on community and economy, etc. I’d bet dollars to donuts that people with pro and anti-immigration feelings would come up with different measures. For instance what about a measure of whether wages are driven down, or of local disruption to social services in communities? It is certainly reasonable to posit these changes, but I doubt that the pro-immigration advocates would not do it.
Methodology: Imagine a needle exchange program evaluation being designed by two different groups: 1) public health advocates, and 2) people who had very strong moral objections to doing anything to help drug addicts take drugs. People in the latter group might reason as follows: “I am not closed minded, but I really think this is a bad idea. So I am going to design the most rigorous methodology possible – random assignment, no treatment control, alternate treatment (no needle exchange) group, and a long term follow-up. Why do I insist on this? Because unless the conclusions are bullet proof, I ain’t gonna change my mind that needle exchange programs are a bad idea.” I can easily imagine how the pro-needle exchange folk would be satisfied with a less rigorous methodology.
Use of a common example
My thought is to invent a program as an example we can all work from. It can be based on something real, but I have found that in order to have an example that illustrates all the points one wants to illustrate, some synthetic elements are necessary. The example has to be something on the large and complicated side, but well within the scope of the kinds of programs that AEA members might actually get a contract to evaluate. I want to make sure the example resonates with people’s experience.
Panel development process
If you are up for it, I’d like to design this panel in a two-tier social networking framework.
Tier 1: Us, i.e. the people with the responsibility to design and submit the panel to AEA. Any advice we get from others is just that, advice. We can use it or not as our wisdom dictates.
Tier 2: The rest of the world. We exchange our ideas in the “fishbowl” of my blog site. Content, design, assignments, etc. In other words, I’d put this email message up and instead of our hitting “reply all” we would post to the blog. This would have two advantages. First, other people might have worthwhile things for us to consider. Second, it would generate publicity for the panel. (And frankly, I would not mind increasing hits on my blog, either.)
What I’m proposing is an experiment, and to be honest, I’m not expecting too much in the way of public comment. But I think it’s worth a try if you do. Let’s do the first round of responses via email to get a sense of whether you are willing to work in the fishbowl. If you are, I’ll transition the material.
What do we want out of this?
Aside from the fact that doing this panel would be a lot of fun, I assume that we have our reasons for wanting to be involved. It might help shape our discussions if we knew the reasons. If you don’t have any that’s fine, but if you do, let’s hear them. For me, there are two.
1- Unintended consequences of program behavior: You all know that this is my favorite subject. Once I finished my book and had a chance to think about the intellectual frontiers I had not explored, I realized that ideology can be an important reason for unexpected behavior. Either there are variables people did not think to include, or the methodology has some blind spots. I want help exploring this frontier.
2- Diversity: AEA is big on gender, ethnic, and racial diversity. I am in favor of this, but I think the field would be a lot better off if there were more emphasis on ideological and methodological diversity as well. How much poorer would economics and political science be if there were no research and theory based on both Marxian and free market paradigms? Quite a lot poorer, I think. I have a strong feeling that the politics of AEA members clusters in the medium-rare range.( I can’t prove it, but I bet I’m right.) I’m not arguing that we go out and recruit other members, but I do think that AEA would be a lot better off if we had a collective appreciation of the implications of what we don’t have.
There are three questions we need to resolve.
1- What TIG should we submit this to? I’m thinking maybe “Evaluation Theory”, but I really don’t have any strong feelings one way or the other.
2- Deadlines. I think we should set ourselves the goal of being able to submit the panel by the end of February.
3- What format should we use? We could do a panel, but there are other possibilities. For instance maybe a think tank, which gives us less time to present, but the audience a lot more opportunity to participate.