A friend of mind and I were discussing the nature of AEA. I have come to quite a few conclusions about this, but my current thinking is more in the way of questions than answers. As I see it, what’s needed is an exploration of four questions.
- Where does the evaluation that we do fit within all the evaluation that is done?
- How does what we call “evaluation theory” fit within the greater domain of social science?
- What is the nature of AEA’s professional support network?
- What are the implications of AEA’s social values for the field of evaluation?
Where does the evaluation that we do fit within all the evaluation that is done?
Every once in a while, in the popular press, I read about the evaluation of a social program. As far as I can tell, the people doing these evaluations have no involvement in AEA. I don’t see this as problematic. It just means that AEA attracts a breed of evaluator that does not work on the kinds of programs that the popular press likes to write about. But it does mean that a lot of evaluation takes place that is outside of the ambit of AEA’s member’s activity. More important, it is outside the ambit of anything AEA claims for evaluation. So, considering the population of evaluations that are done, what subset is covered by AEA members? Why that particular subset?
How does what we call “evaluation theory” fit within the greater domain of social science?
Over the decades there has been plenty of writing about how evaluation differs from other types of social science with respect to how evaluation should be designed and executed, and how the products of evaluation should be used. It is largely this body of work that sets evaluation apart from other social science. Still, evaluation theory does not exist in isolation. Some of it is a novel extension of what went before. Some of it is a new expression of what already exists, that new expression needed to orient evaluation in unique ways.
What is the nature of AEA’s professional support network?
We know that one function of AEA (as with all professional organizations) is to embed its members in a professional support network. But what is the structure of that network? How does it function and influence the work lives of its members? How does it further their professional interests? How does it develop and evolve? What sub-networks does it contain?
What are the implications of AEA’s social values for the field of evaluation?
AEA as an organization pursues a particular set of values with respect to the dissemination of data, the use of data, the value of data, and the social/organizational processes that should drive the development and execution of evaluation. There are consequences to pursuing these values for the kinds of programs that are evaluated, the nature of the information that is produced, and the range of stakeholders who might look to evaluation as a source of knowledge.