Oregon Program Evaluators Network: Context in Evaluation
The Oregon Program Evaluators Network (OPEN) held its Annual Conference last week in Portland. OPEN is a regional affiliate of the American Evaluation Association, with members from government agencies, nonprofits, universities, and private consulting firms. Their annual meeting primarily attracts evaluators from western Oregon and southwestern Washington (Vancouver, WA down through Eugene, OR) but there was at least one international participant and several of us from Seattle. This was a very interesting meeting and I’ll provide my subjective take-aways from it in this post and the next two posts.
The opening keynote speaker was Dr. Debra Rog, 2009 president of the American Evaluation Association, and her talk was titled, “When Background Becomes Foreground: The Importance of Context in Evaluation.” She mentioned ongoing discussions in the evaluation field about whether randomized studies can actually be considered a “gold standard”–they’re great when you can do them, but they’re not always appropriate (they may not be practical or ethical). She spoke of realist evaluation, pointing out that programs are embedded in layers of social reality. For example, power and privilege can influence programs (and the evaluation of those programs) in fundamental ways. Often there are only two “degrees of separation” between the people in power and the people who are providing data–this can lead to a lack of openness and honesty. Context can be difficult to identify and this speaks to the importance of including multiple voices and views. She provided a great insight from her own experience about sharing evaluation results with stakeholders: decision makers are busy and don’t want nuances, they want the bottom line.
The afternoon featured six workshops running in two sets of three (here’s a copy of the agenda). I attended these sessions:
- “What Program Evaluators Can Learn from How We Performance Auditors Do Our Work” and
- “Groovy Graphics: Visual Displays of Quantitative Information”
Handouts and powerpoints from the afternoon workshops will be made available at the OPEN web site.