Evaluation in Today’s World. Veronica G. Thomas
“oppressive” ways. Power in evaluation is more distributed toward the evaluator since it is, in fact, the evaluator who is studying others and generating knowledge (and not vice versa). While evaluators do not generally own the knowledge generated from their evaluations, it is still the case that their perspectives and interpretations are often privileged over those being studied in the evaluation context.
Interplay of Politics and Ethics
It is also worth distinguishing ethical considerations from political issues, although they are oftentimes closely intertwined. Ethical considerations relate to issues of right and wrong, good and bad, whereas the central focus of political issues relates to power and control. Political issues can undermine the integrity of an evaluation and certainly have ethical ramifications by silencing voices and perspectives of the less powerful and rendering these individuals invisible. For example, politics is likely operating when an evaluator is only allowed to evaluate what project administrators or funders believe to be model or successful sites while more troublesome sites are hidden or excluded from consideration. This is a power play that has definite ethical implications related to excluding certain perspectives from consideration.
Power plays, or attempts to gain an advantage by using certain tactics to magnify one’s influence or power, can be exhibited by the evaluator, as well as by various stakeholders. For example, power plays by those being evaluated (e.g., program staff) include denying the need for an evaluation, claiming the evaluation will take too much time away from their normal workload, and/or intentionally providing the evaluator with huge amounts of information so it is difficult to sort out what is relevant and what is not (International Program for Development Evaluation Training [IPDET], 2009). Power plays by the evaluator might include using the “experts know best” line, applying unstated criteria to decision making, and/or applying unstated values and ideological filters to the data interpretation (IPDET, 2009). Other stakeholders, such as community members, can also engage in power plays with ethical ramifications (see the following case studies for additional examples).
Case Studies of: Political Power Plays in Evaluation With Ethical Ramifications
Political Power Plays Engaged in by Evaluatees
Denying the need for the evaluation
Claiming the evaluation will take too much time away from their normal workload
Claiming the evaluation is a good thing, but introducing delaying tactics
Providing the evaluator with huge amounts of information so it is difficult to sort out what is relevant and what is not
Omitting or distorting information they are asked to provide so they do not look bad
Coming up with new data at the end
Arguing that the evaluation findings are irrelevant because things have changed
Political Power Plays Engaged in by Evaluators
Using the “experts know best” line to exclude the perspectives of others
Insisting evaluations should only be quantitative in nature since statistics do not lie
Not stating or shifting the measurement standards
Applying unstated criteria to decision making
Applying unstated values and ideological filters to the data interpretation
Ignoring certain evaluation findings
Political Power Plays Engaged in by Other Stakeholders
Giving their own conclusions to meet their own agenda
Trying to get the media (or powerful others) to criticize (or praise) the organization being evaluated in order to sway opinion
Source: Adapted from International Program Development Evaluation Training (2009).
Summary
A critical task for evaluators in any evaluation is to identify issues, including those of an ethical nature, that might hamper the conduct of a fair, honest, and accurate evaluation. This chapter examined evaluation ethics and the quality standards that are expected to govern the behavior of evaluators and the outcomes of an evaluation. Evaluators must take necessary steps to equip themselves with the knowledge, skills, and dispositions to accomplish this goal. This means having the sensitivity to identify and deal with the ethical challenges in the evaluation context. This chapter highlighted some common ethical challenges and offered possible solutions. Special consideration was given to how conflicts of interest, cultural issues, racial bias, and political issues impact evaluation ethics. The origin of research ethics, why they are important, and the three ethical principles from the Belmont Report were discussed to provide readers with a foundation for better understanding current evaluation ethics. The AEA’s Evaluators’ Ethical Guiding Principles and the Joint Committee on Standards for Educational Evaluation’s Program Evaluation Standards, although unable to cover every possible scenario that an evaluator might face, were discussed to provide a framework that gives guidance to evaluators.
In conclusion, the following is a set of reflective questions, adapted from Patton (2003, pp. 408–409), that evaluators can ask themselves to help them think through some ethical issues that might arise during their work.
How will the evaluation contribute to society, the community, and/or the world?
Why should individuals participate in your project? What are the benefits to them?
How will you explain the purpose of the inquiry and methods to be used in ways that are accurate and understandable to those you are researching?
In what ways, if any, will conducting this research or program evaluation put people at risk? (Consider psychological, legal, and political issues and the possibility of people becoming ostracized by others.)
If you uncover controversial information, how should it be shared?
What are reasonable promises of confidentiality that can be fully honored?
What information can you not promise to keep confidential?
What kind of informed consent, if any, is necessary for mutual protection?
Who will have access to the data, and why?
How will you and your respondent(s) likely be affected by conducting this research or program evaluation?
Who will be the researcher or evaluator’s go-to person(s) during the study regarding ethical issues that might arise?
How hard will you press participants for data? Where will you draw the line?
What ethical framework and philosophy informs your work and ensures respect and sensitivity for those you study, beyond whatever may be required by law?
In the final analysis, evaluators must use their own moral compass, in conjunction with the guidance of the profession’s principles and standards, to take the most ethical and socially just course of action possible.
Supplemental Resources
Practical Strategies for Culturally Competent Evaluation
www.cdc.gov/dhdsp/docs/cultural_competence_guide.pdf
Provided on the website of the Centers for Disease Control and Prevention (CDC), this document includes a crosswalk table in Appendix A, listing each of the Program Evaluation Standards in column 1 with suggested strategies an evaluator can engage in to increase