Evaluation in Today’s World. Veronica G. Thomas
methodologies (e.g., experimental designs) are the only factor necessary for justified conclusions and decisions is a barrier to adherence to the accuracy standards.
Evaluation accountability standards encourage adequate documentation of evaluations and a metaevaluation (evaluation of the evaluation) focuses on improvement and accountability for evaluation processes and products. Attention to accountability guides improvement during all phases of the evaluation, and it encourages reflection and a metaevaluative perspective in evaluators and evaluation users.
In his Voices From the Field interview, Michael Morris stresses that evaluators must uphold the Evaluators’ Ethical Guiding Principles and the Program Evaluation Standards and resist pressure to act unethically. This sometimes takes, as he points out, consideration of potential ethical challenges during evaluation planning, moral courage, and just the willingness to do the right thing.
Voices From the Field
Michael Morris: Ethical Considerations in Evaluation
Evaluators must act with integrity and see themselves as more than just methodological technicians as they uphold the Evaluators’ Ethical Guiding Principles and the Program Evaluation Standards. They should strive to understand the organizational and other cultures in which a project is embedded, because they cannot do justice to the evaluation without such an appreciation. Before the evaluation is designed and implemented, evaluators should consider the ethical challenges that might arise and find a way to introduce these topics into discussions with stakeholders during the contracting and negotiation phase, in addition to soliciting the stakeholders’ concerns. Having mildly uncomfortable conversations with stakeholders early on can reduce the likelihood of having to engage in much more difficult interactions later in the project. Doing this will also enable the evaluator, at a later point, to bring stakeholders’ attention back to those initial discussions, increasing the chances that the latter will act in accordance with whatever guidelines had been agreed upon. For example, pressure to misrepresent or ignore (unflattering) findings is frequently encountered by evaluators. Early discussion of how to deal with potentially unwelcome results in the evaluation report is a worthy investment of everyone’s time. Ultimately, moral courage is key for evaluators, particularly internal ones. Doing the right thing can put an evaluator at risk. Sometimes, however, the only reason for doing the right thing is that it is the right thing to do.
Michael Morris is emeritus professor of psychology at the University of New Haven and a former chair of the AEA Ethics Committee. He is the author of Evaluation Ethics for Best Practice: Cases and Commentaries (Guilford Press, 2008). Veronica Thomas interviewed Dr. Morris in the fall of 2019.
Evaluation Corruptibility and Fallacies
It is often said by numerous scholars and practitioners in the evaluation community that “evaluators must be able to speak truth to power.” Evaluation corruptibility and evaluation fallacies are two factors that can put an evaluator at risk of unethical decision making, jeopardizing evaluation quality, and, thus, an inability to “speak truth to power.” Fitzpatrick, Sanders, and Worthen (2004) use the term evaluation corruptibility to describe ways that evaluators may be convinced to go against ethical standards, thus engaging in ethical compromises or distortions. They point to five specific areas of evaluation corruptibility (Fitzpatrick et al., 2004, pp. 423–424):
Conflict of interest: a willingness to twist the truth and produce positive findings due to conflict of interest or other perceived payoffs or penalties (such willingness may be conscious or unconscious)
Unsubstantiated opinions: an intrusion of unsubstantiated opinions because of sloppy, capricious, and unprofessional evaluation practices
Prejudices and biases: “shaded” evaluation “findings” as a result of intrusion of the evaluator’s personal prejudices or preconceived notions
Inducements: obtaining the cooperation of clients or participants by making promises that cannot be kept
Not honoring commitments: failing to honor commitments that could have been honored
To avoid corruptibility, evaluators must be transparent and disclose any relationships (e.g., previous organizational ties or ties with program staff) that might predispose them to bias or give the appearance of bias. Further, they should in no way profit from the outcome of an evaluation. Familiarity with and adoption of the Evaluators’ Ethical Guiding Principles and the Program Evaluation Standards can provide much-needed guidance for dealing openly with situations that can impact ethical decision making and quality evaluations.
House (1995) considered the issue of evaluator corruptibility from a different perspective than Fitzpatrick et al. (2004). He suggested that evaluators can have the best intentions and may not be corrupt, per se, but, at times, may have a misunderstanding about their responsibilities. House referred to these misunderstandings as evaluation fallacies. A fallacy is a mistaken belief based on unsound argument deriving from reasoning that is logically inaccurate. House (1995, pp. 29–30) identified five evaluation fallacies that can have negative ethical consequences:
Clientism: the fallacy that doing whatever the client requests or whatever will benefit the client is ethically correct
Contractualism: the fallacy that the evaluator must follow the written contract without question, even if doing so is detrimental to the public good
Methodologicalism: the belief that following acceptable inquiry methods ensures that the behavior of the evaluator will be ethical, even when some methodologies may actually compound the evaluator’s ethical dilemmas
Relativism: the fallacy that opinion data the evaluator collects from various participants must be given equal weight, as if there is no basis of appropriately giving less priority to the opinions of peripheral groups than to those of more pivotal groups
Pluralism/elitism: the fallacy of allowing powerful voices to be given higher priority because the evaluator feels they hold more prestige and potency than the powerless or voiceless
Evaluator Role, Power, Politics, and Ethics
Ethical issues can arise centering on the evaluator roles, power imbalances between the evaluator and key stakeholders, and evaluator privilege. Politics can also have ethical dimensions that impact an evaluator’s work. As discussed throughout this book, power and privilege are concepts that extend far beyond an individual evaluator or a particular evaluation. Frequently, relationships between the evaluator and stakeholders and between/among stakeholders are enthralled in power imbalances and hierarchical struggles. Hierarchical arrangements and power imbalances in the evaluation context exist long before the evaluator is on the scene since oppressive systems often shape the conception, design, and implementation of the program that the evaluator is tasked with studying. Evaluators are often asked to assess the effectiveness of social programs that are designed to yield a quick “magic bullet” fix to problems (e.g., racial achievement gaps, poverty) derived from years of racial and other oppressions (Thomas et al., 2018). In order to accomplish this, evaluators must develop a critical consciousness of how institutional, historical, and systemic forces limit and promote the life opportunities for particular groups. Instead of, for example, identifying delinquency, substance abuse, and violence as problems, evaluators should emphasize the root causes by examining the larger political, economic, and social forces that create persistent poverty, thus jeopardizing healthy development (Thomas et al., 2018).
In any given evaluation, the evaluator occupies multiple roles, including those of expert, knower, judge, and educator. For example, an evaluator can be an expert or program facilitator during the program implementation, a researcher when collecting and analyzing evaluation data, a judge during the reporting phase when making an assessment of program merit and worth, and an educator or advisor throughout the entire evaluation process. The roles that evaluators assume are generally all positions of tremendous power with opportunities