Evaluation in Today’s World. Veronica G. Thomas
group and favoritism toward the more powerful stakeholders. Evaluators can maintain what Bazerman and Tenbrunsel (2011) refer to as “ethical blind spots” as a result of seeing the world in a way that obscures one to the fact that a wrong or an unethical action (e.g., ignoring the perspective of the powerless) is occurring. In such instances, evaluators fail to see their own biases and subsequently fail to detect ethical dimensions in certain situations as a result. The activity that follows summarizes three sources of ethical blind spots evaluators might face: implicit bias, temporal distance, and failure to notice others’ unethical behavior.
Activity: Ethical “Blind Spots” in Evaluation
Sezer and colleagues (2015) delineated various sources of ethical blind spots. After a review of these sources of ethical blind spots, summarized as follows, organize into small groups. Then discuss them as sources of evaluator blind spots. Give specific examples of how these sources might affect evaluators’ sensitivities and course of action in practice.
Implicit biases: These include attitudes or stereotypes that affect individuals’ understanding, actions, and decisions in an unconscious manner and can lead those individuals to act against their ethical values.
Temporal distance: Individuals overestimate the extent to which they will behave ethically in the future; therefore, temporal distance from decisions with ethical dimensions can be a source of unintentional ethical behavior.
Failure to notice others’ unethical behavior: Certain factors lead people to ignore the unethical behaviors of others including self-serving biases (e.g., if unethical behavior benefits them), outcome bias, the presence of intermediaries, and gradual erosion of ethical behavior; actions that produce negative outcomes are perceived as more unethical than similar actions that produce positive outcomes.
Another aspect that Sezer et al. (2015) discussed was the slippery slope effect or the gradual deterioration of ethical behavior. Here, individuals are more likely to justify small ethical indiscretions than major ones; however, over time, as they justify more and more, they can be led to justify even big indiscretions. When faced with abrupt and large dilemmas (rather than those that gradually increase), individuals are less likely to be unethical.
There are strategies that an evaluator can use to overcome ethical blind spots and be able to better identify ethical issues in real time. Among these are active listening (i.e., hearing a speaker and avoiding premature judgment, asking questions, reflecting understanding, clarifying information by restating a paraphrased version of the speaker’s message, and summarizing the conversation), imagining the perspectives of others, and practicing cultural humility and openness to others’ points of view.
A second aspect affecting evaluators’ ethical sensitivity is their ability to identify the particular ethical value(s) underlying the issue under consideration. After gaining awareness of a potential ethical issue, evaluators need to reflect by asking themselves what ethical values are being compromised. In other words, an evaluator must first recognize there is an event to react to and then define an event as having an ethical dimension. Identify any ethical issues that might be facing the evaluator in the case study that follows.
Case Study: The Compromised Evaluator?
In an effort to become the long-term evaluator of a five-year project, an evaluator presents the project in a slightly more favorable light in the Year I Annual Report than in her original draft report submitted to the project administrator.
What ethical violations are present in this case, and how might such violations actually hurt (not help) the project being evaluated?
The third aspect affecting evaluators’ ethical sensitivity is their awareness of the intensity of the issue. Evaluators do not treat all ethical issues the same. A decision must be made whether the ethical dimension is significant or important enough for the evaluator to do something about it. Ethical intensity often plays a role in how an evaluator will act. Intensity pertains not only to the awareness of an ethical issue but also to how important such issues are to various stakeholders. Thomas M. Jones (1991) argues that ethical decision making in organizations is generally a function of six factors, each of which we view as relevant to program evaluation:
Magnitude of consequences: the total harm or benefit that participants or the evaluator can derive from an ethical action
Social consensus: agreement among evaluators or the evaluator, client, and other stakeholders whether a behavior is “good” or “bad”
Probability of effect: the chances that something will happen and result in harm to others
Temporal immediacy: the time between the act and the consequences the act produces
Proximity of effect: the social, psychological, cultural, or physical distance of the decision maker (e.g., evaluator) from the beneficiary or victim (e.g., program stakeholders) of the course of action
Concentration of effects: the number of people affected or how much an action affects the average person (pp. 374–378)
All six factors represent characteristics of the ethical issue itself and are expected to have interactive effects. Jones (1991) theorizes that if any factor increases, it is generally expected that the overall level of intensity will also increase, and vice versa, assuming all remaining components are constant. For example, depending on the extent to which evaluators believe that their unethical behavior will result in an immediate impact (i.e., temporal immediacy), they are more or less likely to engage in such behavior.
Sources of Ethical Dilemmas
There are multiple sources of ethical dilemmas for the evaluator. Mathison (2007) identified three sources of ethical dilemmas in evaluation that are not mutually exclusive. One source includes ethical issues that arise from doing the evaluation. These, she argues, are manifested in various ways such as delivering evaluation findings or reports that (a) are laundered to omit negative findings, (b) exaggerate successes and positive findings, (c) are suppressed altogether, (d) are released belatedly so they are no longer relevant, and (e) are prematurely released or leaked to the public. Reducing complex evaluation findings into sound bites can become an ethical issue when they are misleading. For example, based on some positive findings from the evaluation, a program administrator can post flyers or hold press conferences claiming “our treatment works.” This fails to communicate the complexity of the findings that “treatment worked” only for certain groups of participants and not others.
A second source of evaluation dilemmas identified by Mathison (2007) includes ethical issues that are created by the evaluator. Relevant examples include the evaluator’s (a) personal or financial interest in the evaluand (i.e., conflicts of interest), (b) lack of knowledge or skill in the evaluation technique or method being used, (c) lack of cultural competence and sensitivity (such as lack of knowledge and respect for local culture and values), (d) ideological positions or values that can bias the evaluation outcome, and/or (e) propensity to deliver positive evaluations to increase job security. When evaluators make promises that they cannot deliver such as agreeing to totally unrealistic timelines just to secure a contract, another ethical dilemma arises as a result of their behavior. Ultimately, ethical evaluators have a responsibility to be honest, independent, impartial, credible, and accountable for their work and to the public. They also have the ethical responsibility to respect the rights, dignity, and diversity of participants; to do them no harm; and to maintain their dignity and confidentiality.
The third source of ethical dilemmas identified by Mathison (2007) includes ethical issues that do not arise from the conduct of the evaluator or from doing the evaluation, but instead exist within the context of the evaluand and are discovered when planning or conducting the evaluation. When such an issue is uncovered by the evaluator, it unexpectedly places the evaluator in an ethical dilemma. Examples include uncovering (prior to any data collection activity) that program administrators are engaging in illegal activities (e.g., theft) and malfeasance