Evaluation in Today’s World. Veronica G. Thomas
type of ethics relevant to the evaluator, situates ethical action explicitly in relationships recognizing and valuing mutual respect, dignity, and the connectedness between the researcher and the researched and between researchers and the communities in which they live and work (Ellis, 2007).
Ethical Dimension of Racial Bias
Race is only one of many cultural constructs. However, it is a powerful one in American society and one that deserves special attention. Race, as a socially constructed phenomenon, continues to differentially shape the allocation of power and distribution of benefits and burdens among groups within the United States. When considering the prominent ills (e.g., poverty, crime, education gaps, health disparities) of the country, race is always a factor in the equation. Over two decades ago, Patton (1999) questioned how the lens of race shapes and affects evaluators’ understanding and actions. This question still has relevance today.
Racism can be a complex and destructive force in the evaluation context. Evaluators have an ethical obligation to eliminate, or at least mitigate, racial (and other) biases in their work. In 2018, Thomas, Madison, Rockcliffe, DeLaine, and Lowe called for evaluators to use their power and privilege to advance a more equitable society by “calling out” racial biases, policies, and practices and unmasking power inequities and outcomes of people in programs within the context of their racialized environments. Racial bias, particularly as it relates to African Americans, is due in large part to their unique historical position in the United States, and it can be present in both evaluators and evaluation stakeholders such as funders, policymakers, program staff, and others.
Biased conceptions of race can hinder an evaluator’s ability to evaluate culturally and socially different worlds and realities. This, in turn, can prevent an evaluator from rendering ethical, honest, and fair evaluations. As discussed in Chapter 1, implicit bias, or unconscious attitudes and stereotypes, can affect evaluators’ understanding, decisions, and actions in a manner that lends itself to questionable ethical conduct. Such biases are activated involuntarily and without an individual’s awareness or intentional control. Many people do not deliberately discriminate against others, but when they offer preferential treatment to those who are like them, whom they know, and/or whom they like or feel most comfortable with, the resulting outcome can be unconscious bias and discrimination against those who are different. Individuals typically fail to recognize the harm that implicit favoritism of in-group members causes to members of the social out-groups (Sezer, Gino, & Bazerman, 2015). Evaluators, like other professionals and laypersons, are not immune to implicit biases. In the article “Evaluation and the Framing of Race,” House (2017, p. 188) pointed out that
a common error of evaluations is that the conclusions of the study don’t fit the data on which they’re based. This is more likely when the evaluations involve minorities. We need to weigh the empirical evidence carefully against the conclusions. Inferences from data to conclusions are particularly susceptible to bias because they’re based on more background knowledge than just the study.
Evaluators should be prepared for racial biases (House, 2017). As House and others point out, majority evaluators are often poorly informed and even misinformed about minorities and thus ill equipped to evaluate projects serving communities of color. As a result, various scholars have called for use of evaluation approaches, such as those that are culturally responsive, contextually responsive, culturally competent, and transformative and that emphasize inclusiveness, pluralism, and better understanding of minority cultures on the part of majority evaluators (e.g., Frierson, Hood, Hughes, & Thomas, 2010; Hood, Hopson, & Frierson, 2005, 2015; Hopson, 2009; Mertens, 2009; Samuels & Ryan, 2011; Thomas & Stevens, 2004; Thompson-Robinson, Hopson, & SenGupta, 2004). House (2017) calls for (majority) evaluators to check their own predispositions, arguing that no white person growing up in this country can be entirely free of racial framing. He also stresses that evaluators should check the work of their colleagues for such dispositions. Evaluators can be “critical friends” and look for biases that evaluators might harbor in their beliefs, dispositions, and behavior given that such biases can significantly affect evaluation findings (House, 2017, p. 187). This is indeed an ethical responsibility for evaluators.
Case Study: Raising the Issue of Racism in Evaluation of a Program
Mathison (1999) describes how external evaluators working with a group of doctoral students to evaluate an after-school program for inner-city teens was able to raise the issue of racism. She contends that it was much easier for the external evaluator, in contrast to an internal evaluator, to raise issues of racism that were subtle and insidious within the program.
Mathison’s Description of the Case
The program focused on a transition-to-work program and, as such the [inner-city African American] teens were responsible for finding their own internships within the organizations, in this case a museum. This organization provided a vast array of potentially valuable experiences for them [the teens], but for the most part they either floundered about looking for work or worked in menial jobs, such as cleaning the cafeteria. Underlying the program’s inability to provide positive, meaningful work experiences for these teens was a deficit model of inner-city, African American teens—one that presumed they weren’t very able, by definition had psychological problems, and couldn’t really be counted on in things that mattered. The program itself was ethically flawed. As external evaluators we could much more easily raise these issues of racism than could an internal evaluator. We were not seen as having a vested interest (although the staff did feel betrayed by us for raising these issues, and the teens were grateful for being given an opportunity to reveal their perceptions) and were presumed by our expertise to be doing a fair job.
Source: Mathison (1999, p. 33).
The following case study describes an example of how evaluators noticed and highlighted racism inherent in a particular project. Even though Mathison (1999) admits that racism likely did not diminish as a consequence of the evaluation, she contends that at least the evaluation resulted in a range of stakeholders discussing the issue.
Ethical Sensitivity and Dilemmas
Ethics in evaluation is not simply related to doing the right thing or wrong thing, but is also related to ethical sensitivity. One of the early decisions that an evaluator may be confronted with is when a particular configuration of conditions, circumstances, and available choices should be framed and addressed as an ethical problem (Duggan & Bush, 2014). Ethical sensitivity includes an evaluator’s ability to recognize and respond to the ethical dimensions of diverse evaluation contexts. Evaluators with ethical sensitivity continuously anticipate and seek to address ethical dimensions in their work rather than be surprised by them or simply ignore them.
In practice, ethical sensitivity in evaluation involves three aspects: the evaluator’s awareness of or ability to determine whether a situation related to the evaluation or the evaluand involves an ethical issue, the evaluator’s ability to identify the particular ethical value(s) underlying the issue under consideration, and the evaluator’s awareness of the intensity of the issue.
First, the evaluator must be able to determine whether a situation related to the evaluation or the evaluand involves an ethical issue (i.e., awareness). This is a critical skill since oftentimes ethical issues go unnoticed because of their complexity or because of a particular cultural lens or unconscious bias on the evaluator’s part. There is a large body of research arguing that unethical behavior often stems from actions that individuals do not even recognize as unethical (Banaji, Bazerman, & Chugh, 2003; Chugh, Bazerman, & Banaji, 2005; Sezer et al., 2015; Tenbrunsel & Messick, 2004). Individuals have an “illusion of objectivity” by acting against their ethical values without conscious awareness of such behaviors (Bazerman & Tenbrunsel, 2011; Epley, Caruso, & Bazerman, 2006). For example, an evaluator may fail to realize that the perspectives of the most vulnerable program participants are being ignored by the funder or project director. Here, ethical insensitivity may occur due to the evaluator’s prejudices