Evaluation in Today’s World. Veronica G. Thomas
are (or should be). Biases can be explicit—that is, one knows one has a particular bias. For example, we might be biased in favor of people who like Ben & Jerry’s ice cream and biased against those who like Häagen-Dazs ice cream. When a bias is explicit, one can accept it or try to counteract it.
Evaluators can, and most often do, have explicit biases. Our biases may be related to methods—for example, being biased against use of online surveys or biased in favor of programs that include a component for participant reflection. Evaluators may also be biased in terms of what they think participants in a program need to be successful. If evaluators have explicit biases that can impact their work, they need to let others know their biases exist and to have others check to see if those biases are impacting the work. It is important for evaluators to remember that, as Hannum (2018, para. 4) points out,
there is bias and error in all information. Understanding how information can be biased is helpful. Equally helpful is understanding the roots of bias within ourselves. We often think of other people deceiving us, but the best place to begin to whittle away nonsense is within ourselves. The more we know about how to gather, interpret, and use information, the less likely we are to get caught up in assumptions, bias, and outright deception.
The following activity provides an opportunity for readers to reflect on and discuss their own explicit biases.
Reflect and Discuss: My Biases
In small groups, discuss some of the fairly superficial preferences and biases you might have. Then, either speaking in general or personally, discuss preferences that might impact how someone approaches a project.
Implicit Bias
While some biases are explicit, others are implicit. According to the Kirwan Institute at The Ohio State University (Staats, Capatosto, Tenney, & Mamo, 2017, p. 10), implicit bias refers to “the attitudes or stereotypes that affect our understanding, actions, and decisions in an unconscious manner.” Like explicit biases, they can impact assessments and judgments, both favorably and unfavorably. But unlike explicit biases, implicit biases are “activated involuntarily, without awareness or intentional control” (Staats et al., 2017, p. 10). The following are some key characteristics of implicit biases (Staats et al., 2017).
Implicit biases are pervasive. Everyone possesses them, even people with avowed commitments to impartiality, such as judges.
Implicit and explicit biases are related but distinct mental constructs. They are not mutually exclusive and may even reinforce each other.
The implicit associations we hold do not necessarily align with our declared beliefs or even reflect stances we would explicitly endorse.
We generally tend to hold implicit biases that favor our own in-group, though research has shown that we can still hold implicit biases against our in-group.
Implicit biases are malleable. Our brains are incredibly complex, and the implicit associations that we have formed can be gradually unlearned through a variety of de-biasing techniques.
It is difficult to understate the importance of considering the role of implicit bias when analyzing societal inequities. Implicit biases, explicit biases, and structural forces are often mutually reinforcing. Research on implicit bias suggests that many of our decisions regarding racial stereotypes are made at unconscious level (e.g., Greenwald & Banaji, 1995; Staats, 2017). For example, Harvard University’s Project Implicit (2011b) has found most Americans have an automatic preference for white people over Black people, and often have automatic preferences for straight people over gay people and for young people over old people. In addition, the Project Implicit researchers have found stronger links between females and family and between males and careers. Similarly, they have found stronger links between females and the liberal arts and between males and science. This does not mean that people are racist or homophobic or believe that a woman’s role is in the kitchen, but it does mean that people are influenced, consciously and unconsciously, by the environment around them. Referring to race, Project Implicit (2011b, para. 17) explains that “implicit preferences for majority groups (e.g., white people) are likely common because of strong negative associations with Black people in American society. There is a long history of racial discrimination in the United States, and Black people are often portrayed negatively in culture and mass media.” There are also implicit stereotypes, or those that are “relatively inaccessible to conscious awareness and/or control. Even if you say that men and women are equally good at math, it is possible that you associate math more strongly with men without being actively aware of it. In this case we would say that you have an implicit math + men stereotype” (Project Implicit, 2011b, para. 2). One way to explore implicit biases is, as described in the following activity, to take the Implicit Association Test (IAT).
Activity: Take the Implicit Association Test (Optional)
The IAT takes about 10–15 minutes to complete. It measures attitudes and beliefs that people may be unwilling or unable to report and “measures the strength of associations between concepts (e.g., [B]lack people, gay people) and evaluations (e.g., good, bad) or stereotypes (e.g., athletic, clumsy)” (Project Implicit, 2011a, para. 1). The results will be immediately reported to you and will mention possible interpretations that have a basis in research. If you are unprepared to encounter interpretations that you might find objectionable, please do not take the test. You may want to go to https://implicit.harvard.edu/implicit/iatdetails.html for more information about the test.
To take the test, visit https://implicit.harvard.edu/implicit/selectatest.html.
In his 2019 Voices From the Field interview, as follows, Melvin E. Hall discusses objectivity, bias, and why he became an evaluator.
Voices From the Field
Melvin E. Hall: Objectivity, Bias, and Being an Evaluator
I came into evaluation in part because I was aware of the myth of evaluator objectivity. People have built-in biases; we all do. Once you recognize that people have built-in structural biases, you have two choices—you can call them out on their biases, or you can find ways to have those biases balanced with other perspectives. I made the second choice.
The value I bring to evaluation is another worldview, one which will help other people understand their own biases as well as mine. Inherent bias is not only not unavoidable, but you don’t even want to try to avoid it—I believe that any knowledge you have biases you. For example, if I know the world is round, it biases me about believing anything contingent to the world being flat. That’s a positive bias. I don’t see bias as a negative thing. I see it as a necessary thing. The one caveat is not when bias is in the performance of the craft, but when bias is in the assumptions underlying the craft. Bias can be thorny to observe and thorny to deal with when melded into underlying assumptions.
There are little-b biases and big-B biases. Little-b biases impact how you communicate, how you collect data, and how you interpret experiences; it is the little day-to-day stuff. We have to recognize it will be there and be alert to it. I like to think that big-b bias is something I will be upfront about and take steps to mitigate. For example, I had a project that I had a positive bias toward. I know I am pro HBCUs [Historically Black Colleges and Universities]. I asked several colleagues to be my sounding board. I would send research briefs to them for their reaction. The process of writing it and thinking about it was one of the best things I could do to be aware of my biases.
Melvin Hall is a Professor of Educational Psychology at Northern Arizona University, Distinguished Scholar in the Marie Fielder Institute of Fielding Graduate University, and AAC&U Senior Scholar, Office of Undergraduate STEM Education, AAC&U He is also a founding affiliate faculty member of the Center for Culturally Responsive Evaluation