Evaluation in Today’s World. Veronica G. Thomas
meaning that it does not flow back and forth, one day benefiting whites and another day (or even era) benefiting people of color. Instead, the direction of power between whites and people of color is historic, traditional, normalized, and deeply embedded in the fabric of U.S. society (DeAngelo, 2011).
5 Race and racism continue to have tremendous consequences for the work of social and behavioral science researchers, and as such, evaluators are certainly not detached from these socially constructed phenomena.
Other Social Justice Issues
This book has an underlying focus on race and examines evaluation from a racialized perspective, as is covered in detail in Chapter 5. It also focuses on other social justice issues, including those tied to sex and gender where, over time, there has been great change in the ways people are identified and categorized. Traditionally, one identified or was identified as either female or male. When the concept of gender was introduced, it was often used interchangeably with sex, although gender, like race, is a socially constructed phenomenon. It includes how individuals see themselves, how others perceive them and expect them to behave, and the interactions that they have with others (Conger, 2017, para. 21). In terms of both gender and sex, there can be fluidity and change. “Most people—including most transgender people—are either male or female. But some people don’t neatly fit into the categories of ‘man’ or ‘woman,’ or ‘male’ or ‘female.’ For example, some people have a gender that blends elements of being a man or a woman, or a gender that is different than either male or female. Some people don’t identify with any gender. Some people’s gender changes over time. People whose gender is not male or female use many different terms to describe themselves, with non-binary being one of the most common” (National Center for Transgender Equality, 2018, paras. 1–2).
In the case of race or sex or other areas including disability, sexual orientation, and socioeconomic status, there are those with more or less privilege. Those with less privilege are more apt to be discriminated against. Privilege has been defined as “unearned access to resources (social power) that are only readily available to some people because of their social group membership; an advantage, or immunity granted to or enjoyed by one societal group above and beyond the common advantage of all other groups. Privilege is often invisible to those who have it” (National Conference for Community and Justice, n.d., para. 7). Discrimination can be defined as “the unequal allocation of goods, resources, and services, and the limitation of access to full participation in society based on individual membership in a particular social group; reinforced by law, policy, and cultural norms that allow for differential treatment on the basis of identity” (National Conference for Community and Justice, n.d., para. 4). Table 1.2 defines some common belief systems that negatively affect marginalized groups and lead to privilege for dominant groups.
Table 1.2
Source: National Conference for Community and Justice, n.d., paras. 12–13, 15–18.
In many of these areas including sex, race/national origin, and sexual orientation there has been de jura discrimination—that is, legal discrimination. For example, until the 1967 Supreme Court decision in Loving v. Virginia, in some states, Blacks and whites were not allowed to marry (National Constitutional Center, 2019). It wasn’t until 2015 and the decision in Obergefell v. Hodges that same-sex couples were able to marry anywhere in the United States (Liptak, 2015). Straight women were legally banned from many jobs in the armed services (Pruitt, 2018), and lesbian, gay, bisexual, transgender, and queer (LGBTQ) individuals were not permitted to serve at all (Human Rights Campaign, 2020). In 2019, a ban on transgender individuals serving in the military was enacted (D. Phillips, 2019).
Because of the relentless efforts of many people over time, there have been a number of successful efforts to limit or eliminate de jura discrimination. However, another form of discrimination has been much more difficult to dismantle and is much more apt to impact evaluators and evaluations. That is de facto discrimination, or discrimination that is not sanctified by law but happens in fact. For example, racial segregation in schools was allowed by law, but the 1954 Supreme Court decision in Brown v. Board of Education determined that segregation in schools was unconstitutional. Today, however, many public schools remain segregated not by law but in fact (Orfield & Frankenberg, 2014). The situation is the same in terms of housing. In 1968, the Fair Housing Act was passed to protect people from discrimination when they are renting or buying a house, but today there is still great segregation in housing (Schuetz, 2017). Discrimination has real consequences for real people in real programs. For example, being in environments that are racially segregated can impact the context in which programs are implemented and the responses of participants to programs. This needs to be a concern to those evaluating programs.
While de jura job segregation by sex no longer exists, de facto segregation does, with men predominating in the more prestigious and more highly paid careers. This has implications for evaluations done in the workplace in terms of the culture and acceptance of women in fields such as engineering and construction where men predominate and in fields like elementary school teaching and nursing where women predominate.
Much of this de facto segregation is based on stereotypes, or “preconceived notion[s], especially about a group of people” (Vocabulary.com, n.d., para. 1), and assumptions that people have about others because of their race, sex/gender, disability, and other areas. As will be explained in more detail in Chapter 5, everyone fits into more than one demographic group, some of which are marginalized such as being poor, female, a person of color, and a person with disabilities. Crenshaw (2017, para. 4) calls this intersectionality, “a lens through which you can see where power comes and collides, where it interlocks and intersects. It’s not simply that there’s a race problem here, a gender problem here, and a class or L[GB]TQ problem there. Many times that framework erases what happens to people who are subject to all of these things.” Throughout this book are examples of how stereotypes and perceptions about individuals because of their race, sex/gender, disability, and other characteristics can sometimes negatively influence the views of evaluators as well as those of program staff and funders, thus impacting the conceptualization, implementation, and outcomes of evaluations. However, throughout the book we offer some tangible strategies for how this impact can be counteracted.
Objectivity and Bias
Objectivity
For hundreds of years, philosophers of science have commented on the difficulty of attaining scientific objectivity. In 1821, Isaac Watts described the near impossibility of being unbiased: “The eyes of a man in the jaundice make yellow observations on everything; and the soul tinctured with any passion diffuses a false color over the appearance of things” (AZ Quotes, n.d., para. 1). More recently, Nage (1961) wrote about the difficulty of preventing our likes, aversions, hopes, and fears from coloring our conclusions. Looking at the issues of objectivity from a different perspective, Martin, Lee, and Bang (2014, para. 10) suggested that “it is commonly said that scientists should have a professional distance from what they study. But the metaphor of distance is misleading. Science, like a painting, necessarily has a perspective. And that perspective is at least partially shaped by variables such as race, gender and class.” When we move past the concept that scientists and evaluators are objective, we are able to look more clearly at biases, including our own.
Bias
Explicit Bias
We all have biases, and we need to pay attention to the biases people have as individuals and as evaluators. Bias has been defined as a particular tendency or inclination, especially one that prevents reasonable, knowledgeable, thoughtful consideration of a question (Harmon, 1973). While bias can be intentional, it often is not. Bias can grow out of one’s assumptions—the things one accepts as true without questioning. It can