Evaluation in Today’s World. Veronica G. Thomas
Applying Evaluative Thinking
Here are three areas and related questions that readers can ask themselves as part of an evaluative thinking process:
Identify and Challenge Assumptions & Assertions
“What are we assuming? Do we actually know that?”
“How do we know that?”
“What makes you say that?”
Seek Out Blind Spots
“What are we missing?”
“Whose perspective isn’t represented?”
“What other explanations could there be?”
Capture Musings & Learning Questions
“I wonder if …”
“I bet if we …”
“If I knew __________, I could _________.” (IllumiLab, 2018b, “Asking Questions”)
Read the following text about money.
oney can mean so many things to so many people. In evaluation, money can be an outcome, a confounding variable, or even a risk factor. Money is tied to access to resources and power. It is a key component that needs to be considered in evaluations.
Money as defined as annual family income, along with adult educational attainment are the conventional measures of socioeconomic status (SES) or class, although they are not the only ones. Indeed there is no consensus definition of class….
Making assumptions about income based on race, ethnicity or family education is dangerous and should be avoided. While there is a correlation between income and race and ethnicity, as well as between income and educational level in the United States, lower income and higher income families come in all colors and from all educational levels. Race, ethnicity and educational level are not proxy indicators of income or SES and should not be used as such. Indeed, evaluators should consider in their analysis disaggregating by race, ethnicity and educational level to tease out interactions.
Asking about income can be sensitive. Many people don’t feel comfortable discussing their income and often students don’t know their family income. Many evaluators use ranges the participant can choose such as $0–$25,000 or $25,001–$50,000 rather than asking for exact or even approximate numbers.
When working with lower income participants, particularly if they are in programs that provide them with financial support, evaluators should be sensitive to participant fears that if they don’t participate in the evaluation or if they raise concerns, that could impact their continued support from the program. This could impact whether their participation in the evaluation is truly voluntary and if their responses are free from pressure. (Campbell & Jolly, n.d.ah, paras. 1–5)
Now, organize in small groups and apply evaluative thinking by asking some of the preceding questions about the text you just read. Discuss your answers with others.
Courtesy of Campbell-Kibler Associates. Inc.
Race, Racism, Social Justice, and a Racialized Perspective
As will be explored more deeply in Chapter 5, race and racism are deeply embedded in the fabric of the United States and have had a complex and destructive influence on the lives of people of color. This influence extends to people’s participation in programs and even the very design of the programs being evaluated. It is critical that evaluators work toward unpacking how bias, in general, but racism, in particular, is a complex and destructive force including in evaluations. Thomas et al. (2018) point out that evaluators have both an opportunity and a responsibility to illuminate the potential impact of race and racism on the programs that they evaluate and the environments that they engage. They urge readers “to gain a deeper understanding of racism as a complex interplay of individual attitudes, social values, and institutional policies and practices and to bring these understandings to the work they do” (p. 515).
Race has been defined as “socially constructed differences among people based on characteristics such as accent or manner of speech, name, clothing, diet, beliefs and practices, leisure preferences, places of origin and so forth” (Ontario Human Rights Commission, n.d., para. 3). The process of social construction of race is called racialization: “the process by which societies construct races as real, different and unequal in ways that matter to economic, political and social life” (Ontario Human Rights Commission, n.d., para. 3). There is no fixed definition of racial discrimination. However, it has been described as “any distinction, conduct or action, whether intentional or not, but based on a person’s race, which has the effect of imposing burdens on an individual or group, not imposed upon others or which withholds or limits access to benefits available to other members of society” (Ontario Human Rights Commission, n.d., para. 2).
“Racism is a wider phenomenon than racial discrimination…. Racism is an ideology that either directly or indirectly asserts that one group is inherently superior to others. It can be openly displayed in racial jokes and slurs or hate crimes but it can be more deeply rooted in attitudes, values and stereotypical beliefs. In some cases, these are unconsciously held and have become deeply embedded in systems and institutions that have evolved over time” (Ontario Human Rights Commission, n.d., paras. 5–6). Racism is pervasive. As entertainer Beyoncé commented, “It’s been said that racism is so American, that when we protest racism, some assume we are protesting America” (Nyren, 2017, para. 4).
Racism operates at a number of levels—in particular, individual, systemic, and societal. It is important to note that “stating that racism privileges [W]hites does not mean that individual [W]hite people do not struggle or face barriers. It does mean that [they] do not face the particular barriers of racism” (Akintunde, 1999, p. 24).
A racialized perspective is one that explicitly foregrounds the impacts of society’s construction of races in ways that are unequal. This can be a difficult thing to do for members of the dominant culture, the group whose members are in the majority or who wield more power than other groups (SparkNotes, 2020). Members of the dominant culture, which in the United States are whites and other people of European origin, can be and often are influenced by the values, or system of thought, in a society that are most standard and widely held at a given time. This is referred to as the dominant paradigm. Being a member of the dominant culture with its standard and widely held values can impact one’s ability to recognize other, different systems of thought and values (Thomas & Campbell, 2017).
In 2018, Thomas et al. (p. 516) put forth five principled beliefs that explicitly guided their thinking about racism. These principles address concerns about dominant cultures and paradigms and guide the thinking of this text related to social programs, social justice, and evaluation of social programs. The principles are as follows:
1 Race is not a biologically determined reality but instead is a socially constructed phenomenon that continues to differentially shape the allocation of power and distribution of benefits and burden among groups within this country.
2 Racism is real, pervasive, and systematic. Race and racism are timeless, endemic, and permanently entwined within the social fabric of American society (e.g., D. Bell, 1995; Delgado & Stefancic, 2001; Feagin, 2013; C. Lawrence, 1995; Solórzano, 1997). As such, racism is not an aberrant but, instead, the natural order of American life, the usual way business is conducted in this society, and a common everyday experience for most people of color. Short-lived victories for persons of color slide into irrelevance as racial patterns adapt in ways that maintain white dominance (D. Bell, 1992).
3 Individual racists need not exist for institutional racism to persist in the dominant culture (Bonilla-Silva,