.
or even negative. For example, one evaluation that I, coauthor Campbell, did found that doing hands-on science activities created by teachers or by after-school leaders caused students to become more stereotyped and limited in their opinions of who could do science. A second program to encourage women to continue on in their engineering programs reinforced rather than overcame stereotypes, with some women in the program coming to feel that it existed because women weren’t as good as men in engineering. As one participant explained: “[Engineering] theory is easier for boys. That is why they put us together [in the special program]” (Campbell & Hoey, 2000, p. 20). No one wants to hear negative outcomes, but problems can’t be fixed unless they are found and acknowledged.
The Current Climate
Of great concern is the onset of a so-called post-truth era, complete with alternative facts, disdain for expertise, and a diminishing reliance on facts and analytic thinking in public life (Hamburg, 2019, p. 563). Evaluators and indeed people in general are living and working in a time where terms like alternative facts, post-facts, and post-truth are used regularly. As indicated earlier in the chapter, traditionally alternative facts have been defined as falsehoods or the “opposite of reality.” However, now some are viewing alternative facts as merely a different perspective (Zimmer, 2017). Post-fact and post-truth both refer to an environment in which objective facts are a thing of the past. In a post-fact society, facts are viewed as irrelevant, and emotional appeals are used to influence public opinion. In 2016, the Oxford University Press named post-truth as its word of the year, defining it as “relating to or denoting circumstances in which objective facts are less influential in shaping public opinion than appeals to emotion and personal belief” (para. 2). The idea that “truth no longer mattered. Facts were not just unimportant, but barriers to be smashed through with rhetoric” (Hollo, 2017, para. 3) is increasingly characteristic of today’s world. Evaluators need to explore how such constructs impact evaluation work. “Citizens are increasingly asserting their values, hopes and opinions without apparent interest in finding a shared understanding of the actual state of things. Without such a shared understanding, those values and hopes cannot rationally be expressed and realized. Observers speak of ‘truth decay,’ dismissal of expertise, and neglect of evidence” (Hoit, 2019, p. 433).
The implication of living in a post-truth culture is that science becomes just another perspective with evidence becoming no more valid than personal opinion. This blurring of the distinction between evidence and opinion has a corrosive and delegitimizing effect on evaluation. Evaluation findings become just another kind of opinion (Gauchat, 2012).
There are some strategies and ways to combat this trend. Chapter 14 covers ways that the language and images used can impact readers’ or viewers’ response to an evaluation and provides strategies to increase the probability that results will be heard. In addition, having everyone, including evaluators, become more critical consumers of information can make a big difference. This can include the application of the aptly named CRAAP Test to help individuals judge the quality and veracity of information. CRAAP stands for
currency—the timeliness of the information, including when it was published and if it requires current information;
relevance—the importance of the information for your needs, including if it answers your question;
authority—the source of the information, including the names and affiliations of authors as well as their qualifications and credentials;
accuracy—the reliability, truthfulness, and correctness of the content, including where it comes from, if it is supported by evidence, and if it has been reviewed or refereed; and
purpose—the reason the information exists, including determining if that purpose is to inform, teach, sell, entertain, or persuade and if there are political, ideological, cultural, religious, institutional, or personal biases (Meriam Library, 2010).
Drawing on the work of Julian Baggini, Eann Patterson (2018b, para. 1) provides some suggestions for ways individuals, can validate “truth”:
We need to trust experts because we are unable to verify everything ourselves as life is too short and there are too many things to think about. However, this approach exposes us to the risk of being misled and Julian Baggini has suggested that this risk is increasing with the growth of psychology, which has allowed more people to master methods of manipulating us, that has led to “a kind of arms race of deception in which truth is the main casualty.” He suggests that when we are presented with new information then we should perform an epistemic triage by asking:
Is this a domain in which anyone can speak the truth?
What kind of expert is a trustworthy source of truth in that domain?
Is a particular expert to be trusted?
In the following activity, readers apply the CRAAP Test and/or Patterson’s three questions to news stories.
Activity: Validating “Truth”
Pick a front-page story on the same topic or a similar one from two different newspapers and apply the CRAAP Test to them and/or ask Patterson’s three questions with them in mind. Did doing that change your perceptions of the articles? In small groups, discuss your findings.
Summary
The goal of this chapter was to provide readers with an overview of evaluation, evaluative thinking, and the general themes that underlie the book. These include race, racism, social justice, and a racialized perspective. The chapter also introduced issues tied to objectivity and bias including explicit and implicit bias and ways evaluators can reduce these biases. We also introduced the concepts of culture and cultural competence and the impact of politics and the current political climate on evaluators and evaluation. All of these, and other issues, will be explored, in detail, throughout the remainder of the book.
Supplemental Resources
Bruner Foundation: Effectiveness Initiatives
www.evaluativethinking.org/index.html
This website provides a series of resources on developing and building evaluation capacity as well as on evaluative thinking including a series of four booklets on applying evaluative thinking to data collection.
Equitable Evaluation Initiative
This website provides principles, frameworks, and resources on equitable evaluation as well as an overview of the equitable evaluation projects in which the organization was involved.
Evaluation Resource Hub: Evaluative Thinking
This website provides a variety of evaluation resources including the mindset, skill set, and values underpinning evaluative thinking.
National Parenting Education Network: Introduction to Evaluation
This brief introduction to evaluation includes