Evaluation in Today’s World. Veronica G. Thomas
might consult with trusted colleagues about an ethical dilemma in order to get the perspective of other evaluators before taking action.
Personal values and beliefs: Evaluators use their personal values, visions, and beliefs to make a decision about the right course of action to take when mistreatment of clients was observed while visiting the program site for the purpose of meeting with the program staff.
Rules and codes: Evaluators refer stakeholders to guidelines and standards provided by the AEA’s Evaluators’ Ethical Guiding Principles and the Joint Committee on Standards for Educational Evaluation’s Program Evaluation Standards to justify their behavior.
Source: Adapted and updated from Newman and Brown (1996).
Any one, or a combination, of these sources can be appropriate (or inappropriate) under certain circumstances. For example, evaluators can have “educated intuition,” making them more sensitive and alert to the nuances of potential ethical conflicts if they have thought through conflicts before either in simulations or in real life (Newman & Brown, 1996). However, overreliance on one particular source can become problematic. Evaluators who, for instance, primarily rely on intuition or past experience to guide their ethical thinking might inappropriately generalize what was seemingly ethical behavior in a previous setting to a different context that might render that behavior problematic. Evaluators who exclusively use their own personal values and beliefs to make ethical decisions may be on ethically shaky grounds, particularly when they are from the dominant group and working in a setting that primarily serves persons of color and other marginalized groups such as people with disabilities and the elderly. For example, sometimes the values, beliefs, and experiences of the dominant group are used as the yardstick, point of reference, norm, or standard by which persons from the dominant group judge the values, beliefs, and behaviors of marginalized groups. Since values and beliefs are influenced by individuals’ cultural background, the use of a dominant standard by the evaluator can be inappropriate and lead to faulty conclusions. For example, as shown in the following case study, the fact that an instrument has demonstrated validity in a dominant setting does not automatically make it ethical to use that same instrument in another setting.
Case Study: Moving Beyond Past Experience
In the Centers for Disease Control and Prevention’s (CDC, 2014, p. 16) Practical Strategies for Culturally Competent Evaluation, the following scenario is presented:
An existing validated instrument was piloted as part of an evaluation that assessed risk factors related to heart disease and stroke. Some of the items in the instrument dealt with sensitive issues (e.g., cultural eating practices, cultural perceptions of attractive body images, cultural views on prescribed medications). Respondents were offended by some of the items, which they viewed as racial stereotypes. The inappropriate items led evaluators to conclude that participants would be reluctant or refuse to complete the evaluation protocol. Consequently, the evaluation team members discussed these issues, which resulted in a revised protocol for culturally appropriate communication and the subsequent revision of the data collection instrument.
Cultural Competence as an Ethical Imperative
Evaluators oftentimes work in settings that represent a very different cultural context than their own. The updated AEA (2018b) Evaluators’ Ethical Guiding Principles highlight the need to mindfully and proactively attend to diversity, equity issues, and common good as prerequisites for ethical practice. Evaluators have an ethical obligation to be culturally competent and to create an inclusive climate in which everyone invested has an opportunity to fully participate in the evaluation process. In the updated preface to the Evaluators’ Ethical Guiding Principles, a culturally competent evaluator is described as one who
draws upon a wide range of evaluation theories and methods to design and carry out an evaluation that is optimally matched to the context; the evaluator reflects the diverse values and perspectives of key stakeholder groups. (AEA, 2018b, p. 1)
The AEA (2011) has also issued a Public Statement on Cultural Competence in Evaluation. This statement challenges evaluators to deepen their self-awareness and sensitivity in terms of their own cultures and those of others and to acquire the necessary skills to bridge the cultural gaps between themselves and those in the evaluation context. Cultural competence in evaluation is a reflective activity requiring evaluators to achieve and maintain a high degree of self-awareness and self-examination to better understand how their own cultural backgrounds and life experiences can serve as either assets or limitations in the conduct of an evaluation. Specific examples of ethical practice related to cultural competence offered by the AEA in its Public Statement on Cultural Competence include the following:
Use approaches that are appropriate to the context; for example, verbal consent can be used in communities with oral traditions, high levels of concern about privacy, or low levels of literacy.
Engage issues of culture directly, respectfully, and fairly when collecting data, making interpretations, and forming value judgments.
Incorporate ways to make findings accessible to all stakeholders, including forms of communication beyond written texts and the use of languages other than English.
Consider unintended consequences when reporting findings; for example, in some cultural contexts, participants in evaluations who are proud of their accomplishments may want to forgo anonymity and have their names attached to their stories. While this may be appropriate in some instances, in other situations the identification of participants may infringe on the rights of people who have not given informed consent.
Reflect and Discuss: Self-Exploration
Consider diverse cultural experiences that have led you to further “self” exploration. What did you learn about yourself that you were not consciously aware of prior to this self-exploration?
After reviewing the AEA’s (2011) Public Statement on Cultural Competence (available at www.eval.org/ccstatement) in full, discuss how exploring the “self” can improve one’s work as an evaluator.
In culturally incongruent settings, ethical issues can arise relative to the evaluator’s respect (or lack thereof) for local customs, values, and belief systems. Three types of ethics are particularly relevant in the evaluation of programs serving nonmajority and culturally diverse populations. These include procedural ethics, situational ethics, and relational ethics (Ellis, 2007; Guillemin & Gillam, 2004). Procedural ethics involve those mandated usually by institutional review boards to ensure that the study’s procedures adequately address the ethical concerns of informed consent, confidentiality, right to privacy, freedom from deception, and protection of participants from harm.
While institutional review boards certainly have tremendous value and are entrusted with ensuring the ethical conduct of research, scholars vigorously pushing for use of antiracist methodologies often cite problems with traditional institutional review boards’ ethics review procedures based in positivism, which separates thought from action and subject from object and assumes that research can and should be value free (Chavez, Duran, Baker, Avila, & Wallerstein, 2003). Further, it has been argued that institutional review boards generally give emphasis to assessing risks to individuals without paying attention to risk to communities (Minkler, 2004), a condition that has ethical implications for evaluations focusing on marginalized communities. Institutional review boards and the protection of human participants are covered in more detail in Chapter 12.
Situational ethics, or ethics in practice as it is sometimes referred to, include concerns involving the day-to-day unpredictable, often subtle, yet ethically important periods that arise while conducting research and evaluations. Situational ethics often reveal vulnerability, and the evaluator must decide what to do and how far to probe the situation