Evaluation in Today’s World. Veronica G. Thomas
such as the American Evaluation Association (AEA), National Society of Professional Engineers (NSPE), American Psychological Association (APA), American Sociological Association (ASA), National Association of Social Workers (NASW), and American Nurses Association (ANA) have a publicly disclosed, aspirational set of ethical guidelines for professionals. While the guidelines may differ in content and detail across various organizations, all represent an aspirational set of principles, values, and beliefs that help define the organization and address four overarching issues: (a) respect others’ rights to act freely and make their own choices while protecting the rights of those who may be unable to fully protect themselves; (b) do no harm including both physical injury and psychological harm such as damage to an individual’s reputation, self-esteem, or emotional well-being; (c) act fairly by treating individuals equitably and without regard to race, gender, socioeconomic status, or other characteristics; and (d) help or benefit others through promoting the common good and interests of individuals and society.
In contrast to an organization’s code of conduct, which is a directional document focusing on compliance and rules that describe how its members should behave in specific situations (e.g., forbid sexual harassment or racial intimidation), ethical guidelines include broad aspirational values and principles intended to provide an organization’s members with a general idea of the types of decisions and behaviors (e.g., treat others with respect) that are acceptable and encouraged by the profession. Individuals must interpret the organization’s ethical principles and adapt them in practice. A discipline’s ethical statements are used to guide practitioners of that discipline in determining the right course of action in a situation. On the one hand, evaluation shares ethical challenges similar to those of other types of social and behavioral science research. For example, public scrutiny of evaluation, like that occurring in other types of research, has resulted in heightened attention to examining whose views are included or excluded, determining design choices, considering how findings are checked for accuracy, and focusing on how results are reported (Wolf, Turner, & Toms, 2009). On the other hand, evaluation can be challenged by a variety of ethical issues beyond those confronted in other social and behavioral science research. In a presentation on ethical land mines in program evaluation delivered at the 1997 American Educational Research Association (AERA) Annual Meeting in Chicago, Linda Mabry stated that
evaluation is the most ethically challenging of the approaches to research inquiry because it is the most likely to involve hidden agendas, vendettas, and serious professional and personal consequences to individuals. Because of this feature, evaluators need to exercise extraordinary circumspection before engaging in an evaluation study. (quoted in McDavid, Huse, & Hawthorn, 2013, p. 468)
In 1999, Mabry added that of all the methods of social science inquiry, evaluation occurs in the most intensely political milieu where the heaviest assaults to ethics are threatened. Evaluation, she contended, involves risks that are rare in research such as generating final reports that can lead to expansion of a program or to its reconfiguration, shrinkage, or termination. Furthermore, evaluators are “often flattered, coddled, granted selective access, indoctrinated, misinformed, disregarded, challenged, or discredited according to the interests and opportunities of clients, program personnel, or other stakeholders” (Mabry, 1999, p. 200). Barnett and Camfield (2016) point out that the wholesale adoption of research ethics may not provide the most appropriate solution for evaluation. Instead, they propose a different approach to evaluation ethics that addresses stakeholder relationships, helps rebalance the primary focus on the respondent, and focuses on the duties and responsibilities of evaluation to society more broadly.
Research ethics has a predominant focus on the researcher–research participant relationship. In evaluation, however, other equally important relationships must be considered from an ethical perspective. These include relationships with a host of other stakeholders such as program administrators, service providers, program participants, and community members. Evaluators may experience role conflict with certain stakeholders, such as program staff, funders, or program clients, that raises ethical dilemmas. Happy clients (e.g., program administrators) make for pleasant working conditions for the evaluator, and these stakeholders’ favorable comments can generate additional contracts for the evaluator that ultimately enhance both income and professional standing (Mabry, 1999). Thus, client or stakeholder appeasement creates positive bias, which raises not only a validity issue but also an ethics issue. Evaluators who choose to comfort or enrich themselves by providing reports that cheer more than inform may hinder clients’ access to the information promised, foreclose on opportunities for program improvement, warp managerial decisions, and yield preventable negative human consequences (Mabry, 1999). The following case study provides an example of a key stakeholder (i.e., the director) attempting to cajole an evaluator in a manner that could present an ethical dilemma depending on the evaluator’s action.
Case Study: Identifying Hidden Agendas and Ethical Land Mines
The following description was adapted from one of the Ethical Challenges found in the American Journal of Evaluation. Review the case and then respond to the questions at the end of the scenario.
The Case of the Sensitive Survey
The Health Services Center (HSC) at North Central Southeastern State University, a small public institution (popularly known as Where Are U?), has recently engaged you, an external consultant, to conduct an evaluation of its programs and services. Last year, the state legislature mandated that the various administrative units within public colleges and universities be systematically evaluated on a periodic basis, and actually went so far as to allocate funds for this purpose.
The HSC has been included in the “first wave” of offices to be reviewed at Where Are U?, and the HSC director is less than thrilled. He has occupied the director’s position for the past 10 years, and he is convinced that (1) several highly placed administrators at the university would like to replace him and (2) these administrators view the evaluation as an opportunity to build a case to support such a move. This afternoon you are meeting with the director to discuss a draft version of a survey that you have prepared, focusing on students’ experiences with and opinions of the HSC. One section of the survey contains an extensive list of items that describe both positive and negative encounters that the diverse student body might have had with the HSC, and asks respondents to indicate which ones they have personally experienced.
In previous evaluations, you have found the survey to be an extremely useful, efficient, and comprehensive method for examining stakeholders’ perceptions. The survey will also give students an opportunity to rate how important these various issues and experiences are to them in terms of influencing their overall opinion of the HSC.
It is noteworthy to mention that the survey represents just one of the data-gathering strategies that you intend to use in this project. The director is, to put it mildly, unhappy with certain aspects of the survey. He believes that the items focusing on negative experiences will turn the HSC into a “punching bag” (the director’s words) for disgruntled students with an “axe to grind,” and that it would only be fair to use such a list if comparable ones were employed in the evaluation of other departments on campus. You, of course, are not involved in the evaluation of these other departments.
The director also maintains that many of the negative survey items pertain to matters that are not fully under the control of the HSC (e.g., the hours when certain medical specialists are available at the HSC). As the discussion continues, you become increasingly convinced that the key issue here is not the methodology you have proposed for gathering data. Rather, it seems to be the director’s intense desire to declare certain domains of the HSC’s functioning as “off-limits” in this evaluation. Although you can understand why he might be motivated to take such a position, you also strongly believe that an adequate, professionally respectable evaluation of the HSC cannot omit consideration of the areas that the director wants you to avoid. Interestingly, during the entry/contracting stage of this project, the director had not voiced the “domain concerns” that appear to worry him so greatly now. Your conversation with the director is cut short when he is called out of the office due to an emergency.
What