Evaluation in Today’s World. Veronica G. Thomas
Principles and Standards for Evaluators and Evaluations
One of the best ways evaluators can avoid and resolve ethical dilemmas is to know both what their ethical obligations are and what resources are available to them. In the evaluation field, there are explicit principles and standards for guiding the ethical behavior of evaluators and achieving quality evaluations. This section summarizes and discusses two major sources of guidance for evaluators: the AEA’s (2018b) Evaluators’ Ethical Guiding Principles and the Program Evaluation Standards developed by the Joint Committee on Standards for Educational Evaluation (Yarbrough et al., 2011). Both of these sources were developed in the United States and represent the longest-standing professional principles and standards in the evaluation profession. Many other regions of the world have also developed their own statements of standards. Notable examples include the Canadian Evaluation Society’s Guidelines for Ethical Conduct, the African Evaluation Association’s African Evaluation Guidelines, and the Australasian Evaluation Society’s Code of Ethics (links to these guidelines are provided in the additional resources section at the end of the chapter). It is imperative that evaluators become intimately familiar with the Evaluators’ Ethical Guiding Principles and the Program Evaluation Standards to be better positioned to understand how they should respond in the evaluation context in order to produce the most ethical and highest-quality evaluations possible.
The Evaluators’ Ethical Guiding Principles
The AEA’s Evaluators’ Ethical Guiding Principles (referred to as the Guiding Principles for Evaluators until 2018) were first adopted in 1994 and have subsequently undergone multiple revisions. They are intended to proactively guide and inspire the ethical conduct of evaluators at all stages of the evaluation process. These guidelines build, implicitly and explicitly, upon the three principles (respect for people, beneficence, and justice) in the Belmont Report. The Evaluators’ Ethical Guiding Principles stress that it is the primary responsibility of the evaluator to initiate discussion and clarification of ethical matters with relevant parties to the evaluation. (See Appendix A for a full presentation of the Evaluators’ Ethical Guiding Principles.)
There are five major Evaluators’ Ethical Guiding Principles. Each of these ethical principles is accompanied by several directives or subprinciples to amplify the meaning of the overarching five principles and to provide guidance for their application. The five guiding principles, briefly described as follows, do not imply priority among them, but instead, priority will vary by situation and evaluator role.
Systematic inquiry: Evaluators conduct data-based inquires that are thorough, methodical, and contextually relevant. This principle focuses most directly on methodological decisions made during the evaluation, although it renders no judgments favoring some methodologies over others. There are six subprinciples under systematic inquiry.
Competence: Evaluators provide skilled professional services to stakeholders. The principle of competence focuses on issues of the evaluator’s education, experience, relevant expertise, cultural competence, and professional development. This guiding principle includes four subprinciples.
Integrity: Evaluators behave with honesty and transparency in order to ensure the integrity of the evaluation. Here, evaluators must cultivate openness and full disclosure with stakeholders throughout the entire evaluation process. There are seven subprinciples under integrity.
Respect for people: Evaluators honor the dignity, well-being, and self-worth of individuals and acknowledge the influence of culture within and across groups. At all times, evaluators must demonstrate respect in terms of their interactions with stakeholders (regarding ethnicity, class, gender, orientation, etc.), including not judging them; not discrediting them; ensuring that their views are faithfully recorded, as appropriate; and giving them due consideration in the evaluation process. This guiding principle includes four subprinciples related to the overarching respect for people principle.
Common good and equity: Evaluators strive to contribute to the common good and advancement of an equitable and just society. Prior to the August 2018 Evaluators’ Ethical Guiding Principles revision, this principle was labeled “responsibilities for general and public welfare.” Because the revised principle places more explicit focus on common good and equity, it was renamed as such. There are five subprinciples under common good and equity.
The five Evaluators’ Ethical Guiding Principles are not independent, but instead, they overlap in many ways. For example, being honest and transparent (integrity principle) overlaps with honoring the dignity, well-being, and worth of individuals (respect for people principle). Conversely, sometimes these principles will conflict, and so evaluators will have to choose among them. When this occurs, evaluators must use their own values and knowledge of the evaluation context to determine the appropriate course of action. The following case study involves the external evaluation of a health program and was developed in 2006–2007 by the AEA Ethics Committee Professional Development Task Force. The first author, Veronica Thomas, was a member of that task force, and this case has been used as part of a training package on the Evaluators’ Ethical Guiding Principles.
Case Study: Application of the Evaluators’ Ethical Guiding Principles
Read the following case example, keeping in mind the AEA’s (2018b) Evaluators’ Ethical Guiding Principles. Then, organize into small groups and discuss the case. Complete the worksheet and question at the end of the case.
Evaluation Context. The Health Care Collaborative program grew out of a multiyear effort funded in many sites by a national foundation. That initiative promoted local collaboration among health care providers and residents in poorly served or underserved neighborhoods. The Health Care Collaborative office uses trained residents as outreach health workers to raise health-issues awareness among residents and to give them options for accessing health care. Health care providers who are collaboration partners deliver a range of services to neighborhood residents. A local funding source supports the Health Care Collaborative, which has a program director, administrative staff, and a small network of outreach workers. The Health Care Collaborative Board of Directors consists of a small group of health care providers.
The Health Care Collaborative serves an economically challenged neighborhood in a small metropolitan area: Average income is one-third to one-half of its metro and national counterparts. The neighborhood is quite diverse along many dimensions, including age, household composition, sexual identity, education, religious preference, race, and ethnicity. The neighborhood has a large African American population, an increasing population of refugees from African and Eastern European nations during the past 20 years, and a rapidly growing Hispanic population in recent years.
Entry, Contracting, and Design. The Health Care Collaborative Board and local funders found that they needed more information than the program’s reporting system alone could provide about how program participants viewed the Health Care Collaborative, how the staff viewed the program and the neighborhood, and how the program met or did not meet identified service needs. The funder provided $20,000 for this purpose, and the Board established a one-year schedule for completing an evaluation. The funder and the program director approached a local faculty member, an evaluator who also teaches evaluation, to ask for a proposal. The faculty member has previously served on the Health Care Collaborative Board. Discussions with the funder, the program director, and some members of the Board identified key expectations and constraints.
The faculty evaluator proposed a multimethod approach for a formative evaluation. The design included surveys of participants (brief), program staff, and other health care provider partners. The surveys would include questions about racial and ethnic identity. Selected program participants would be asked to keep journals and to participate either in a focus group or in an observed service delivery for a small group. Three focus groups were proposed: one for senior citizens; another for adult, nonsenior males; and a third for adult, nonsenior females. The Health Care Collaborative focus group participants would be offered a $25 gift card for their time. The institutional review board’s approvals