Evaluation in Today’s World. Veronica G. Thomas
must always be concerned with actual and potential conflicts of interest and must deal with them openly and honestly so that they do not compromise the evaluation process and results. Conflicts of interest in evaluation are inevitable, and they emerge in and affect all groups of stakeholders, including evaluators (Yarbrough et al., 2011).
A conflict of interest, in particular, occurs when two or more competing or contradictory interests relate to an activity by an individual or an institution. In evaluations, conflicts of interest include situations in which financial or other personal considerations may compromise, or have the appearance of compromising, an evaluator’s judgment in conducting the evaluation and/or reporting the findings. When evaluators are caught in conflicts of interest, biases often distort findings (House, 2016). Conflicts of interest are distinct from bias, inasmuch as conflicts of interest occur when evaluators’ judgment concerning their primary interest (i.e., the production of valid and useful evaluations) is clearly influenced by some secondary and competing interest. Here, conflicts of interest exist regardless of whether the evaluator’s judgment and behavior can be demonstrated to have adversely influenced the evaluation. Instead, the conflict exists simply as a condition of the evaluator having competing interests (Tobin, 2003).
Currently, there are two major conflict of interest areas generally considered in the evaluation field that can negatively impact the ethical integrity of the evaluator and the resulting evaluation. These include financial conflicts of interest (e.g., monetary arrangements with sponsors) and professional conflicts of interest (e.g., personal friendships, professional relationships). In evaluation, conflicts of interest extend beyond simple personal or financial interest; they can also occur when different individuals or groups try to influence when evaluations are commissioned, which purposes and questions are addressed, who can serve as evaluators or evaluation staff, when data are collected, which methods are used, who can provide or later access information, who has primary access to findings, and how findings are interpreted (Yarbrough et al., 2011). Examples of conflicts of interest in evaluation include
any personal benefit the evaluator (or the evaluator’s spouse, child, etc.) might gain in a direct or predictable way from the developments of the program or projects the evaluator is reviewing or asked to review in the future;
any previous involvement the evaluator has had with the program or projects he or she has been asked to review, such as serving on the advisory board or having an undisclosed relationship with the program administrators or staff; and
financial interest held by the evaluator (or the evaluator’s spouse, child, etc.) that could be affected by his or her evaluation.
In addition to financial and professional conflicts of interest, we propose a third conflict of interest area; that is, cultural conflict of interest is about evaluator power, status, knowledge of, and identification with the goals and values of the dominant culture and how these factors become a secondary yet competing interest, with the primary interest of the individuals and communities under consideration. This secondary interest might impact the evaluator’s questions, methods, measures, definitions of success, and interpretations. Evaluators, particularly those from the dominant culture, are at risk of experiencing a cultural conflict of interest because they think the world is the way they see it or, if not, it ought to be that way. The evaluators may be well intentioned but still demonstrate cultural arrogance or lack of respect. Here, they need to work to educate themselves about relevant cultures in the evaluation context. A cultural conflict of interest may be unconscious and thus not recognized as a conflict by the evaluator. Such unrecognized conflicts can result in ignoring important perspectives and rendering some important (yet marginalized) stakeholders and communities as relatively invisible.
Ethical Challenges and Dilemmas Across the Evaluation Process
Scholars have researched and written extensively on ethics and evaluation (e.g., Barnett & Camfield, 2016; House, 2011; Mathison, 1999, 2007; Morris, 2008, 2015; Newman & Brown, 1996). Morris and colleagues (Morris, 2015; Morris & Cohen, 1993) identify common ethical challenges evaluators face at various stages of the evaluation. The following case study provides a list of commonly reported ethical challenges faced by evaluators across various phases of the evaluation process.
Case Study: Ethical Challenges Commonly Reported by Evaluators
Entry/Contracting Phase
A stakeholder has already concluded what the findings “should be” or plans to use the findings in an ethically questionable fashion (e.g., to support a decision previously made).
A conflict of interest exists.
The type of evaluation to be conducted is not adequately specified or justified.
A stakeholder declares certain research questions “off limits” in the evaluation despite their substantive relevance.
Legitimate stakeholders are omitted from the planning process.
Various stakeholders have conflicting expectations, purposes, or desires for the evaluation.
The evaluator has difficulty identifying key stakeholders.
Designing the Evaluation Phase
The evaluator fails to gain acceptance of the overall design from all relevant stakeholders.
The evaluator believes evaluation design is fundamentally flawed.
Insufficient time and resources are available to conduct a credible evaluation.
Data Collection Phase
The rights or dignity of those providing data are compromised in some fashion (e.g., violations of confidentiality, anonymity, informed consent).
The evaluator discovers behavior that is illegal, unethical, or dangerous while conducting the evaluation.
The evaluator discovers staff incompetence.
Data Analysis and Interpretation Phase
The evaluator fails to distinguish between findings and his or her opinions in data analysis.
Methodological choices highlight some findings while downplaying others of equal or greater importance.
Communication of Results Phase
The evaluator is pressured by a stakeholder to misrepresent findings.
The evaluator is pressured by a stakeholder to violate confidentiality.
Although not pressured to violate confidentiality, the evaluator is concerned that reporting certain findings could represent such a violation.
Utilization of Results Phase
Findings are suppressed or ignored by the stakeholder.
Disputes or uncertainties develop concerning ownership/distribution of the final report, raw data, and so on.
Findings are used to punish the evaluator or someone else.
Findings are deliberately modified by a stakeholder prior to release.
Findings are misinterpreted by a stakeholder.
Plagiarism/misrepresentation of authorship occurs.
Information gathered for one purpose is used for another.
Source: Morris, M. (2015). Research on evaluation ethics: reflections and an agenda. In Paul R. Brandon (Ed.), Research on evaluation. New Directions for Evaluation, 148, 31–42.