An Introduction to Evaluation. Chris Fox
expected to intrinsically possess ethical principles (see, for instance, Stake and Mabry 1998).
Ethical guidelines
There are many professional societies and associations for evaluators, some organised on national lines and some on subject or sector lines. Many of these have published guidelines on ethical evaluation.
For example, the American Evaluation Association’s Guiding Principles for Evaluators sets out five principles. The headline principles are as follows:
Systematic Inquiry Evaluators conduct systematic, data-based inquiries
Competence Evaluators provide a competent performance for stakeholders
Integrity/Honesty Evaluators display honesty and integrity in their own behaviour, and attempt to ensure the honesty and integrity of the entire evaluation process
Respect for People Evaluators respect the security, dignity and self-worth of respondents, programme participants, clients, and other evaluation stakeholders
Responsibilities for General and Public Welfare Evaluators articulate and take into account the diversity of general and public interests and values that may be related to the evaluation
Each headline is then elaborated on by the Association. It is beyond the scope of this book to survey the many guidelines that associations and societies have produced in detail. Simon (2006) suggests that published ethical guidelines vary on a number of dimensions, including:
how they distinguish between ethical issues and methodological issues (for example, is methodological competence regarded as an ethical issue?)
whether they address ethical issues for all stakeholders or solely those of professional evaluators
whether and how they acknowledge difference
Ethics committees
Evaluations will sometimes be scrutinised by an Ethics Committee. Ethics Committees have long operated in the field of medication research and evaluation but are increasingly common in other areas of social science. For example, it is common practice for universities to constitute ethics committees to scrutinise social science research and evaluation proposals.
Simons (2006) identifies five concerns about ethics committees. Firstly, while they provide scrutiny at the start of a research process they cannot anticipate all the ethical issues that might emerge during the research. Secondly, while they tend to adopt the principle of ‘respect for persons’ this cannot entail respect for every human action. In moral and ethical debate there are always ‘trade-offs’. Thirdly, because such committees often have their origins in medical research they tend to be more familiar with quantitative than qualitative research methods. Fourthly, such committees increasingly act as ‘guardians’ of research design, making judgements about what counts as research methodology. And finally, some ethics committees are concerned not only with ethical issues, but also with acting as research gatekeepers or defenders of institutional reputations, or with preventing litigation.
The guiding principles
As with research, evaluations pose a number of issues that relate more specifically to the way the ‘subjects’ of the evaluations – the stakeholders – are treated by the evaluator. The ethics in this case relate mostly to the stakeholders’ rights, which the evaluator must respect and uphold in the process of the evaluation. For instance, all stakeholders have the right to be informed about the purpose and objectives of the evaluation, and all the evaluation stakeholders who will be interviewed or surveyed have the right to confidentiality and to have their data protected from wider access.
Overall, these are often called the guiding (ethical) principles and these days it is a widely accepted fact that these principles must apply to all evaluations. Simons (2006), drawing on Newman and Brown (1996), distinguishes between:
ethical rules which are specific statements about ethical behaviour
ethical codes and standards which are compilations of ethical rules
ethical principles with are broader than rules and serve as the foundation for codes
In this section we set out some commonly accepted principles related to participants’ and stakeholders’ rights:
informed consent
voluntary participation
do no harm
confidentiality and anonymity
and guiding principles that should govern to evaluators’ inherent ethics and ethical behaviour:
professional integrity
openness and respect for diversity
cultural competency
These principles apply to all evaluations. However, as Simons notes:
[T]he choice of methodology and the procedures adopted in using methodology reflect a particular ethical and political stance which will affect how the evaluator resolves dilemmas. (Simons 2006: 257)
Particularly difficult is the question of whether the selection of evaluation methods and the quality standards adhered to in using those methods are themselves ethical questions. We touch on this issue again in Chapter 12 when we look at different research paradigms.
Principles relating to participants’ and stakeholders’ rights
Informed consent
Those who will be participating in the evaluation must be fully informed about the its purpose and must explicitly provide their consent for the outcomes of their participation to be used. By ‘participation’ it is meant those who will be interviewed, surveyed, invited to participate in scoping workshops, and so forth.
However, in some contexts it can be very difficult to draw the line between formal participation and informal participation. In some cases the evaluator will spend a good number of days in context, holding several conversations with different people who may not regard these as moments for data gathering.
Therefore, at the beginning of any evaluation, all stakeholders should be informed about the evaluation, about the team, and about the roles of those involved. Indeed good practice here is to provide all stakeholders with the original terms of reference, clearly state who is commissioning the evaluation, and inform participants of the planned objectives and future uses of the findings.
In most research projects, participants are required to sign a consent letter. In evaluations this is not always common practice although it should be. Those who manage evaluations should – by default – ask evaluators to collect the signed consent forms/letters and annex these to the final evaluation documentation. This is more important in evaluations where the legal risk is heightened.
Evaluators should also consider the need for ‘rolling consent’ particularly when either:
an emergent evaluation design is being used, making it difficult for consent given at the start to the evaluation process to fully anticipate what participation will later entail (Simons 2006), and/or
participant engagement will be sustained over time
Voluntary participation
Participants must willingly participate in the evaluations, namely in the workshops, interviews, focus groups, and all the other situations by which data and information are to be collected. The option not to participate should be made clear and available to them as an equally valid and respected option.