An Introduction to Evaluation. Chris Fox

      1 What is Evaluation?

      Introduction

      Evaluation is a broad concept and one that is sometimes difficult to distinguish both from other types of research and from related practices such as monitoring, performance management and audit. We start this chapter by looking at the various definitions of evaluation, distinguishing those that concentrate on the purpose of evaluation, the methods used in evaluation, and the significance of judgements of value in evaluation. We consider each of these briefly before adopting a definition of evaluation based on judgements of value. This is a definition that emphasises the political dimension of evaluation – a key distinguishing feature that will help us differentiate it from other types of research activity.

      Evaluation covers a range of activity types. People commonly talk and write about formative and summative evaluation, process (implementation) and impact (outcome) evaluation, and economic evaluation and theory-led evaluation. All of these terms raise questions about the nature and role of evaluation which we will address at various points in the book. In this chapter we will look at types and models of evaluation before discussing some key trends that have shaped the world of evaluation over recent decades.

      The final section of this chapter looks briefly at the history of evaluation. It aims to introduce readers new to evaluation to some of the ‘big’ ideas that have helped shape the sector and are still debated today. In particular the flux between ‘scientific’ and ‘naturalistic’ approaches to evaluation is discussed. Many of the ideas raised in this chapter will be developed in detail in later chapters.

      Defining evaluation

      Mark et al. (2006) distinguish everyday informal evaluation (How good was breakfast at the restaurant? How did the meeting with the client go?) from systematic evaluation, which they define as:

      a social and politicized practice that nonetheless aspires to some position of impartiality or fairness, so that evaluation can contribute meaningfully to the well-being of people in that specific context and beyond. (Mark et al. 2006: 5–6)

      They identify three groups of evaluation definitions which concentrate on the purpose of evaluation, the methods used, and the importance of judgements of value in evaluation. We will consider each of these in turn.

      Defining evaluation according to purpose

      Mark et al. (2006) identify a group of definitions that concentrate on the purpose of evaluation, typically providing information for policymaking or programme improvement. Their example is Patton’s definition:

      Program evaluation is the systematic collection of information about the activities, characteristics, and outcomes of programs to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming. (Patton 1997: 23, emphasis added)

      A note on terminology: policy, programme or project?

      At this point it is useful to note that in the literature on evaluation we often come across references to projects, programmes and policies. Building on Eggers’ work, Palfrey et al. (2012) argue that it is necessary to distinguish between these three evaluation subjects because they might offer the evaluator different opportunities to contribute to decision making. They suggest that:

      In this book, while we accept that projects, programmes and policies present different evaluation opportunities and challenges, for the sake of brevity our default position will be to refer to ‘programme’ evaluation unless there is a need to distinguish one from another.

      Defining evaluation according to method

      Another group of evaluation definitions identified by Mark et al. (2006) outline evaluation in terms of methods. An example comes from Rossi et al.:

      Program evaluation is the use of social research methods to systematically investigate the effectiveness of social intervention programs in ways that are adapted to their political and organizational environments and are designed to inform social action to improve social conditions. (Rossi et al. 2004: 16, emphasis added)

      Defining evaluation in terms of methods can be potentially helpful in distinguishing it from similar practices such as monitoring, performance management, auditing and accreditation. However, in turn this raises questions about what distinguishes evaluation from research (Palfrey et al. 2012).

      Evaluation distinguished from monitoring, performance management, audit and accreditation

      Monitoring, performance indicators (PIs) and broader performance management process have proliferated across the public, private and, increasingly, the not-for-profit sectors in the last few decades. We can link this proliferation to the development of ‘New Public Management’. Often associated with reforms to the public sector that were introduced during the administrations of Prime Minister Margaret Thatcher in the UK and President Ronald Reagan in the US, New Public Management (NPM) involved structural changes to the public sector and the introduction of business methods into government (Hill and Hupe 2014), as well as practices such as shrinking the size of the state so that government reduced