Evaluation in Today’s World. Veronica G. Thomas
and Ellett (1980, p. 211) stated that “evaluation, more than any science, is what people say it is; and people currently are saying it is many different things.” Even today, we contend that if you ask a sociologist what program evaluation is, you will likely get a very different answer, focusing on different things, than if you ask an economist; and their answers both likely differ from how an epidemiologist, educator, or psychologist defines program evaluation. Our definition of evaluation has elements of many of the various definitions of evaluation provided in Chapter 1 but also includes distinctive features:
Evaluation is a disciplined inquiry involving the systematic, contextually responsive, and ethical application of research tools and methods to collect data that assess the effectiveness and operations of programs within the various social, political, and cultural contexts in which they operate. Evaluation’s ultimate goal is to provide credible evidence that fosters greater understanding and improves decision making, all aimed at improving social conditions and promoting healthy, just, and equitable communities.
Because multiple disciplines contributed to evaluation as currently understood and practiced, unifying its historical roots is not an easy task. The history of evaluation, like histories of many topics, differs depending on who is telling the story and the disciplinary background of the writers (Mertens & Wilson, 2019). A common misconception in writing a history chapter is that it simply involves finding out what happened and then documenting those events in chronological order. Undoubtedly, documenting notable evaluation-related events in temporal sequence can be useful because knowing the order in which events occur can help us understand potential causes (or influences) and effects of those events. This, in turn, allows us to step back and view the “big picture” of evaluation history. In this chapter, we seek to present a more inclusive history of evaluation than is generally presented in other evaluation books and to weave together a story of consequential events in the growth of evaluation.
History of Evaluation Through a Social Justice Lens
Published accounts of evaluation’s historical roots and evolution are relatively scant. The few existing histories were written in the 1990s or early 2000s, and they often covered narrow aspects of evaluation’s evolution such as theories or 20th-century developments. Shadish and Luellen (2005) wrote a brief history of evaluation in the Encyclopedia of Evaluation. Probably the most comprehensive and frequently cited historical overview is Madaus and Stufflebeam’s (2000) chapter characterizing evaluation over seven periods from 1792 to 2001. Stufflebeam and Coryn (2014) extended this historical overview to also include accounts of events from 2005 to 2014. Accounts of how evaluation has continued to evolve during the 21st century are still quite limited.
Since evaluations, in some form, have taken place over the many centuries in various arenas, chronicling the full history of evaluation in a single chapter is virtually impossible. There exist far too many events, theoretical and methodological perspectives, influential figures, and evolving trends to cover. In this chapter, we seek to provide a historical account of evaluation that infuses a social justice lens within our synthesis by situating evaluation within the political, sociocultural, and racialized contexts of the times. Existing published historical accounts virtually ignore the early contributions of persons who were engaged in evaluative efforts to advance social justice agendas. Hopson and Hood (2005, p. 88) argue that “the rich intellectual history of the evaluation profession may be improvised as a result of an overemphasis by those who some have anointed as ‘fathers’ (or ‘mothers’, in recent history) of the field.” We seek to redress this by providing a historical account that is more inclusive of important, but often ignored, events and people who utilized evaluation as a tool to ameliorate prominent social inequities of the time.
Evaluation Prior to Modern Times of the 20th Century
If one thinks of evaluation simply in terms of assessing the value of something, then there is clearly no single moment in time that we can pinpoint when individuals began to use evaluative judgments to draw conclusions. During ancient times, individuals were faced with making determinations (i.e., evaluations) about the best course of action for securing the essentials of life (i.e., food, clean water) and avoiding and treating diseases. In this sense, as Scriven (1996, p. 395) stated, “evaluation is a very old practice yet a very young discipline.”
Evaluations of people, programs, and events have a long history that has been loosely documented. While certainly not exhaustive, Table 3.1 highlights notable evaluation activities taking place in various settings and across different fields prior to the modern times of the 20th century. Some of these events are elaborated in more detail in the text that follows.
Table 3.1
Mosteller (1981) argues that the earliest known evaluation is described in the Old Testament book of Daniel. It spoke of what can be essentially viewed as a nutrition evaluation study based on what Daniel (Belteshazzar), Hananiah (Shadrach), Mishael (Meshach), and Azariah (Abednego) ate, in comparison to the diet of other males, during their time in Babylon as captives. According to the Old Testament, Daniel said to the guard over him and others: “Please test your servants for ten days: Give us nothing but vegetables to eat and water to drink. Then compare our appearance with that of the young men who eat the royal food, and treat your servants in accordance with what you see.” The guard agreed, and at the end of this trial period, Daniel and his mates looked healthier and better nourished than any of the young men who ate the royal food. While clearly not meeting scientific standards, this is still loosely an example of an evaluation study during ancient times.
Intersection Between Education and Evaluation Pre–20th Century
Evaluation, in large part, as practiced today evolved from pre-20th-century work in two areas of education: student assessment and measurement and the accreditation of schools and colleges. Many contemporary evaluation methods and procedures, such as the use of comparison groups, standardized testing, and surveys, are built on systematic strategies and techniques that were applied in the 1600s in the field of education.
The first evidence of formal educational program evaluation recognized in the United States took place between 1897 and 1898 when Joseph Mayer Rice evaluated the spelling performance of 33,000 students in relation to their spelling instruction (Madaus & Stufflebeam, 2000). Because of this groundbreaking work, Rice is recognized as a pioneer of educational research and the originator of comparative methodology in the field. Further, Rice’s work remains critical in the history of evaluation because it began a long tradition of using achievement test scores, or standardized testing, as a key indicator for judging the effectiveness of a school or instructional program (Madaus & Stufflebeam, 2000; Morra-Imas & Rist, 2009).
Educational accreditation in this country represents another major and long-standing influencer on educational evaluations (Kellaghan, Stufflebeam, & Wingate, 2003). The first regional accrediting agencies formed in the 1880s with particular focus on educational standards and admissions procedures. Accreditation of educational institutions was conceived of as a quality assurance process whereby educational institutions, programs, services, and operations are evaluated and verified by an external body to determine if recognized standards are met. It became (and continues to be) a means of conducting nongovernmental peer evaluations of educational institutions and programs.
Early Social Experiments
As Rossi, Lipsey, and Freeman (2004, p. 2) pointed out, one of the technically challenging forms of contemporary evaluation research, “social experiments,” is not hardly a recent invention. Probably one of the most-cited areas of evaluation taking place in the 1600s includes the experiments with scurvy (a disease resulting from lack of vitamin C) (Mosteller, 1981; Rossi et al., 2004). In the early 1600s, British captain admiral James Lancaster, in an effort to address the incidence of scurvy among crews on ships, ran an evaluation experiment by serving daily dosages of lemon juice to men