Economic Evaluation in Education. Henry M. Levin
tasks: (1) calculating costs, (2) identifying effects or estimating benefits, and (3) combining the two to report an economic metric. Potentially, such studies might require resources equivalent to those needed for an impact evaluation. The main guiding principle is the proportionality principle—that is, the idea that the amount of effort required should correspond to the value of the knowledge acquired and how that knowledge influences the results of the analysis. Suffice it to say, we hope to establish in this book that the knowledge gained from most economic evaluations is unlikely to be trivial and that more, rather than fewer, such evaluations are worthwhile.
2.5. Conclusions
Before undertaking an economic evaluation, it is important to understand the research question in detail. This understanding should start with the formal identification and understanding of the problem and the theory of change that each educational intervention adverts to. It should include explicit consideration and selection of alternatives to be evaluated; once these alternatives are set, the economic evaluation will be directed to determining which alternative is the most cost-effective or efficient. The analyst must recognize which audiences will use the evidence. Cognizant of this audience, the analyst must determine the perspective. We propose that each economic evaluation should begin with the presumption that a social perspective should be considered as the first priority even as other perspectives—fiscal, private, or per agency—are valid and important. Next, the analyst must choose the appropriate mode of analysis; the more detailed the investigation of the research question, the easier this choice will be. Finally, the analyst should perform an appraisal as to whether an economic evaluation is worthwhile.
This process requires a substantial research commitment: Each item needs to be decided in tandem, and these decisions will significantly shape the research inquiry. However, once these decisions are made, it is possible to proceed to the next stage of the analysis—how to undertake cost analysis.
Discussion Questions
1 A nonprofit agency wishes to introduce a mentoring program for at-risk youth and wants to perform BC analysis. What are some of the important issues to decide before beginning any data collection?
2 What types of assistance would you need in doing an economic evaluation, and what types of expert could contribute to the activity?
3 Describe potential audiences for different types of cost analyses. What would be the purpose of cost analysis for each audience? What type of analysis would be appropriate for each audience and why?
4 Federal policy in the United States constrains the use of BC analysis to only “economically significant” federal regulations. How should the term economically significant be defined?
Exercises
1 For the following interventions, describe how you would determine whether formal cost analysis is worthwhile.The education department in a particular state has mandated that school districts implement a specific math curriculum, including textbooks, teacher training, computer software, and other materials.A team of second-grade teachers approached their principal with a proposal to spend 10 minutes each morning performing mindfulness exercises with the students to help reduce stress, increase focus, and increase learning throughout the day.A community college is considering expanding its online education program to offer a wider range of courses and possibly even full degree programs.
2 For each of the following situations, determine which types of analysis (CE, BC, CU, cost, or CF) are most appropriate:A school district wishes to increase the employability of students who terminate their formal education at high school graduation. Accordingly, it seeks an answer to the question of whether it should expand vocational educational offerings for students who are presently in the general education program.The school board wishes to accommodate budget cuts by reducing some of the elective course offerings in the high school. A reduction in the budget of $60,000 has been targeted.A university must decide if it is desirable to establish a new online degree program.The state legislature wishes to consider the introduction of tablet computers into every high school in the state. However, it is not clear that the school budget is adequate.A school district is seeking approaches to improve students’ writing skills. Proposed solutions include (a) having smaller class sizes with more stress on writing and more writing assignments, (b) hiring college students with excellent writing skills to assist teachers in grading writing assignments, and (c) developing special writing courses for students in addition to their regular English classes.A community college must reduce its course offerings in the next academic year to accommodate a dismal budgetary situation. The college offers more than 1,400 courses in some 38 departments and programs. Enough courses must be cut to achieve savings of $500,000.Both computer-assisted instruction and smaller class sizes are being discussed as ways to improve the mathematics competencies of youngsters in a particular school district. The administration wishes to ascertain which alternative is preferable.
3 Identify a potential problem associated with each of the following five situations. Describe at least two alternatives that might be considered for addressing each problem. Suggest the hypothetical primary and secondary audiences for the evaluations that would follow in each of the five cases. What types of analysis would seem appropriate for each?Student test scores at the high school level have been declining for the past 5 years.The physics department of a college is having little success in placing its graduates.A school district faces an anticipated budget deficit for the next year of $200,000.A university wishes to consider replacing its computer servers and data management system.Local residents are unhappy that there is no college in their town.
3 Cost Concepts
Objectives
1 Specify the concept of costs in terms of opportunity costs.
2 Define cost terms for educational interventions.
3 Relate costs to the theory of change.
4 Distinguish between cost information and budgetary data.
5 Provide motivation for performing cost analysis.
To most of us, the notion of cost is something that is both as obvious as the price of a good or service and as mysterious as the columns of data on an accounting statement or budget. In this chapter, we will introduce a concept of costs that will differ somewhat from both of these, and we will discuss in detail how an education analyst might think about costs. Any social intervention or program has both an outcome and a cost. The outcome refers to the result of the intervention. Outcomes of educational interventions include such common indicators as higher student achievement, fewer dropouts, improved attitudes, greater employability, and so on. But why are all interventions associated with costs, and what is meant by costs?
For an economist, understanding the costs of an educational program is a way of understanding how the program works in relation to its theory of change (Hummel-Rossi & Ashdown, 2002). For example, if we know that one of the costs of a new instructional program is the wages paid to a highly trained teacher, we would expect the effects of the program to be mediated through the teacher’s competence. As an even easier example, if a new curriculum intervention requires no extra teacher training and no new facilities but simply the substitution of the old textbook by a new textbook, the extra cost will be very low; consequently, we might not expect a very significant effect on academic outcomes (and little effect on teacher performance and probably zero effect on student behaviors). Put simply, the analyst needs to think about costs in conjunction with program implementation. Educational interventions can be implemented in various ways—both by design and inadvertently—and so the resources required for the intervention will vary accordingly.
In this chapter, we describe the key concept that economists use to think about costs—the idea of opportunity cost. We then define cost terms that apply to educational interventions and describe the components of a cost estimate. As a final contribution, we make an important distinction between cost information and budgetary data, emphasizing that budgets do not accurately represent costs.
A