Economic Evaluation in Education. Henry M. Levin
early childhood program Head Start has been found to influence parental time: Head Start children receive more help from their parents at home (Gelber & Isen, 2013; for a similar family-mediated effect with the Chicago Child-Parent Center program, see Reynolds, Ou, & Topitzes, 2004). These family resources mediate the effect of Head Start on longer-term outcomes. From a social perspective, these family resources are a cost.
For many interventions, therefore, it is important to estimate these induced costs and distinguish them from the costs of the intervention. In fact, these costs might be larger than the cost of delivering the intervention. Nevertheless, if the theory of change is clearly specified, these induced resources will be an explicit part of how the intervention is expected to work.
Another cost that is often neglected is raising funds to pay for a program. The ingredients might include the “deadweight loss” associated with raising funds to support the program (Haveman & Weimer, 2015). The deadweight loss is the distortion in other markets. For example, a city may wish to implement a new public school program in a city funded through a local sales tax. This tax will distort consumption decisions away from the taxed goods, leading to a deadweight loss or marginal excess tax burden (METB). Estimates of the METB for the United States are approximately 13% (Allgood & Snow, 1998)—that is, to generate $100 in funds for a program imposes a distortion of $13. Other countries may have larger METBs. If every public program imposes a deadweight loss, then the incremental effect may be very small. However, the size of the loss depends on how the tax is levied and at what level of government. It is important to check that the necessary funding for a given education program does not impose large distortions.
One other cost that is often ignored is that of reallocation of resources from one program to another. Often, school personnel assume that there is no cost because the shifts came from inside the school. But there is an opportunity cost if the shifted resources were producing valued outcomes in the previous program—that is, the cost is the value of the resources in their previous use, which is likely to be nonzero.
3.4. Costs Data and Budgetary Information
A very common question often arises: Why should we go to all this trouble to estimate costs? Almost all social programs have budgets and expenditure statements, which presumably contain expenditure data that can be used to address the cost issues. However, there are many reasons why budgetary information is inadequate for cost analysis.
First, although the existence of budgets is universal, the assumption that they will contain all the cost information that is needed is usually erroneous. First, budgets often do not include cost information on all the resources used in the intervention. Contributed resources such as volunteers, donated equipment and services, and other “unpaid” inputs are not included in budgets.
Second, when resources have already been paid for or are included in some other agency’s budget, they will not be easily observed. For example, a building that is provided to a school district by some other unit of government or one that is fully paid for will not be found in the budget of a school district. Many education interventions receive ingredients from multiple levels. For example, Reading Partners, a reading program coordinated through a national office, operates through reading centers within schools and utilizes resources from AmeriCorps, the national community service organization, as well as relying on volunteers (Jacob, Armstrong, Bowden, & Pan, 2016). To accurately estimate costs, all resources used to implement Reading Partners should be calculated.
Third, the standard budget practices may distort the true costs of an ingredient. Typical public budgets and expenditure statements charge the cost of major repairs or facility reconstruction only to the year in which the cost was incurred. Thus, when the roof or heating system of a school is replaced, the expenditures are found in the budget for the year in which the repairs were financed—typically the year in which the reconstruction was done rather than being amortized properly. Yet, a new roof or heating system may have a 30-year life so that only about one-thirtieth of it should be charged to the cost of programs in any given year. Budgetary conventions would typically charge the costs of such capital investments to a single budgetary year, overstating the true costs for that year and understating the costs of operating the program for the 29 subsequent years.
Fourth, the costs of any particular intervention are often embedded in a budget or expenditure statement that covers a much larger unit of operation. Therefore, it may be difficult to isolate the unique costs of a new reading program in a school district budget, since the budget is not constructed according to the costs of particular interventions or activities. In fact, most educational budgets are “line item” classifications of expenditures according to functions and objects. Examples of functions include administration, instruction, and maintenance. Examples of objects include teachers, supplies, clericals, and administrators. It is not only difficult to tie such budget listings to particular activities or interventions but also often impossible even to ascertain what the costs are for a given school or broad instructional program such as a language program, since no such breakdowns are usually provided.
Finally, most budgetary documents represent plans for how resources will be allocated rather than classifying expenditures after they have taken place. This means that, at best, they refer to planned disbursements rather than actual ones. Accordingly, beyond all their other limitations for cost analysis, budgets may not provide precise figures for actual resource use. Actual statements of expenditures may be more accurate, but they are still subject to the shortcomings mentioned before.
Finally, budgets rarely help elucidate how a program is implemented—or at least not in a way that is related to the theory of change or the counterfactual. This point is illustrated in the cost analysis of Read 180 by Levin, Catlin, and Olson (2007). Using the ingredients method, the researchers calculated program costs at three selected sites where Read 180 was being delivered; they also calculated the program costs based on the developer’s recommended version of Read 180 from interviews and documents. Given that Read 180 is a reasonably formulaic reading intervention, we might anticipate that the actual costs at the three sites would be close to the recommended cost. The developer’s recommended version of Read 180 required changes in both personnel and materials. However, as implemented, some sites relied on adding more personnel than others while others yet varied in materials. The result was a significant difference in average costs for each site.
For these reasons, cost analysis cannot place primary reliance on budgetary or expenditure documents to ascertain the costs of interventions. Of course, these documents may still provide supplementary data that will be very useful. However, they cannot serve as a principal source for constructing cost estimates.
3.5. Motivation for Cost Analysis
Understanding the costs of an intervention is of course a prerequisite for performing CE and BC analyses. In itself, cost analysis is also valuable. It allows us to perform a cost study and a cost feasibility analysis (we provide examples of these in subsequent chapters). Even before that, however, cost analysis helps the analyst learn about the intervention in two key respects. These are the main motives for performing a cost analysis per se.
First, cost analysis is a way of describing an intervention. Many educational interventions are complex or multifaceted; by providing details on the resources, we are explaining what is required for the intervention. Information on costs can indicate the quality or intensity of a program. For example, the number of counselors per student may indicate the quality of counseling programs, as would data on the number of days of training required for counselors before the program begins operation. As well, the cost per student for a program will reflect the dosage of the treatment each student receives. For example, the annual cost of a mentoring program might be $500, but the cost per student will depend on how many years each student participates.
Information on costs sheds light on the types of resources needed to implement a given program. Some interventions may require a substantial commitment of in-kind resources; these include volunteer-intensive youth supports such as Big Brothers Big Sisters of America and elementary school literacy programs such as Time to Read (Grossman & Tierney, 1998; Miller & Connolly, 2012). Other interventions may require significant