Impact Evaluation of Transport Interventions. David A. Raitzer

       Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO)

      © 2019 Asian Development Bank

      6 ADB Avenue, Mandaluyong City, 1550 Metro Manila, Philippines

      Tel +63 2 632 4444; Fax +63 2 636 2444

       www.adb.org

      Some rights reserved. Published in 2019.

      ISBN 978-92-9261-586-4 (print), 978-92-9261-587-1 (electronic)

      Publication Stock No. TCS190095-2

      DOI: http://dx.doi.org/10.22617/TCS190095-2

      The views expressed in this publication are those of the authors and do not necessarily reflect the views and policies of the Asian Development Bank (ADB) or its Board of Governors or the governments they represent.

      ADB does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. The mention of specific companies or products of manufacturers does not imply that they are endorsed or recommended by ADB in preference to others of a similar nature that are not mentioned.

      By making any designation of or reference to a particular territory or geographic area, or by using the term “country” in this document, ADB does not intend to make any judgments as to the legal or other status of any territory or area.

      This work is available under the Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO) https://creativecommons.org/licenses/by/3.0/igo/. By using the content of this publication, you agree to be bound by the terms of this license. For attribution, translations, adaptations, and permissions, please read the provisions and terms of use at https://www.adb.org/terms-use#openaccess.

      This CC license does not apply to non-ADB copyright materials in this publication. If the material is attributed to another source, please contact the copyright owner or publisher of that source for permission to reproduce it. ADB cannot be held liable for any claims that arise as a result of your use of the material.

      Please contact [email protected] if you have questions or comments with respect to content, or if you wish to obtain copyright permission for your intended use that does not fall within these terms, or for permission to use the ADB logo.

      Corrigenda to ADB publications may be found at http://www.adb.org/publications/corrigenda.

      Notes:

      In this publication, “$” refers to United States dollars.

      ADB recognizes “China” as the People’s Republic of China and “Vietnam” as Viet Nam.

      On the cover: Impact evaluation needs to consider the effects of transport sector interventions on various sectors (photos by ADB).

      Cover design by Joe Mark Ganaban.

       Foreword

      Strategy 2030 of the Asian Development Bank emphasizes the creation of knowledge from investment operations and the use of evidence from past projects to design new interventions. The Strategy also seeks a more proactive role for research to help to replicate good project practices across Asia and the Pacific region. For these ambitions to be fulfilled, evidence on the intended and unintended effects of interventions needs to be accumulated. Impact evaluation is a central means for generating evidence.

      To build a broader body of evidence on “what works” in development, impact evaluation needs to be mainstreamed across a range of development investments. In recent years, impact evaluation coverage has made substantial progress in the health and education sectors. Although sectors dominated by “hard infrastructure,” such as transport, account for far more development investment, the number of impact evaluations on these sectors has remained limited. Moreover, transport investments have become very complex and more oriented toward making transport sustainable and inclusive. Along with these innovations has come an increasing array of behavioral assumptions underpinning interventions, which impact evaluations can help to test.

      At the same time, transport sector interventions have special challenges for impact evaluation. Many transport investments, such as highways or mass transit, are “small-n” interventions, which affect large geographies, so that there are insufficient numbers of treated and untreated units for enabling conventional statistical analyses. Transport costs condition where households and firms chose to be located, so that exposure to transport interventions can be endogenous to household and firm characteristics over long time periods. Capturing spillover effects can also be a challenge, as changes to transport costs can be transmitted across large transit networks.

      This review is intended to help offer insights to orient future impact evaluations on the transport sector amidst these challenges. It attempts to characterize theories of change associated with many transport interventions, entry points for impact evaluation, impact evaluation challenges, and potential impact evaluation designs. It then reviews the impact evaluation literature to date, in terms of methods applied, interventions covered, and outcomes evaluated, and identifies gaps for future studies to address. In the process, it offers practical examples that might inspire future studies.

      The literature survey finds a rapidly growing body of impact evaluation studies related to transport. Of the 91 studies identified, more than 65% were published after 2012. Impact evaluation designs have also become diversified, with expanding use of more robust methods, such as regression discontinuity design, as well as greater use of “big data.” Evidence has been offered on a range of effects of transport interventions including income, poverty, agricultural production, air quality, employment, education, health, and much more.

      Yet, many evidence gaps remain. Existing impact evaluation effort has mostly pertained to road development, of which rural roads are most frequently studied, and its effects on socioeconomic outcomes. There is much less attention to urban transport, and there is limited literature on non-road interventions. Efficiency and behavior change interventions central to the “avoid–shift–improve” approach, which seeks to reduce the need for travel, shift travel toward public transit, and reduce traffic bottlenecks, have also received less attention. Nor have effects via intermediate services, such as health or educational facilities, been often evaluated, when “corridor” approaches increasingly seek to link other investments with transport.

      With “big data” increasingly available, it is hoped that the creative examples identified in this review can help to inspire new impact evaluation studies that address evidence gaps on transport. This work is more relevant than ever, as transport investments increasingly seek to experiment with high level technologies, such as intelligent transport systems, and increasingly focus on behavior change interventions, such as transport demand management, pricing, driving restrictions, and road safety measures. Impact evaluation can play a critical role to ensure that these innovations can be tested to help transport operations improve over time. This review helps to show where future impact evaluation investments can best fill evidence gaps and the methods that may