Handbook of Web Surveys. Jelke Bethlehem
their studies at the different steps of the survey process. It is also useful in discussing the errors that can occur in different steps. The chapter provides an introduction to the framework and its structure and discusses the relevance of bearing in mind the framework and the survey steps when considering web survey errors. It then goes on to describe the concept of the step and the structure of the flowchart, breaking down the web survey process into six main steps. These are analyzed in detail, and an overview of survey errors is provided.
Chapter 4 examines the aspects of sampling. It is stressed that valid population inference is possible only if some form of probability sampling is used and that a proper sampling frame is required for this. A number of sampling designs and estimation procedures useful for web surveys are discussed.
A researcher conducting a survey may encounter a number of practical problems, and Chapter 5 provides an overview of possible errors, with two types of error examined in further detail. The first concerns errors in measurement. These can be caused by specific issues in questionnaire design, as well as a number of other aspects such as technology, incorrect unit definition, and so on. The second type of error regards nonresponse. This is a phenomenon that can affect all surveys, but the specific aspects of nonresponse in web surveys require particular attention. The chapter provides advice on relationships between errors and information on the various types.
A web survey is just one form of data collection. There are others, such as face‐to‐face, telephone, mail, and mobile surveys. Chapter 6 compares these various methods with online data collection, discussing the advantages and disadvantages of each one.
As web surveys do not involve interviewers, the respondents complete the questionnaire on their own. Furthermore, when questionnaires are sent out, they may very well be received and even completed on a mobile device (such as smartphones, which are very widespread). This means that questionnaire design is of crucial importance. Questionnaires must be adapted in order to be suitable for mobile devices; otherwise they cannot be used for this purpose. Small imperfections in the questionnaire may have serious consequences in terms of data quality. Questionnaire design issues are addressed in Chapter 7.
Chapter 8 examines strategies for data collection with adaptive/responsive survey design. In this case, strategies are not defined in advance, but instead are adapted, if necessary, during fieldwork. These designs may contribute to countering growing problems of nonresponse. This chapter was written by Annamaria Bianchi and Barry Schouten, who applied their particular expertise in this field to the subject examined.
A web survey may not always be the best solution for providing reliable and accurate statistics, with quality being affected by problems of under‐coverage and low response rates. An interesting alternative is to set up a mixed‐mode survey, in which several data collection methods are combined either sequentially or concurrently. This approach is less expensive than a single‐mode interviewer‐assisted survey (conducted either face to face or by telephone) and solves under‐coverage problems, but at the same time it poses other difficulties, known as mode effects, with one of the most significant of these being measurement error. Mixing modes is also of critical importance, as is the fact that in practice, a web survey is always mobile, unless questionnaire access via mobile device is restricted. All these aspects, as well as others concerning mixed‐mode surveys, are discussed in Chapter 9.
Chapter 10 is devoted to the problem of under‐coverage. This remains an important problem in many countries due to poor Internet coverage and the fact that Internet access is often unevenly distributed throughout the population. The chapter demonstrates how this can lead to survey estimates being biased. A number of techniques that may reduce under‐coverage bias are discussed.
Chapter 11 examines self‐selection. The correct and scientifically well‐founded principle is to use probability sampling in order to select survey subjects and therefore allow reliable estimates regarding population characteristics to be calculated. Nowadays, it is easy to set up a web survey. Even those without any survey knowledge or experience can create one through dedicated websites. Many of the resulting web surveys do not apply probability sampling, but instead rely on self‐selection of respondents. This causes serious problems with estimation. Self‐selection and its consequences in terms of survey results are discussed in this chapter, demonstrating that correction techniques are not always effective, and there are many reasons why web‐survey‐based estimates are biased.
Nonresponse, under‐coverage, and self‐selection are typical examples, and adjustment weighting is often applied in surveys in order to reduce such biases. Chapter 12 describes various weighting techniques, such as post‐stratification, generalized regression estimation and raking ratio estimation. The effectiveness of these techniques in reducing bias caused by under‐coverage or self‐selection is examined.
Chapter 13 introduces the concept of response probabilities, describing how they can be estimated through response propensities. If estimated accurately, response probabilities can be used to correct biased estimates. Here, two general approaches are described: response propensity weighting and response propensity stratification. The first attempts to adjust the original selection probabilities, while the second is a form of post‐stratification.
Chapter 14 is devoted to web panels. There are many such panels, particularly in the field of commercial market research. One crucial aspect is how the panel members (households, individuals, companies, and shops) are recruited. This can be carried out via a proper probability sample, or through self‐selection. There are consequences for the validity of the results of the specific surveys conducted with the panel members. The chapter discusses several quality indicators.
The accompanying website, www.web‐survey‐handbook.com, provides the survey data set for the general population survey (GPS), which has been used for many examples and applications in the book. The data set is available in SPSS (SPSS Corporation, Chicago, IL) format.
Silvia Biffignandi
Jelke Bethlehem
The editors acknowledge the contributions of:
Lon Hofman (Manager Blaise, Statistics Netherlands) and Mark Pierzchala (owner of MMP Survey Services, Rockville, USA) who wrote Section 1.3.1.
Annamaria Bianchi (University of Bergamo) and Barry Schouten (Statistics Netherlands) who wrote Chapter 8.
Chapter One The Road to Web Surveys
1.1 Introduction
Web surveys are a next step in the evolution process of survey data collection. Collecting data for compiling statistical overviews is already very old, almost as old as mankind. All through history, rulers of countries used statistics to take informed decisions. However, new developments in society always have had their impact on the way the data were collected for these statistics.
For a long period, until the year 1895, statistical data collection was based on complete enumeration of populations. The censuses were mostly conducted to establish the size of the population, to determine tax obligations of the people, and to measure the military strength of the country. The idea of sampling had not emerged yet.
The