Handbook of Web Surveys. Jelke Bethlehem
the advantages and disadvantages, areas of application, and specific related problems.
2.2 Theory
Collecting data using a web survey has much in common with other modes of data collection. There are the usual steps, like survey design, fieldwork, data processing, analysis, and publication. At each step, however, consider the suitability of concepts and methods taken from traditional survey approaches (face‐to‐face, paper, telephone). Where necessary, implication for eventual questionnaire reception and completion on mobile devices needs to be borne in mind, and adequate adaptions need to be applied. This handbook examines the most important practical and methodological aspects of only web surveys and of mobile web surveys that need careful consideration. They relate on the following questions:
How to select the sample?
How to contact potential respondents?
How to construct a web questionnaire?
How mobile web surveys differ or are similar to web surveys?
How to make proper statistical inference based on a web survey data?
What is the impact of sampling and non‐sampling errors?
What are the problems and solutions for mixed‐mode surveys with web component?
How to handle web panels?
Detailed answers to these questions are in the other chapters of this handbook. The current chapter gives a general overview of different approaches of conducting web surveys. For each approach and each situation, different problems may occur, and therefore different methodological solutions are required.
2.2.1 TYPICAL SURVEY SITUATIONS
In this section, identification of typical situations, in which to conduct a web survey, is presented. A number of different key aspects lead to different survey situations:
Target population. There are general population surveys (among individuals or households), business surveys, specific population surveys (among specific populations like company employees, company customers, students at a university or school, or members of club), or open population surveys (among ill‐defined populations like consumers of a product or service).
Survey administrator. This can be a national statistical institute (NSI) or other official statistical government body, a commercial market research company, a university, or another research institute.
Cross‐sectional versus longitudinal data collection. A cross‐sectional web survey measures the status of a population at one specific point in time, based on a sample selected for that purpose. A longitudinal web survey (or web panel) is recruited; it is maintained to allow measuring change over time. Also, surveys on specific topics can be selected from the web panel.
Technical implementation. The questionnaire can be designed as a website on the World Wide Web. In this case, questionnaires are completed online. It is also possible to use the Internet as just a vehicle to transport a questionnaire form to the respondents. For example, a form in an Excel spreadsheet can be send as an attachment of an e‐mail. In this case, the questionnaire is completed offline. This approach was used at early stages of Internet diffusion. Now, the questionnaire is mostly, even not exclusively, completed online. The challenge is now the choice of the device used for reception and completion of the questionnaire and/or on the choice of the mode, if a mixed‐mode approach is adopted.
The choice of the target population for a web survey may have an impact on the magnitude of survey errors and particularly on non‐sampling errors. Such errors may partly be caused by selection problems (for example, under‐coverage) and partly by measurement errors (due to the lack of interviewer assistance).
If the target population is the general population (households or individuals), there is a problem with the sampling frame. Some countries have a population register. Such a register contains addresses. Therefore, use it as a sampling frame for a face‐to‐face or mail survey. Sometimes telephone numbers link to addresses, which makes it possible to use it as a sampling frame for telephone surveys. Unfortunately, these registers do not contain e‐mail addresses, nor can e‐mail addresses link to it.
Internet penetration varies greatly between countries (see Chapters 1 and 10 about under‐coverage problems). Presently, Internet coverage is relatively high, say, between 60% and 90% in a number of European countries. These coverage rates seem to suggest that general population web surveys are possible in these countries and that they can compete with traditional data collection modes. However, note that a large Internet penetration does not imply high Internet use. Moreover, it also does not imply high quality if fast Internet connections are available. For example, not everyone with Internet access has broadband.
One should always bear in mind that not everyone has access to the Internet. One example is that not every employee of a company is allowed to use the Internet. Moreover, Internet access is substantially lower in many countries for specific subpopulations. For example, Hispanic blacks are underrepresented in the United States. Another example of underrepresented groups are people with low education and people living in rural areas. This situation exists in many countries. The elderly, also, are often underrepresented among Internet users. Under‐coverage leads to web surveys that lack of representativity. Therefore, there is a risk to draw wrong conclusions from the survey results.
If the target population consists of businesses, it is quite probable, in most countries, that each business has Internet access and therefore has an e‐mail address. Thus, the collection of business to sample for a web survey is rather close to the target population. However, obtaining a complete list of e‐mail addresses for businesses may be a very difficult task. Partial lists sometimes exist, but complete lists are often lacking. NSIs regularly contact large enterprises for surveys. Therefore, they may have a complete list of e‐mail addresses of certain economic branches or specific size classes of companies. In most cases, obtaining an e‐mail population list for small enterprises and businesses could well be a difficult task for NSIs. Even if such a list is available, it requires a lot of effort to maintain it. In European countries, the maintenance of the business register—requested from Eurostat—allows for the updated list of businesses and their stratification variables and address (e‐mail address, in most cases). NSIs are now going to run many surveys via web. Even census data collection is on the web, thus sharing many methodological problems with the web surveys.
If the target population is a closed population (employees of a company or students of university), often there is a sampling frame containing e‐mail addresses of all members of the population. In such situations, there is no difference between the target population and the sampling frame. There are no coverage problems. This is the ideal case for a web survey.
With respect to the survey administrator, differences may occur with respect to the amount of information for setting up the surveys and the topics that are addressed in the web survey. NSIs and official statistics bodies probably have the largest amount of information available on the general population (households and individuals) and businesses (or institutions). They may have access to population registers, they may have census data, and they may manage demographic databases, the business register, and other sources of information. Therefore, although this huge amount of data may be insufficient for generating a sampling frame of e‐mail addresses for the target population, they may well be in a rather good position to obtain this information. This is happening in the near future, since the trend in the diffusion of web surveys is gaining importance and the Internet penetration within the population continues to increase. Currently, the advantages of the NSIs are twofold:
They often have a sampling frame for the general population of individuals of households based on addresses. This means that they are able to select a suitable probability based sample from