Handbook of Web Surveys. Jelke Bethlehem
The possibility of obtaining server‐side and client‐side information allows for easily monitoring response burden in web surveys. This makes it possible to record how much time respondents need to complete the questionnaire. Analysis may show how the response burden is related to the response rate.
The use of short questionnaires reduces the response burden. It may help to split a large questionnaire into a number of small questionnaires. The administration of small questionnaires is possible at different moments in time. This does not increase the costs of the survey.
Web surveys are less intrusive, and they suffer less from social desirability effects.
Geographical boundaries are not a problem. Geography is not limiting web surveys in the same way as face‐to‐face interviews and mail and telephone surveys. Therefore, international target populations may be easily reached without special additional costs or time delays.
2.2.2.2 Disadvantages and Problems
A major problem of web‐based surveys is sample selection. For research applications, a random sample is desirable and often essential, and researchers may simply not have a comprehensive sampling frame of e‐mail addresses for people who drink fruit juices or go to church. Despite the huge growth of the Internet, there are still many people who do not have access to, or choose not to use, the Internet. There are also wide disparities in Internet access among ethnic, socioeconomic, and demographic groups. A sampling frame, including e‐mail addresses, of all members of the target population should be available to draw a random sample. In practice, this list is very rarely available. Therefore, large coverage problems arise, and this is the most relevant issue.
Sampling problems may particularly be an issue for general population surveys. For many specific populations, there are no problems. Examples are companies collecting customer satisfaction data, employers measuring job satisfaction, educators collecting course evaluations and conducting examinations, bloggers wanting to consult with their readers, and event organizers checking proposed attendance and meal and other preferences. While there is still a need for some caution, in terms of learning how to use the new technology with confidence, the use of web surveys has been growing rapidly and will clearly continue to grow. Inside the innovative contest web surveys present new methodological challenges, like the integration with other data sources.
A disappointing aspect of web surveys is that they do not contribute to solving the problem of decreasing response rates. It is widely recognized that they usually result in a rather low response rates. It should be noted that, despite low response rates, the use of server‐side and client‐side paradata can help to focus efforts on specific population that most need it.
2.2.3 AREAS OF APPLICATION
Web surveys may be used in any field of application provided that the elements in the population have Internet access and that they have some basic computer skills. In some cases, as described in detail in Chapter 12, a probability‐based sample from the general population is selected. Then some people without a computer may receive one (with Internet access), together with basic instructions for use. This solution has typically been adopted for general population web panels.
If the web data collection is possible for all potential respondents, a web survey can be a very useful data collection tool, combining low costs and high quality.
Unfortunately, often not all elements in the target population have Internet access, or computers with adequate processing power to process questionnaires, or sufficient computer literacy. This problem holds true for general population surveys as well as for many other possible target populations. Even if Internet penetration is growing, differences may exist between countries and between groups within populations. Large differences in computer equipment, screen settings, and technical literacy may have a substantial impact. Thus, to carry out a good web survey, a statistical sound approach is needed, which attempts to minimize a possible bias as much as possible. And if a bias cannot be avoided, there should be statistical techniques applied to correct for this bias afterward.
In practice, despite the methodological challenges, many surveys (especially commercial surveys) are conducted on the web without properly taking into consideration the impact on the reliability of the results arising from the lack of Internet coverage and/or lack of computer literacy. Such surveys are administered exclusively via the web and therefore reach only one part of the target population. If the survey is not blocked, the web survey is a mobile web survey and will reach the participants also through the mobile device. Nevertheless, only who is using Internet is contacted. When using web survey results, one should be aware of potential problems. Therefore, it is important to assess the quality of the web survey by analyzing the methodological description in the documentation.
EXAMPLE 2.3 Reliable web surveys
Certain surveys are not affected by the instrument bias a web survey may cause. When measuring job satisfaction among high‐tech workers, the bias will be minimal. Getting feedback from employees on a benefit package can have a slightly higher bias if not all employees have computer access. However, an attempt to determine what role of the United States should have been in the Libyan war of 2011 would probably produce highly biased estimates, because one would only obtain the opinions from computer‐literate people with Internet access. They will not be representative for the whole population.
Web surveys may be conducted for profiling purposes. Examples are member surveys, audience profiling, and donor profiles. Web surveys may also be used for data collection by asking people to provide information about themselves. Other interesting applications are socioeconomic research, planning support, and social behavior studies. A web survey can also be used for attitude polls, opinion polls, program evaluation, and community cultural planning surveys. Other possible topics are economic aspects and performances, as well as market trends and customer/employee satisfaction.
EXAMPLE 2.4 The Kauffman Firm Survey (KFS)
The Kauffman Firm Survey (KFS) is a panel study of new businesses founded in 2004. They are tracked over their early years of operation. The survey focuses on the nature of new business formation activities, characteristics of the strategy, offerings, employment patterns of new businesses, the nature of the financial and organizational arrangements of these businesses, and the characteristics of their founders.
The KFS created the panel by using a random sample from a Dun & Bradstreet (D&B) database list of new businesses started in 2004. The list contained in total about 250,000 businesses.
The KFS oversampled these businesses based on the intensity of research and development employment in the businesses' primary industries. The KFS sought to create a panel that included new businesses created by a person or team of people, purchases of existing businesses by a new ownership team, and purchases of franchises. To this end, the KFS excluded D&B records for businesses that were wholly owned subsidiaries of existing businesses, businesses inherited from someone else, and not‐for‐profit organizations. Previous research on new businesses showed also variability in how business founders perceive the operation of their starting businesses. Therefore, a series of questions was asked to business owners about indicators of business activity and whether these were conducted for the first time in the reference year (2004). These indicators were payment of state unemployment (UI) taxes, payment of Federal Insurance Contributions Act (FICA) taxes, presence of a legal status for the business, use of an Employer Identification Number (EIN), and use of Schedule C to report business income on a personal tax return.
A self‐administered web survey and computer‐assisted telephone interviewing (CATI) were used for data collection.
Over time several changes in the original sample of businesses occurred and went for different reasons out of the panel. Due to panel attrition the number of units is becoming slightly smaller each year. Since