Handbook of Web Surveys. Jelke Bethlehem
year 1895 marks a fundamental change. Populations had grown bigger and bigger. It was the period of industrialization. Centralized governments required more and more information. The time was ripe for sample surveys. The first ideas emerged around 1895. There was a lot of discussion between 1895 and 1934 about how to select samples: by means of probability sampling or some other sample selection technique.
By 1934, it was clear that only surveys based on probability sampling could provide reliable and accurate estimates. Such methods of data collection were accepted as a scientific. In the period from 1940s to the 1970s, most sample surveys were probability based. Questionnaires were on paper forms. They were completed in face‐to‐face, telephone, or mail.
Somewhere in the 1970s another significant development started. The fast development of microcomputers made it possible to introduce computer‐assisted interviewing (CAI). This made survey data collection faster, cheaper, and easier and increased data quality. It was time in which acronyms like CATI (computer‐assisted telephone interviewing) and CAPI (computer‐assisted personal interviewing) emerged.
The next major development was the creation of the Internet around 1982. When more and more persons and companies got access to the Internet, it became possible to use this network for survey data collection. The first Internet surveys were e‐mail surveys. In 1989 the World Wide Web was developed. This software allowed for friendly graphical user interfaces for Internet users. The first browsers emerged and the use of Internet exploded. In the middle of 1990s, the World Wide Web became widely available, and e‐mail surveys were increasingly replaced by web surveys.
Web surveys are attractive because they have a number of advantages. They allow for simple, fast, and cheap access to large groups of potential respondents. Not surprisingly, the number of conducted web surveys has increased rapidly over time. There are, however, also potential methodological problems. There are ample examples of web surveys not based on probability sampling. Therefore, generalization of survey results to the population is questionable. The interviewed may access the Internet using various types of devices. Thus, web surveys can be completed and received not only on personal computer (PC) or laptop; it is highly probable the survey to be received in the mobile phone. The so‐called mobile web surveys are fully part of web surveys. This implies some methodological problems to be considered, and further research on the impact of mobile is called for.
This chapter describes the historical developments that have led to the emergence of web surveys. As an illustration, Section 1.3 shows how these developments were implemented at Statistics Netherlands and led to new software for survey data collection.
1.2 Theory
1.2.1 THE EVERLASTING DEMAND FOR STATISTICAL INFORMATION
The history of data collection for statistics goes back in time for thousands of years. As far back as Babylonian era, a census of agriculture was carried out. This already took place shortly after the invention of the art of writing. The same thing happened in China. This empire counted its people to determine the revenues and the military strength of its provinces. There are also accounts of statistical overviews compiled by Egyptian rulers long before Christ. Rome regularly took censuses of people and of property. The collected data were used to establish the political status of citizens and to assess their military and tax obligations to the state.
Censuses were rare in the Middle Ages. The most famous one was the census of England taken by the order of William the Conqueror, King of England. The compilation of his Domesday Book started in the year 1086 AD. The book records a wealth of information about each manor and each village in the country. Collected information was about more than 13,000 places. More than 10,000 facts were recorded for each country.
To collect all this data, the country was divided into a number of regions. In each region, a group of commissioners was appointed from among the greater lords. Each county within a region was dealt with separately. Sessions were organized in each county town. The commissioners summoned all those required to appear before them. They had prepared a standard list of questions. For example, there were questions about the owner of the manor; the number of free man and slaves; the area of woodland, pasture, and meadow; the number of mills and fishponds, to the total value; and the prospects of getting more profit. The Domesday Book still exists, and many county data files are available on CD‐ROM and the Internet.
Another interesting example of the history of official statistics is in the Inca Empire that existed between 1000 and 1500 AD. Each Inca tribe had its own statistician, called the quipucamayoc. This man kept records of the number of people, the number of houses, the number of llamas, the number of marriages, and the number of young men that could be recruited for the army. All these facts recorded on quipus, a system of knots in colored ropes. A decimal system was used for this. At regular intervals, couriers brought the quipus to Cusco, the capital of the kingdom, where all regional statistics were compiled into national statistics. The system of quipucamayocs and quipus worked remarkably well. The system vanished with the fall of the empire.
An early census also took place in Canada in 1666. Jean Talon, the intendant of New France, ordered an official census of the colony to measure the increase in population since the founding of Quebec in 1608. Name, age, sex, marital status, and occupation were recorded for every person. It turned out there lived 3,215 people in New France.
The first censuses in Europe took place in the Nordic countries. The first census in Sweden–Finland took place in 1749. Not everyone welcomed the idea of a census. Particularly religious people believed that people should not be counted. They referred to the census ordered by King David in biblical times, which was interrupted by a terrible plague and never completed. Others said that a population count would reveal the strengths and weaknesses of a country to foreign enemies. Nevertheless, censuses took place in more and more countries. The first census in Denmark–Norway has been in 1769. In 1795, at the time of the Batavian Republic under Napoleon's influence, the first count of the population of the Netherlands took place. The new centralized administration wanted to gather quantitative information to devise a new system of electoral constituencies (see Den Dulk and Van Maarseveen, 1990).
In the period until the late 1880s, there were some applications of partial investigations. They were statistical inquiries in which only part of a complete human population has been interviewed. The way the persons were selected from the population was generally unclear and undocumented.
In the second half of the 19th century, so‐called monograph studies became popular. They were based on Quetelet's idea of the average man. According to Quetelet, many physical and moral data have a natural variability. This variability can be described by a normal distribution around a fixed, true value. He assumed the existence of something called the true value. Quetelet introduced the concept of average man (“l'homme moyenne”) as a person of which all characteristics were equal to the true value (see Quetelet, 2010, 2012).
The period of the 18th and 19th centuries is called the era of the Industrial Revolution, too. It led to important changes in society, science, and technology. Among many other things, urbanization started from industrialization and democratization. All these developments created new statistical demands. The foundations for many principles of modern statistics were laid. Several central statistical bureaus, statistical societies, conferences, and journals, were established soon after this period. First ideas about survey sampling emerged in the world of official statistics. If a starting year must be chosen, 1895 would be a good candidate. Anders Kiaer, the founder and first director of Statistics Norway, started in this year a fundamental discussion about the use of sampling methods. This discussion led to the development, acceptance, and application of sampling as a scientific method.
Anders Kiaer (1838–1919) was the founder and advocate of the survey method that is now widely applied in official statistics and social research. With the first publication of his ideas in 1895, he started the process that ended in the development of modern survey sampling theory and methods. This process is described in more detail in Bethlehem (2009).
There