Damned Lies and Statistics. Joel Best
(for example, criminologists believe that the vast majority of homicides are recorded, simply because dead bodies usually come to police attention).
So, when reporters or officials ask activists about the size of a newly created social problem, the activists usually have to guess about the problem’s dark figure. They offer estimates, educated guesses, guesstimates, ballpark figures, or stabs in the dark. When Nightline’s Ted Koppel asked Mitch Snyder, a leading activist for the homeless in the early 1980s, for the source of the estimate that there were two to three million homeless persons, Snyder explained: “Everybody demanded it. Everybody said we want a number…. We got on the phone, we made a lot of calls, we talked to a lot of people, and we said, ‘Okay, here are some numbers.’ They have no meaning, no value.”2 Because activists sincerely believe that the new problem is big and important, and because they suspect that there is a very large dark figure of unreported or unrecorded cases, the activists’ estimates tend to be high, to err on the side of exaggeration. Their guesses are far more likely to overestimate than underestimate a problem’s size. (Activists also favor round numbers. It is remarkable how often their estimates peg the frequency of some social problem at one [or two or more] million cases per year.3)
Being little more than guesses—and probably guesses that are too high—usually will not discredit activists’ estimates. After all, the media ask activists for estimates precisely because they can’t find more accurate statistics. Reporters want to report facts, activists’ numbers look like facts, and it may be difficult, even impossible to find other numbers, so the media tend to report the activists’ figures. (Scott Adams, the cartoonist who draws Dilbert, explains the process: “Reporters are faced with the daily choice of painstakingly researching stories or writing whatever people tell them. Both approaches pay the same.”4)
Once a number appears in one news report, that report is a potential source for everyone who becomes interested in the social problem; officials, experts, activists, and other reporters routinely repeat figures that appear in press reports. The number takes on a life of its own, and it goes through “number laundering.”5 Its origins as someone’s best guess are now forgotten and, through repetition, it comes to be treated as a straightforward fact—accurate and authoritative. Soon the trail becomes muddy. People lose track of the estimate’s original source, but they assume the number must be correct because it appears everywhere—in news reports, politicians’ speeches, articles in scholarly journals and law reviews, and so on. Over time, as people repeat the number, they may begin to change its meaning, to embellish the statistic.
Consider early estimates for the crime of stalking.6 Concern about stalking spread very rapidly in the early 1990s; the media publicized the problem, and most state legislatures passed anti-stalking laws. At that time, no official agencies were keeping track of stalking cases, and no studies of the extent of stalking had been done, so there was no way anyone could know how often stalking occurred. After a newsmagazine story reported “researchers suggest that up to 200,000 people exhibit a stalker’s traits,”7 other news reports picked up the “suggested” figure and confidently repeated that there were 200,000 people being stalked. Soon, the media began to improve the statistic. The host of a television talk show declared, “There are an estimated 200,000 stalkers in the United States, and those are only the ones that we have track of.”8 An article in Cosmopolitan warned: “Some two hundred thousand people in the U.S. pursue the famous. No one knows how many people stalk the rest of us, but the figure is probably higher.”9 Thus, the original guess became a foundation for other, even bigger guesses (chapter 3 explores how repeating statistics often alters their meaning).10
People who create or repeat a statistic often feel they have a stake in defending the number. When someone disputes an estimate and offers a very different (often lower) figure, people may rush to defend the original estimate and attack the new number and anyone who dares to use it. For example, after activists estimated that there were three million homeless in the early 1980s and the Reagan administration countered that the actual number was closer to 300,000, the activists argued that the administration’s figures could not be trusted: after all, the administration was committed to reducing expenditures on social programs and could be expected to minimize the need for additional social services.11 Various social scientists set out to measure the size of the homeless population. When their findings confirmed that the 300,000 figure was more reasonable, the social scientists came under attack from activists who charged that the research had to be flawed, that the researchers’ sympathies must have been with the administration, not the homeless.12 In general, the press continued reporting the large estimates. After all, activists and reporters knew that the actual number of homeless persons was much higher—didn’t everyone agree that three million was the correct figure? This example suggests that any estimate can be defended by challenging the motives of anyone who disputes the figure.
In addition, the dark figure often plays a prominent part in defending guesses. There are always some hidden, unnoticed, uncounted cases and, because they are uncounted, we cannot know just how many there are. Arguing that the dark figure is large, perhaps very large (“The cases we know about are just the tip of the iceberg!”), makes any estimate seem possible, even reasonable. We know that some victims do not report rapes, but what proportion of rapes goes unreported? Is it two in three? Surveys that ask people whether they’ve been victimized by a crime and, if so, whether they reported the crime to the police, find that about two-thirds of all rapes go unreported.13 But surely these surveys are imperfect; some rape victims undoubtedly refuse to tell the interviewer they’ve been victimized, so there still must be a dark figure. Some antirape activists argue that the dark figure of unreported rapes is very large, that only one rape in ten gets reported (this would mean that, for every two victims who fail to report their attacks to the police but tell an interviewer about the crimes, seven others refuse to confide in the interviewer).14 Such arguments make an impassioned defense of any guess possible.
Activists are by no means the only people who make statistical guesses. It is difficult to count users of illicit drugs (who of course try to conceal their drug use), but government agencies charged with enforcing drug laws face demands for such statistics. Many of the numbers they present—estimates for the number of addicts, the amounts addicts steal, the volume of illicit drugs produced in different countries, and so on—cannot bear close inspection. They are basically guesses and, because having a big drug problem makes the agencies’ work seem more important, the officials’ guesses tend to exaggerate the problem’s size.15 It makes little difference whether those promoting social problems are activists or officials: when it is difficult to measure a social problem accurately, guessing offers a solution; and there usually are advantages to guessing high.
There is nothing terribly wrong with guessing what the size of a social problem might be. Often we can’t know the true extent of a problem. Making an educated guess—and making it clear that it’s just someone’s best guess—gives us a starting point. The real trouble begins when people begin treating the guess as a fact, repeating the figure, forgetting how it came into being, embellishing it, developing an emotional stake in its promotion and survival, and attacking those who dare to question what was, remember, originally just someone’s best guess. Unfortunately, this process occurs all too often when social problems first come to public attention, because at that stage, a guess may be all anyone has got.
DEFINING
Any attempt to talk about a social problem has to involve some sort of definition, some answer to the question: “What is the nature of this problem?” The definition can be—and often is—vague; sometimes it is little more than an example. For instance, a television news story may tell us about a particular child who was beaten to death, and then say, “This is an example of child abuse.” The example takes the place of a precise definition of the problem. One difficulty with this practice is that media coverage usually features dramatic, especially disturbing examples because they make the story more compelling. Using the worst case to characterize a social problem encourages us to view that case as typical and to think about the problem in extreme terms. This distorts our understanding