Social Work Research Methods. Reginald O. York
correct. The approach of the advocate for pseudoscience is to reverse the burden of proof and claim that the new approach should be considered correct until science clearly proves that it is not.
A good source on this topic is the book Science and Pseudoscience by Thyer and Pignotti (2015). You can see in this book a discussion of many treatment approaches that fall into the category of pseudoscience. For example, you will find information on Reiki assessment, thought field therapy, neurolinguistics programming, holding therapy for children, and militaristic boot camps for youth. There are many more. These are just a few examples.
If you see a model of practice that has met the criteria for being pseudoscience, you do not necessarily have evidence that this practice is effective or that it is not effective. Instead, you have information suggesting that there is a lack of evidence of its effectiveness. You also have information suggesting that the basis for the claim of success is not consistent with a scientific basis for decision making. It may be effective but without evidence to prove it. It may be ineffective. In fact, it may even be harmful. We will not know unless we have full evidence.
There have been treatments that have been found, through scientific evidence, to be harmful. An example is the Scared Straight approach to the prevention of delinquency. This program exposes at-risk youth to the perils of prison life by taking them to prison for the day and having them listen to the messages of the prisoners about how bad prison life is. The assumption of this program is that this exposure will scare these youth sufficiently to cause them to avoid a life of crime. The results, however, have shown that it makes things worse. Here is the plain language summary of a review of many studies of this program:
Programs such as “Scared Straight” involve organized visits to prison facilities by juvenile delinquents or children at risk for becoming delinquent. The programs are designed to deter participants from future offending by providing firsthand observations of prison life and interaction with adult inmates. This review, which is an update of one published in 2002, includes nine studies that involved 946 teenagers, almost all males. The studies were conducted in different parts of the USA and involved young people of different races whose average age ranged from 15 to 17 years. Results indicate that not only do these programs fail to deter crime, but they actually lead to more offending behavior. The intervention increases the odds of offending by between 1.6 to 1 and 1.7 to 1. Government officials permitting this program need to adopt rigorous evaluation efforts to ensure that they are not causing more harm to the very citizens they pledge to protect. (Petrosino, Turpin-Petrosino, Hollis-Peel, & Lavenberg, 2013)
In Figure 1.2, you can see a graphic depiction of how pseudoscience sometimes convinces people of the credibility of a practice that does not deserve it.
What should you do if you see a claim about a practice that might be based on pseudoscience? You should find evidence with regard to the approach that you see advocated. A quick review of literature databases can reveal if there is such evidence and what the evidence shows. If you fail to find evidence, this should not be interpreted to mean that a given service is not effective. But if there is no evidence, why should you embrace it in view of the fact there are likely many alternatives that have been subjected to scientific testing? And, of course, if there have been a lot of studies with consistent negative findings, you should see this as clear evidence that this is not a good approach to service.
Figure 1.2 ■ Making the Felonious Case for Scared Straight
Keep in mind that science relies on relevant facts, objectively reviewed, on a carefully articulated question. You can review evidence about a particular question through a review of literature databases that will show you articles that have examined the particular question scientifically.
Applying the Basic Principles of Science: Does the Full Moon Make Us Different?
Let’s employ our learning about these principles with regard to a research question. This is an example that was presented in another text (York, 1997). We will use one that is simple and may even be fun. Have you ever heard someone say “It must be the full moon” when they witness strange behavior? When this author asked whether the full moon affects the behavior of mental health clients, he often received an affirmative response from social workers and others. Many people are convinced that the full moon has such an effect.
Let’s suppose that we have decided to conduct a study to see if this proposition is true. We will go through the research process to examine it. We will examine the knowledge base currently available to decide where to go from there. If the question has already been well answered by existing research, we will conclude that another study is not necessary; so we will stop our inquiry there.
What is the purpose of our study? Which is the better way to state our purpose given the spirit of scientific inquiry?
1 To prove that the full moon causes unusual behavior
2 To demonstrate that there is more strange behavior during the full moon than when the moon is not full
3 To determine if there is more strange behavior during the full moon than when the moon is not full
4 To prove that the full moon is not related to unusual behavior
What is your choice? If you choose the first one or the last one, you will clearly find yourself outside the bounds of the spirit of scientific inquiry. Remember that science is a method of finding out rather than a method of proving a point. If you choose the second one, you will make the same mistake. If your purpose is to demonstrate that something is true, you are not speaking according to the spirit of scientific inquiry. So this leaves the third option, which is consistent with the spirit of scientific inquiry.
If we seek to prove a point, we will naturally fall into various traps that will hamper our pursuit of knowledge about the subject. Remember that research is a process of discovery, not justification. We should engage in a process of inquiry that is designed to provide an objective appraisal of our research subject. The reduction of the potential of human bias is key to accomplishing this. So our first principle of science is as follows: Scientific research is a process of finding out, not a process of justification.
Another principle that undergirds the scientific method is that you should not reinvent the wheel. This means that you should start your inquiry with a review of what is already known about your research question. In our review of the question about the full moon and strange behavior, we found that there have been a great number of studies undertaken.
So do we need to engage in another one? Maybe but only if the existing research has left some point of the debate uncovered. If you search enough, you will usually find a special aspect of your inquiry that has not yet been fully investigated. The main point here is not to discourage the continued pursuit of a theme but to find what is already known, so that you can couch your inquiry in a manner that is more likely to add to our existing knowledge base.
There have been numerous studies on this question. One such study was conducted on attempted suicides (Mathew, Lindsay, Shanmjganatan, & Eapen, 1991). The records of the Accident and Emergency Department of a large urban hospital were examined to determine if the rate of suicide attempts that came to the attention of this hospital was different during the full moon and at other times. The number of suicides attempted for each day of 1 month were recorded. The full moon fell on Day 15. On that day, a total of 19 suicide attempts were recorded. The largest number of suicide attempts (23) were recorded on Days 3 and 17, one of which was close to the full moon, but the other was at a great distance from it. The number of attempts on the day of the full moon was slightly higher than the average for the entire month, but the difference was determined to be nonsignificant (statistically). When we examine this question, we might also want to compare the 3 days when the moon was at its fullest with the 3 days when it was the least full. If the full moon causes suicide, we would expect the 3 days of the full moon to have a significantly higher suicide rate. For the data from this study, the average number of suicide attempts during the full