Social Work Research Methods. Reginald O. York
It Is Evidence-Based Practice
For a practice to be evidence based, it must have been subjected to various studies that revealed that it was effective with regard to a given outcome. For example, cognitive–behavioral therapy has been subjected to many studies that revealed it was effective in the treatment of depression.
Many human service agencies advertise that their services are evidence based. The social worker in these agencies should inquire into the basis for this statement. Can it be found on a legitimate list of evidence-based practices (some may refer to this as empirically supported practices)? Is there a list of publications that provide the evidence?
There is a chapter in this book that deals with this theme. For now, let’s be aware of what it means to have evidence as support for a service decision. It means that the evidence in support of this practice is greater than the evidence not in support of it, to lead a reasonable social worker to conclude that it is evidence based.
Avoiding Errors in Decision Making
There are several common errors in decision making when we avoid the spirit of scientific inquiry and the methods that support it. Among these errors to be discussed here are (a) incomplete information, (b) illogical reasoning, and (c) personal investment. Sometimes we make decisions without all the information we need. At other times we fall into the trap of a common method of illogical reasoning (i.e., a rationale that is not logical). Finally, we sometimes come to an opinion based on what will satisfy our ego investment in a situation (e.g., what will make us look better or gain more power).
Incomplete Information
A popular form of incomplete information is the cherry-picking of information to prove a point. This refers to the search behavior of only reporting the facts that support a given position and intentionally suppressing known facts that refute it.
Suppose you reviewed 12 studies on the effectiveness of a given form of therapy in the reduction of postpartum depression and found that only 2 studies show effectiveness but the other 10 fail to show that clients were better off because of this form of therapy. Which one of these are you most likely to report to others?
1 Would you report only on the two studies that showed effectiveness and declare this form of therapy to be an evidence-based therapy for the treatment of postpartum depression?
2 Would you report on all studies and conclude that the evidence, taken as a whole, fails to support the classification of this form of therapy as evidence based in the treatment of postpartum depression?
3 Would you report on all studies and conclude that this shows that this form of therapy is evidence based in the treatment of postpartum depression?
4 Would you report only on the 10 studies that failed to reveal success and conclude there is no evidence that supports this form of therapy for postpartum depression?
The social worker, as a social scientist, would select Option 2 above. There were far more studies with negative findings than positive ones. Unfortunately, some social workers, not embracing the spirit of scientific inquiry, would select Option 1, especially if the social worker has a personal investment in this form of therapy. Perhaps this is a form of therapy this social worker has been suggesting, and this revelation will be embarrassing.
Personal Investment
The personal investment in a given outcome was illustrated in the previous sentence. If you have been advocating for a certain action, you do not want to find evidence that suggests that it is a bad idea. If a certain kind of evidence has the potential to lead to the loss of your job, you will not want to embrace it.
A good issue to understand with someone is whether the information would normally be good news to that person or the kind that would not. In other words, what does this person need to believe?
Illogical Reasoning
Illogical reasoning refers to arguments that are not logical. It is not logical, for example, for you to say that the reason I am grumpy today is because of what I will have for breakfast tomorrow. This is not logical because it is not possible for a cause of something (my grumpiness) to come after it. It must come before. If I am grumpy right after breakfast today, you might logically say that one reason for it might be my breakfast. But not what I am going to have for breakfast tomorrow.
One form of illogical reasoning is the false dichotomy—the claim that there are only two options in an argument when, in fact, there are more. Consider this conversation the author once had with a friend. The friend asked, “Do you believe in doing a lot of testing of elementary school students?” I replied, “Yes, I do.” The friend responded, “Don’t you believe in instilling a love of learning?” The tone of voice suggested that this friend did not think that you could believe in testing and also believe in instilling a love of learning. He was presenting only two options as being possible when there were more.
According to this false dichotomy, if you believe in testing you must not believe in instilling a love of learning. But there are four alternatives here, not just two. You could possibly (1) believe in testing and not believe in instilling a love of learning, (2) not believe in testing but believe in instilling a love of learning, (3) not believe in testing and also not believe in instilling a love of learning, and (4) believe in testing and also believe in instilling a love of learning. By the way, my position is the last one: I believe in both. I do not think that you fail to believe in instilling a love of learning if you believe in testing.
My friend had entered this conversation with a perception that entailed incomplete information. There were four alternatives rather than two. This is the false dichotomy, which is a false claim that there are only two options (yes/no) when, in fact, there are more options logically in the argument.
There are other forms of illogical reasoning and other ways we make mistakes in decision making when we avoid the spirit of scientific inquiry. The next section will begin our review of this spirit and show how it helps with practice decisions.
Science as a Way of Knowing
A social scientist is someone who applies the principles and methods of science to learn more about social phenomena. This might include being able to describe the students in a club with enough clarity to draw some basic conclusions about what the members are like. It might include examining whether there is a relationship between variables (e.g., Do people who regularly exercise have fewer illnesses than those who do not?). It might include the examination of whether a social program is effective (e.g., Is there a lower rate of absenteeism in schools that employ the strengths model of schooling than in schools that do not?). Science employs certain methods of inquiry that are different from other ways of searching for meaning. These methods are designed to reduce human error in observation.
What Do You Already Know?
In this section are a few situations that will illustrate various aspects of the use of science to inform decisions. You will review each and decide if there is a problem.
Let’s examine the following situation:
A group of researchers started their study process with the purpose of determining the extent to which clients treated for depression have shown improvement in their levels of depression. They collected scores on a depression scale for a group of clients in treatment for depression. They collected these scores both before and after a service was provided that was designed to reduce depression. These scores showed improvement from the first to the last measurement. They were subjected to statistical analysis and were found to be statistically significant. The researchers concluded that these clients had experienced improvement with regard to depression, but they were not clear about the population to whom these results could be generalized because these clients had not been selected on a random