Social Work Research Methods. Reginald O. York
you will encounter in both explanatory and evaluative research is whether your data can be explained by chance. Statistical significance refers to the likelihood that a given set of data would occur by chance. You will see much on this theme later. If your data can be explained by chance, you cannot logically conclude that the data are meaningful. For example, if you find that your clients’ posttest scores for anxiety are better than their pretest scores but you find that the data failed to be statistically significant, you cannot logically declare that you found that the clients gained with regard to reduced anxiety. You cannot do this because you found data that can be explained by chance, so the next time you do the same study, you are just as likely to find that they did not gain as to find that they did. Your data must do better than being the result of chance to be taken seriously.
Step 4: Drawing Conclusions
Based on your data findings, you draw conclusions about your research question. Is this the place for you to offer your opinions about the research question? No! Your conclusions should stick to the data you analyzed. It does not matter how strong your opinion is or how much experience you have had with it. The research study is about the collection and analysis of data in a certain manner using certain measures of certain variables. Your conclusions must adhere to the data analyzed. However, you may offer suggestions regarding future research or your opinion about the strengths and limitations of your research methods. But your opinions on the research question of your study should be kept apart from the study conclusions.
To summarize this presentation on the process of social work research, let’s take an example from evaluative research. The target behavior is depression because you have a group of 11 clients who have entered your special program for the treatment of depression. The research question is whether these 11 clients will have lower depression at the end of the treatment than before. Your method of measurement is the Beck Depression Inventory, a highly tested tool for measuring depression. Your intervention is cognitive–behavioral therapy, which has more positive evidence for the treatment of depression than any other treatment method. The outcome is that the mean score for these 11 clients at the end of the treatment is 40% higher than their scores before the treatment began. These differences between the scores before and after treatment were found to be statistically significant. This means that the gain cannot be easily explained by chance and, therefore, can be taken seriously. You draw the conclusion that cognitive–behavioral therapy was effective in the reduction of depression for this group of 11 clients. However, you are not in a position to generalize these findings to depressed people who were not in your study, because you did not employ a random sample. Instead, you used a convenience sample. You used this type of sample because your primary concern was to find out whether your treatment was effective with this group of clients.
The research process is illustrated in Figure 2.1. In this presentation, a group of at-risk middle school students were randomly divided into two subgroups: one subgroup would get tutoring during the first grading period and the other subgroup would get tutoring in the second grading period. To test the effectiveness of this tutoring intervention, the grades of these two groups would be compared for the first grading period. It would be expected, of course, that the first subgroup (who had the tutoring) would have higher grades during this first grading period than the second subgroup (who did not have tutoring in this grading period). In this example, the tutored group had grades that were 34% higher than the nontutored group.
Figure 2.1 ■ The Research Process
Chapter Practice Exercises
For this chapter, there are two practice exercises. In the first exercise, you will examine a selected aspect of program evaluation for a familiar human service agency. The second exercise calls on you to provide a brief report on certain aspects of an evaluative study you might undertake in a familiar agency.
Practice Exercise 1: Evaluating a Human Service Program
Here are a few parts of the service system that could serve as the focus of one part of a program evaluation. You should select one of these questions and see what you can find in the information available to you at your agency for this program.
1 This program should be serving people in the target population. Important characteristics of this population might include poverty, having young children, being pregnant, being at risk for delinquency, facing death or dying, having marital problems, and so forth.How would you characterize the target population?Does this agency seem to be serving this population? Do you have any data that confirm it? Do you have informed opinions of staff that would support this conclusion?
2 Services should be accessible to those in need. In other words, prospective clients should be able to take advantage of what your agency offers. For example, employed persons may have a difficult time going to your agency before 5:00 p.m. If so, does your agency have service hours after 5:00 p.m.? Certain persons may have transportation problems. Does your agency facilitate transportation in any way?
3 Clients should be appropriately screened at intake to ensure that services are being used by those most in need. What does your agency do to ensure this? Is there any evidence available to you that would suggest that this is true?
4 The program should be documenting outcomes for the program (e.g., higher grades in school, lower depression, higher employment, fewer acts of delinquency, improved marital relations, etc.). Does the agency collect data to evaluate client outcomes?
5 Clients should be appropriately terminated from a service when the need has been met. What does the agency do to ensure that this happens?
6 Services should have an impact on indicators of need, such as rate of child abuse, rate of delinquency, and so forth. Does the agency have data on this?
Competencies Demonstrated by Practice Exercise 1
1 The ability to distinguish between a human service program and a human service intervention
2 The ability to collect information on one important program evaluation question for a given human service program
3 The ability to identify what a given human service agency needs to do to improve its program evaluation information
Activities for Practice Exercise 1
1 Select a human service organization and a human service program within that agency for your analysis.
2 Select one of the six statements given above to serve as the focus of your report.
3 Investigate what the selected agency does with regard to your question about program evaluation.
4 Prepare a report on what you found regarding your selected question for your selected program.
What You Will Report From Practice Exercise 1
1 The name of your agency and the name of your human service programExamples:Parker Middle School is the agency, and school social work is the program.The Harper County Department of Human Services is the agency, and child protective services is the program.Oak Hill Behavioral Health is the agency, and adult therapy services is the program.
2 A brief description of your selected program with regard to (a) basic goals and (b) servicesExamples:The social work services program at Parker Middle School has the goal of improving the ability of at-risk students to perform in school at the level of their abilities. It provides a school social worker who works with individual at-risk students and their families in a variety of ways.The child protective services program of Harper County Department of Human Services has the goal of preventing child abuse and neglect. It provides an array of social work services including investigation