Experimental Design and Statistical Analysis for Pharmacology and the Biomedical Sciences. Paul J. Mitchell
it be a figure (scatter graph, bar chart, etc) or table, largely determines every single step in the preceding experimental design, including the strategy of your statistical analysis.
Figure 1.1 shows the final output of an experiment which examined the effect of pre‐treatment with mesulergine (an antagonist at 5‐HT2C receptors) on the ability of m‐chlorophenylpiperazine (mCPP; a 5‐HT2C receptor agonist) to reduce the locomotor activity of rats.
Figure 1.1 The effect of mesulergine on mCPP‐induced hypolocomotion. Vertical bars indicate mean locomotor activity counts ± Standard Error of the Mean. Saline‐pre‐treated animals were subcutaneously treated with either saline (open bar) or mCPP (vertical lines), while mesulergine‐pre‐treated subjects received either saline (stippled bar) or mCPP (solid bar). Two‐way ANOVA revealed main effects of pre‐treatment [F(1,28) = 74.799] and challenge treatment [F(1,28) = 110.999] and an interaction between pre‐ and challenge treatments [F(1,28) = 76.095], p < 0.001 in all cases. Post hoc analysis (Tukey) revealed that saline + mCPP combination‐treated animals exhibited significantly lower levels of activity than the other three treatment combination groups (***, p < 0.001 in all cases). For all other pairwise comparisons, p > 0.05. Data on file.
The bar chart contains four bars, each corresponding to the treatment combination administered to the subject animals, which are aligned along the x‐axis and whose height equates to the calculated arithmetic mean value of the corresponding locomotor activity as indicated on the y‐axis. Below the figure is the legend which describes the contents of the bar chart. The legend is divided into three sections. The first part is the figure number and the title, and usually these are in bold type. The second part is first half of the legend text in normal type and is a summary of the axis parameters arising from the experimental protocol, together with a summary of the Descriptive Statistics used to produce the bars and the key to differentiate each bar in the figure. The last part of the legend is a summary of the Inferential Statistics and includes, in this example, the data arising from both the ANOVA model and post hoc tests used to analyse the data (including an explanation of any indicators in the Figure, for example, the stars, used to identify significant differences between data sets); Screenshots of the statistical analysis are provided at the end of Chapter 17. This final output of your experiment is the last in a series of steps that comprise the complete experimental design process, and just as if you were planning your holiday to Iceland (the island, not the frozen food store!) or a sunny Mediterranean beach, it is easy to identify these steps in reverse order. Thus:
The step immediately prior to producing such a summary of experimental data is the Inferential Statistical tests employed to analyse the data. In the example provided here, this would be the two‐way ANOVA test followed by a suitable the post hoc test (here the Tukey test was used); why these tests were deemed appropriate will be explained later (see Chapter 17). The statistical test employed, however, is determined by the type and number of data sets produced by the experiment, but may include tests of data distribution and skewness, pairwise comparisons, other models of ANOVA, etc.
The step immediately prior to the Inferential Statistical analysis is the calculation of the Descriptive Statistics. These are the calculated summary values used to describe the data which are subsequently used to generate the data in the figure (e.g. bar height, etc.). Most experimental data are generally summarised by a measure of central tendency, such as the Mean (of which there are three types – but more about that later), together with the Standard Deviation or Standard Error of the Mean, Median (together with the range or semi‐quartile ranges), or Mode. However, note here that the measure of central tendency you report must be appropriate to the data your experiment has generated (see Chapters 5, 8, and 9).
The step prior to the statistical procedures is the input of your experimental data into your favourite statistical package. All statistical packages differ slightly from each other, but the most common method is to use a data spreadsheet similar to that seen with Microsoft Excel (see Figure 1.2).Figure 1.2 Excel spreadsheet showing original rodent locomotor activity data examining the effect of mesulergine on mCPP‐induced hypolocomotion (see Figure 1.1). The functionality of spreadsheets such as Microsoft Excel allows the calculation of simple Descriptive Statistics such as Mean, Standard Deviation, Standard Error of the Mean. Data on file.
Of course, you must generate your data before you are able to input such data into the spreadsheet and to achieve this you must decide on your experimental methodology – the process which generates a series of values which eventually allows you to draw conclusions about your experiment.
Before you decide on your methodology, however, you must have a working hypothesis which, in turn, is the result of your
experimental aims that address the
problem you have identified and is the raison d'etre of the whole experimental design process.
If we reverse these stages, then we have a list of events that summarise the experimental design process;
Experimental design process
1 What is the problem?
2 What is the aim?
3 Hypothesis
4 Experimental methodology
5 Data collection
6 Data input
7 Descriptive Statistical data
8 Inferential Statistical data
9 Final output
Notice that the Descriptive and Inferential Statistical steps (steps 7 and 8) are integral to the overall experimental design process. It is absolutely vital, therefore, that you decide on your statistical approach, both how you are going to convey (i.e. your summary figure or table) and analyse your data (Descriptive Statistics and Inferential Statistics, respectively) before you perform your experiment! It is fundamentally unacceptable to complete your experiment, obtain your data, and then scratch your head wondering what statistical strategy you need to adopt to analyse your data; all this demonstrates is very poor experimental technique. (At this point refer back to Example 1 in the Foreword.)
Statistical analysis: why are statistical tests required? The eye‐ball test!
But why do we need statistics? Surely life would be so much easier, less complicated, and with reduced levels of anxiety if we just did not bother. However, as scientists we have a duty to communicate the results of our studies to the wider world, and so we need to have confidence in the data generated by our studies. Consequently, we must provide evidence that supports the conclusions derived from our studies, and that support is provided by the statistical analysis of our data – but we should not rely purely upon our statistical analysis. The statistical data generated by our analysis is there to support our observations and not to replace what we can