Applied Univariate, Bivariate, and Multivariate Statistics. Daniel J. Denis
a variance σ2 of 1.
z‐scores are useful for comparing raw scores emanating from different distributions. Standardization transforms raw scores to a common scale, allowing for comparison between scores.
Binomial distributions are useful in modeling experiments in which the outcome can be conceptualized as a “success” or “failure.” The outcome of the experiment must be binary in nature for the binomial distribution to apply.
The normal distribution can be used to approximate the binomial distribution. In this regard, we say that the limiting form of the binomial distribution is the normal distribution.
The bivariate normal density expresses the probability of the joint occurrence of two variables.
The multivariate normal density expresses the probability of the joint occurrence of three or more variables.
The mean, variance, skewness, and kurtosis are all moments of a distribution.
The mean (arithmetic), the first moment of a distribution, either of a mathematical variable or a random variable, can be regarded as the center of gravity of the distribution such that the sum of deviations from the mean for any distribution is equal to zero.
The variance, the second moment of a distribution, can be computed for either a mathematical variable or a random variable. It expresses the degree to which scores, on average, deviate from the mean in squared units.
The sample variance with n in the denominator is biased. To correct for the bias, a single degree of freedom is subtracted so that the new denominator is n − 1.
The expectation of the uncorrected version of the sample variance is not equal to σ2. That is, E(S2) ≠ σ2. However, the corrected version of the sample variance (with n − 1 in the denominator) is equal to σ2. That is, E(s2) = σ2.
Skewness, the third moment of a distribution, reflects the extent to which a distribution lacks symmetry.
Kurtosis, the fourth moment of a distribution, reflects the extent to which a distribution is peaked or flat and also having much to do with a distribution's tails.
Covariance and correlation are defined for both empirical variables and random variables. Both measure the extent to which two variables are linearly related. Pearson r is the standardized version of the covariance, and is dimensionless, meaning that its value is not dependent on the variance in each variable. Pearson r ranges from −1 to +1 in value.
One popular use of correlation is in establishing reliability and validity of psychometric measures.
In multivariable contexts, covariance and correlation matrices are used in place of single coefficients.
There are numerous other correlation coefficients available other than Pearson r. One such coefficient is Spearman's rs, which captures monotonically increasing (or decreasing) relationships. Monotonic relationships do not necessarily have to be linear.
The issue of measurement should be carefully considered before data is collected. S.S. Stevens proposed four scales of measurement, nominal, ordinal, interval, and ratio. The most sophisticated level of measurement is that of the ratio scale where a value of zero on the scale truly means an absence of the attribute under study.
A random variable is a mathematical variable that is associated with a probability distribution. More formally, it is a function from a sample space into the real numbers.
An estimator is a function of a sample used to estimate a parameter in the population.
An interval estimator provides a range of values within which the true parameter is hypothesized to exist.
An unbiased estimator is one in which its expectation is equal to the corresponding population parameter. That is, E(T) = θ.
An estimator is consistent if as sample size increases without bound, the variance of the estimator approaches zero.
An estimator is efficient if it has a relatively low mean squared error.
An estimator is sufficient for a given parameter if the statistic tells us everything we need to know about the parameter and our knowledge of it could not be improved if we considered additional information (e.g., such as a secondary statistic).
The concept of a sampling distribution is at the heart of statistical inference. A sampling distribution of a statistic is a theoretical probability distribution of that statistic. It is idealized, and hence not ordinarily empirically derived.
The sampling distribution of the mean is of great importance because so many of our inferences feature means.
As a result of , we can say that , that is, the mean of all possible sample means we could draw from some specified population is equal to the mean of that population.
The variance of the sampling distribution of the mean is equal to of the original population variance. That is, it is equal to .
The square root of the sampling variance for the mean is equal to the standard error, .
The central limit theorem is perhaps the most important theorem in all of statistics. Though there are different forms of the theorem, in general, it states that the sum of random variables approximates a normal distribution as the size upon which each sample is based increases without bound.
Confidence intervals provide a range of values for which we can be relatively certain to lay the true parameter we are seeking to estimate. Key to understanding confidence intervals is to recognize that it is the sample upon which the interval is computed that is the random component, and not the parameter we are seeking to estimate. The parameter is typically assumed to be fixed.
Student's t distribution, derived by William Gosset (or “Student”) in 1908, is useful when σ2 is unknown and must be estimated from the sample. Because in the limit f(t) = f(z) (i.e., ), for large samples, whether one uses z or t will make little difference in terms of whether or not the null hypothesis is rejected.
The t‐test for one sample compares an obtained sample mean to a population mean and evaluates the null hypothesis that the sample mean could have reasonably been drawn from the given population.
As degrees of freedom increase, the variance of the t‐distribution approaches 1, which is the same as that for a standardized normal variable. That is, .
The t‐test for two samples tests the null hypothesis that both samples were selected from the same population. A rejection of the null hypothesis suggests the samples arose from populations with different means.
Power is the probability of rejecting a null hypothesis given that it is false. It is equal to 1 − β (i.e., 1 – type II error rate). Power is a function of four elements: (1) hypothesized value under H1, (2) significance level, or type I error rate, α, (3) variance, σ2, in the population, and (4) sample size.
Experiments or studies suffering from insufficient power make it difficult to ascertain why the null hypothesis failed to be rejected.
The paired‐samples t‐test is useful for matched‐pairs (elementary blocking) designs.
The paired‐samples t‐test usually results in an increase in statistical power because the covariance between measurements is subtracted from the error term. In general, anything that makes the error term smaller helps to boost statistical power.
The paired‐samples t‐test and the matched design which it serves provides a good entry point into the discussion of the randomized block design, the topic of Chapter 6.
In multivariable contexts, linear combinations of variables are generated of the form ℓi = a1y1 + a2y2 + … + apyp. Means and variances of linear combinations can be obtained, as well as the covariance and correlation between linear