Statistics and the Evaluation of Evidence for Forensic Scientists. Franco Taroni
say. This statistic provides a measure of the difference, known as a standardised difference, between the two sets of measurements that takes account of the natural variation there is in the refractive index measurements of glass fragments from within the same window. If the absolute value of
The second stage is known as the significance stage. This stage attempts to determine the significance of the finding from the first stage that the two sets of fragments were similar. The significance is determined by calculating the probability of the result that the two sets of fragments were found to be similar, under the assumption that the two sets had come from different sources. If this probability is very low then this assumption is deemed to be false. The fragments are then assumed to come from the same source, an assumption that places the PoI at the crime scene. This assumption says nothing about how the fragments came to be associated with the PoI. This may have occurred in an innocent manner. See Section 5.3.2 for further discussion of this point in the context of activity level propositions.
The procedure can be criticised on two points. First, in the comparison stage the threshold provides a qualitative step that may provide very different outcomes for two different pairs of observations. One pair of sets of fragments may provide a value of
This sudden change in decision due to crossing a particular line is likened to falling off a cliff, one moment you are safe, the next dead. In fact, rather than a cliff we have merely a steep slope. Other things being equal, the more similar the samples the stronger the evidence that they had a common origin, and the less similar the samples the stronger the evidence that they came from different sources. (p. 118)
A related problem is that of cut‐off where a decision is taken dependent on whether a statistic is above or below a certain value (see Section 7.7.5).
A better approach, as suggested in the quotation from Robertson and Vignaux (1995b) above and that is described in Section 7.3, provides a measure of the value of the evidence that decreases as the distance between the two sets of measurements increases, subject, as explained later, to the rarity or otherwise of the measurements.
The second criticism is that the result is difficult to interpret. Because of the effect of the comparison stage, the result is not just simply the probability of the evidence, assuming the two sets of fragments came from different sources. A reasonable interpretation, as will be explained in Section 2.4, of the value of the evidence is the effect that it has on the odds in favour of the true guilt of the PoI. In the two‐stage approach this effect is difficult to measure. The first stage discards certain sets of measurements, which may have come from the same source and does not discard other sets of measurements which may have come from different sources. The second stage calculates a probability, not of the evidence but of that part of the evidence for which
A review of the two‐stage approach and the development of a Bayesian approach is provided by Curran et al. (2000) and Curran and Hicks (2009).
As with DNA profiling, there are problems associated with the definition of a suitable population from which probability distributions for refractive measurements may be obtained; see, for example, Walsh and Buckleton (1986).
These examples have been introduced to provide a framework within which the evaluation of evidence may be considered. In order to evaluate evidence, something about which there is much uncertainty, it is necessary to establish a suitable terminology and to have some method for the assessment of uncertainty. First, some terminology will be introduced followed by a method for the measurement of uncertainty. This method is probability. The role of uncertainty, as represented by probability, in the assessment of the value of scientific evidence will form the basis of the rest of this chapter. A commentary on so‐called knowledge management, of which this is one part, has been given by Evett (1993b, 2015).
1.4 Terminology
It is necessary to have clear definitions of certain terms. First note that the term evidence is generally used in the literature and in practice rather than terms such as finding, outcome, or material. The term evidence may be confusing. In some legal contexts it can refer to a judicial qualification of a finding. Forensic scientists are interested in the probative value of an observation before it qualifies as evidence in a trial. The European Networks of Forensic Science Institutes (ENFSI) Guideline for evaluative reporting (ENFSI 2015) provides definitions of both terms (pp. 19–20). Despite the risk of confusion, the term evidence will be used for the material examined by the scientist and for which the value is required.
The crime scene and suspect (associated with a PoI) materials have fundamentally different roles. The assignment of a probability for a correspondence between two randomly chosen sets of materials is not the important issue. One set of materials, crime scene or suspect, can be assumed to have a known source. It is then required to assess the probability of the corresponding material, suspect or crime scene, corresponding in some sense, to the known set of materials, under two competing hypotheses. Examples 1.1 and 1.2 serve to illustrate this.
Example 1.1. (continued) A crime is committed. A bloodstain is found at the scene of the crime. All innocent explanations for the presence of the stain are eliminated. A PoI is found. Their DNA profile is found to match that of the crime stain. The crime scene material is the crime stain. The suspect material is a swab with saliva. The finding to evaluate is the observed correspondence between the DNA profiles of the crime scene and suspect materials.
Example 1.2 (continued) As before, consider the investigation of a crime. A window has been broken during the commission of the crime. Several fragments are taken for investigation and measurements made of their refractive indices. These fragments, as their origin is known, are sometimes known as control fragments and the corresponding measurements are