Probability and Statistical Inference. Robert Bartoszynski

Probability and Statistical Inference

and frequency is (and had been for a long time) very well grounded in everyday intuition. For instance, loaded dice were on several occasions found in the graves of ancient Romans. That indicates that they were aware of the possibility of modifying long‐run frequencies of outcomes, and perhaps making some profit in such a way.

Today, the intuition regarding relationship between probabilities and frequencies is even more firmly established. For instance, the phrases “there is 3% chance that an orange picked at random from this shipment will be rotten” and “the fraction of rotten oranges in this shipment is a 3%” appear almost synonymous. But on closer reflection, one realizes that the first phrase refers to the probability of an event “randomly selected orange will be rotten,” while the second phrase refers to the population of oranges.

The precise nature of the relation between probability and frequency is hard to formulate. But the usual explanation is as follows: Consider an experiment that can be repeated under identical conditions, potentially an infinite number of times. In each of these repetitions, some event, say images , may occur or not. Let images be the number of occurrences of images in the first images repetitions. The frequency principle states that the ratio images approximates the probability images of event images , with the accuracy of the approximation increasing as images increases.

Let us observe that this principle serves as a basis for estimating probabilities of various events in the real world, especially those probabilities that might not be attainable by any other means (e.g., the probability of heads in tossing a biased coin).

We start this chapter by putting a formal framework (axiom system) on a probability regarded as a function on the class of all events. That is, we impose some general conditions on a set of individual probabilities. This axiom system, due to Kolmogorov (1933), will be followed by the derivation of some of its immediate consequences. The latter will allow us to compute probabilities of some composite events given the probabilities of some other (“simpler”) events.

2.3 Axioms of Probability

Let images be the sample space, namely the set of all outcomes of an experiment. Formally, probability, to be denoted by images , is a function defined on the class of all events,¹ satisfying the following conditions (usually referred to as axioms):

Axiom 1 (Nonnegativity):

Axiom 2 (Norming):

Axiom 3 (Countable Additivity):

for every sequence of pairwise disjoint events images (such that images for all images ). images

If the sample space images is finite or countable, one can define a probability function images as follows: Let images be a nonnegative function defined on images , satisfying the condition images . Then, images may be defined for every subset images of images as images . One can easily check that images satisfies all three axioms.

Indeed, images because images is nonnegative, and images . Finally, let images be a sequence of disjoint subsets of images . Then,

Passing from the first to the second line is allowed because images are disjoint, so each term appears only once. The sum in the second line is well defined (i.e., it does not depend on the order of summation because the terms are nonnegative).

However, if images is not countable, one usually needs to replace summation by integration, images . This imposes some conditions on functions images and on the class of events images . For a detailed discussion, the reader is referred to more advanced probability texts (e.g., Chung, 2001).

Illustration for hitting a target depicting that the point of hit lies somewhere in the shaded annulus A.

Figure 2.1 Hitting a target.

Example 2.1 Geometric Probability

One of the first examples of an uncountable sample space is associated with “the random choice of a point from a set.” This phrase is usually taken to mean the following: a point is selected at random from a certain set images in a finite‐dimensional space (line, plane, etc.), where images has finite measure (length, area,

Скачать книгу