StatisticalFoundations2ProbabilityDistributions

1、2.Probability Distributions2.1Random VariablesTo be able to exploit fully the power of probability, we must introduce a new concept, that of the random variable. We have already encountered examples of random variables: the outcome of a toss of a coin, a roll of a die or a draw of a card, and the nu

2、mber of girls in a family of five children, are all random variables. A random variable is one whose outcome or value is the result of chance and is therefore unpredictable. But we must be clear what we mean here by unpredictability: it does not mean that we know absolutely nothing at all about the

3、values that a random variable can take; rather, it means that the values cannot be predicted with complete certainty. We know that when a coin is tossed it will land either heads or tails, but before the toss we do not know what the outcome will be with complete certainty (unless, of course, it is a

4、 two-headed coin!) Similarly with rolling a die and picking a card: the possible outcomes are determined by the sample space of the experiment, but none of them are individually certain to occur (although one of them will) and thus they all have probabilities associated with them. For example, rolli

5、ng a 6 will occur with probability if the die is fair, but it is impossible to roll a 7, which will thus have zero probability associated with it. These are all examples of discrete random variables, where the sample space is defined over a finite number of outcomes. Many random variables have sampl

6、e spaces associated with them that have an infinite number of outcomes, and these are known as continuous random variables. An example would be the height of a student drawn from those taking this module. In principle, this height could be measured to any degree of accuracy and thus could take on an

7、 infinite number of values. Of course, it may be argued that in practice measuring instruments are limited in their precision, so that we can only measure height over a finite number of values. A response to this would be that, although finite, the number of values could nevertheless be extremely la

8、rge, so that we may as well act as if the random variable was continuous, and this is something that is done regularly in statistical analysis.2.2Probability DistributionsAs we have seen, the values or outcomes taken by a discrete random variable will have probabilities associated with them. A listi

9、ng of these values and accompanying probabilities is called a probability distribution. For example, consider the probability distribution of the random variable defined to be the outcome of rolling the biased die in which even numbers were twice as likely as odd numbers: if the random variable is d

10、enoted X, and defined as , , then we can list the probability distribution of X as123456This shows clearly that a probability distribution has the following properties, which follow directly from the probability axioms:Expected value and varianceJust as we can compute the mean of a sample of data, w

11、e can also compute a (weighted) mean for a random variable X. This is called the expected value, denoted , and is defined as the weighted sum of the n values taken by X, with the weights given by the associated probabilities:For the weighted die, This has the following interpretation: if we rolled t

12、his die a large number of times, recorded the results of the throws, and calculated the cumulative average, this will eventually settle down to . Thus the expected value is often referred to as the mean of X.Similarly, we can define the variance of X asso, for our weighted die,Alternatively, we can

13、note thatwhere we use the result that , because is, by definition, a single number, i.e., a constant, and the expectation of a constant is itself. Thus and may thus be interpreted as measures of the central tendency and dispersion of the probability distribution of X. As with sample variances, we ma

14、y take the square root of to be the standard deviation of X: . For the above distribution, .We may be able to associate particular probability distributions with random variables. For example, a fair six-sided die would have the probability distribution This is an example of the uniform distribution

15、, which in general can be written aswhich can be shown to have expected value and variance . For , and , so that X takes the values 4, 7, 10, 13, 16, 19, 22 and 25 with probability and all other values have zero probability, a graphical representation of the uniform distribution is shown below: it h

16、as and .For the fair die, , and , and thus and . When a variable X follows a uniform distribution with parameters k, h and n, we use the notationwhich is to be read as X is distributed as uniform with parameters k, h and n. The random variable defined to be the result of tossing a fair die is thus,

17、and we can say that X is a uniformally distributed random variable.2.3The Binomial DistributionA second popular probability distribution defined for a discrete random variable is the binomial. This provides the general formula for computing the probability of r successes from n independent trials of

18、 an experiment that has only two mutually exclusive outcomes, generically defined as success and failure, and where the probability, p, of success (and hence the probability, , of failure), remains constant between trials. As an example of a binomial experiment, recall the family of five children of

19、 whom three are girls. To calculate the probability of the event of three girls from five children, we have to be able to assume that the outcome of successive trials, i.e., births, are independent and that the probability of a girl birth remains constant at p across trials. As we saw, there are ten

20、 possible orderings of three girls and two boys, and each of these will occur with probability , where we make use of the multiplication rule under independence. Thus, if X is the random variable defined to be the number of girls,Thus, if ,whereas if More generally, the probability of r successes fr

21、om n trials is thusIt can be shown that and that and . These formulae show that the binomial distribution has two parameters, the number of trials n and the probability p, so that we can write for a binomially distributed random variable.Only allowing two outcomes is not as restrictive as it may see

22、m. For example, we can compute the probability of, say, three 5 or 6s from six rolls of a fair die by defining the outcome of 5 or 6 as a success, with probability , and not 5 or 6 as a failure, with probability . The required probability is thenMore complicated probabilities may be calculated by us

23、ing the mutually exclusive version of the additive rule. For example, the probability of getting more than two girls in a family of five is given byFor many binomial probability calculations, tables may be used. For example, the Statistical Tables produced by the Department of Economics contains, in

24、 Table 6, probabilities for and . 2.4The Poisson DistributionThese tables do not help when the number of trials n exceeds 20 or when the probability of success p is less than 0.05. However, in such circumstances, another discrete distribution, the Poisson, provides an excellent approximation. To see

25、 this, consider a typical example in industrial quality control. Here the number of trials (the number of items of a good that is produced) is large and, hopefully, the probability of a defective item is very small. Suppose that a manufacturer gives a two-year guarantee on the product that he makes

26、and from past experience knows that 0.5% of the items produced will be defective and will fail within the guarantee period. In a consignment of 500 items, the number of defectives will be and the probability that the consignment will contain no defectives is then 8.16%, calculated usingHowever, it t

27、urns out that, if n is large and p is small, such that , binomial probabilities are closely approximated by the values calculated using the following mathematical expression (2.1)Thus, since , we can approximate the probability of no defectives byi.e., 8.21%, which is probably accurate for most purp

28、oses. This approximation becomes more useful when more involved probabilities are required. The probability of more than three items being defective is given byUsing the above approximation, the calculation is much simpler and almost as accurate:Equation (2.1) in fact provides the formula for the Po

29、isson distribution, which is also known as the distribution of rare events. Since is the expected value (mean) of the binomial distribution, with the Poisson we can denote this as , write (2.1) asand say that . In this setup, there is no natural number of trials: rather, we consider the number of trials to be the number of time or spatial intervals that the random variable is observed over, for which we know the mean number of occurrences per interval. Thus, suppose a football team scores a