經(jīng)管學(xué)會(huì)網(wǎng)盤光華方向?qū)I(yè)概統(tǒng)lecture

1、1Lecture 24Method of Least SquaresSimple Linear RegressionThe Method of Least SquaresSuppose that each of 10 patients is treated first with a certain amount of a standard drug A and then with an equal amount of a new drug B, and that the change in blood pressure induced by each drug is observed for

2、each patient. i xi (drug A) yi (drug B)11.90.7 20.8-1.031.1-0.240.1-1.25-0.1-0.164.43.474.60.081.60.895.53.7103.42.0 We might be interested in describing the relationship between the response y of a patient to drug B and his response x to drug A.We might also wish to predict the response y of a futu

3、re patient to the new drug B on the basis of his response x to the standard drug A.We can try to fit a straight line to the points. outlierThe Least Square LineConsider an arbitrary straight line y=b0+b1x. The vertical distance between the point (xi,yi) and the line is |yi-(b0+b1xi)|. Let Q denote t

4、he sum of the squares of the vertical distances at the n points.The method of least squares specifies that the values of b0 and b1 must be chosen to minimize Q.normal equationsThe least square line isFitting a PolynomialSuppose that we wish to fit a polynomial of degree k (k=2) of the following form

5、The method of least squares specifies that the constants b0, bk should be chosen to minimize A set of k+1 normal equations can again be obtained by setting each of the partial derivative equal to 0 for j=1,k+1.ExampleSuppose that we wish to fit a polynomial of the formto the previous 10 observations

6、. The normal equations are found to beThe solution to the equations areSo the least square parabola isFit A Linear Function of Several VariablesSuppose that we wish to represent a patients response to drug B as a linear function involving not only his response to drug A but also some other relevant

7、variables. For each patient i (i=1,n), we measure his response yi to drug B, his response xi1 to drug A, and his values xi2,xik for the other variables. We wish to fit a linear function having the formWe need to minimize The Normal EquationsExampleSuppose for each patient i, xi1 denotes his response

8、 to drug A, xi2 denotes his heart rate, and yi denotes his response to drug B. We wish to fit a linear function having the formi xi1xi2yi11.9660.7 20.862-1.031.164-0.240.161-1.25-0.163-0.164.4703.474.6680.081.6620.895.5683.7103.4662.0 The normal equations areThe solution to the equations areSo the l

9、east squares linear function is RegressionConsider the conditional distribution of some random variable Y for given values of some other variables X1,Xk.Some of the variables X1,Xk might be random variables.Some of the variables X1,Xk might be control variables whose values are to be chosen by the e

10、xperimenter.The conditional expectation of Y for any given values of X1,Xk is called the regression of Y on X1,Xk. Linear RegressionAssume that the regression function is linear:The coefficients b0,bk are called regression coefficients.We can get least square estimators of b0,bk. Simple Linear Regre

11、ssionConsider the regression of Y on just a single variable X. Assume that for any given value X=x, the random variable Y can be represented by Y=b0+b1x+ewhere e is a random variable andThereforen pairs of observations (x1,Y1),(xn,Yn): We condition on the values x1,xn before computing the joint dist

12、ribution of (Y1,Yn)The random variables Y1,Yn are independent. For i=1,n, The M.L.EThe likelihood function of b0, b1 and :The values of b0 and b1 for which the likelihood function is a maximum will be the values that minimizeSo the M.L.E. of b0 and b1 are the least square estimators.The M.L.E. for is Simple Linear Regression AssumptionsLinearity: E(Y|x)=b0+b1x.Normality: each variable Yi has a normal distribution.Independence: the variables Y1,Yn are independent.Homoscedasticity: the variables Y1,Yn have the s