北大暑期課程《回歸分析》(Linear-Regression-Analysis)講義pku4教學(xué)教材

1、學(xué)習(xí)-好資料Class4:Inferenceinmultipleregression.I.TheLogicofStatisticalInferenceThelogicofstatisticalinferenceissimple:wewouldliketomakeinferencesaboutapopulationfromwhatweobservefromthesamplethathasbeendrawnrandomlyfromthepopulation.Thesamples'characteristicsarecalled"pointestimates."Itisa

2、lmostcertainthatthesample'scharacteristicsaresomewhatdifferentfromthepopulation'scharacteristics.Butbecausethesamplewasdrawnrandomlyfromthepopulation,thesample'scharacteristicscannotbe"verydifferent"fromthepopulation'scharacteristics.WhatdoImeanby"verydifferent"?T

3、oanswerthisquestion,weneedadistancemeasure(ordispersionmeasure),calledthestandarddeviationofthestatistic.Tosummarize,statisticalinferencesconsistoftwosteps:Pointestimates(samplestatistics)(2)Standarddeviationsofthepointestimates(dispersionofsamplestatisticsinahypotheticalsamplingdistribution).II.Rev

4、iew:Forasampleoffixedsizei-1,n,yisthedependentvariable;X=1兇，Xpcontainsindependentvariables.Wecanwritethemodelinthefollowingway:頭二二,Undercertainassumptions,theLSestimatorb=(XX)XyAscertaindesirableproperties:A1=>unbiasednessandconsistency12A1+A2=>BLUE,withV(b)=(XX)ai+A2+A3=>bueV(b)=(XX)'二

5、2(evenforsmallsamples)，and variance ofIII. TheCentralLimitTheoremStatement:Themeanofiidrandomvariables(withmeanof_2、,.,.-1)approachesanormaldistributionasthenumberofrandomvariablesincreases.Xn-N(,二2/n),asn一二學(xué)習(xí)-好資料Thepropertybelongstothestatistic-samplemeaninthesamplingdistributionofallsamplemeans,ev

6、enthoughtherandomvariablesthemselvesarenotnormallydistributed.Youcannevercheckthisnormalitybecauseyoucanonlyhaveonesamplestatistic.Inregressionanalysis,wedonotassumethat;isnormallydistributedifwehavealargesample,becauseallestimatedparametersapproachtonormaldistributions.Why:allLSestimatesarelinearfu

7、nctionof;(provedlasttime).Recallatheorem:alineartransformationofavariabledistributedasnormalisalsodistributedasnormal.IV. InferencesaboutRegressionCoefficientsA. PresentationofRegressionResultsCommonpractice:giveastarbesidetheparameterestimateforsignificancelevelof0.05,twostarsfor0.01,andthreestarsf

8、or0.001.Forexample:DependentVariable:EarningsIndependentVariable:Father'seducation0.900*Mother'seducation0.501*Shoesize-2.16Whatistheproblemwiththispractice?First,wewanttohaveaquantitativeevaluationofthesignificancelevel.Weshouldnotblindlyrelyonstatisticaltests.Forexample,Father'seducati

9、on0.900*(0.450)Mother'seducation0.501*(0.001)Shoesize-2.16(1.10)Inthiscase,isfather'seducationmuchmoresignificantthanshoesize?Notreally.Theyareverysimilar.Bycontrast,mother'sisfarmoresignificantthantheothertwo.Asecondpracticeistoreportthetorzvalues:Coeffi.t.Father'seducation0.9002.0M

10、other'seducation0.501500.Shoesize-2.16-1.96Thissolutionismuchbetter.However,veryoften,ourhypothesisisnotaboutdeviationfromzero,butfromotherhypotheticalvalues.Forexample,weareinterestedinthehypothesiswhetheraone-yearincreaseinfather'seducationwillincreaseson'seducationbyoneyear.Thehypothe

11、sishereis1insteadof0.學(xué)習(xí)-好資料Thepreferredwayofpresentationis:Coeff.(S.E.)Father'seducation0.900(0.450)Mother'seducation0.501(0.001)Shoesize-2.16(1.10)B. DifferencebetweenStatisticalSignificanceandtheSizeofanEffectStatisticalsignificancealwaysreferstoastatedhypothesis.Youwillseealotofmisusesint

12、heliterature,sometimesbywell-knownsociologists.Theywouldsaythatthisvariableishighlysignificant.Thatoneisnotsignificant.Thisisnotcorrect.Iamnotresponsiblefortheirmistakes,butIwanttowarnyounottocommitthesamemistakesagain.Inourexample,youcouldsayMother'seducationishighlysignificantfromzero.Butitisn

13、otsignificantfrom0.5.HadyourhypothesisbeenthattheparameterforMother'seducationis0.5,theresultwouldbeconsistentwiththehypothesis.Thatis,statisticalsignificanceshouldalwaysbemadewithreferencetoahypothesis.Followz=(b2-0)/S是Anothercommonmistakeistoequatestatisticalsignificancewiththesizeofaneffect.A

14、variablecanbestatisticallysignificantfromzero.Buttheestimatedcoefficientissmall.Thecontributionoffather'seducationtothedependentvariableislargerthanthatofmother'seducationeventhoughmother'seducationismorestatisticallysignificantfromzerothanfather'seducation.Important:youshouldlookatb

15、othcoefficientsandtheirstandarderrors.C. ConfidenceIntervalsforSingleParameters'-j-bj-1.96SE(bj)D. HypothesisTestingforSingleParametersComparez=(bj，j°)/SE(bj),ifzisoutsidetherangeof-1.96and1.96,thehypothesisisrejected.Otherwise,wefailtorejectthehypothesis.學(xué)習(xí)-好資料V. InferencesaboutLinearCombi

16、nationsofTwoParametersExample 1: I'1=：2(equalityhypothesis),=>：i_：2=0Example 2: 卜11=IO!：(proportionalityhypothesis),=>-1-10-2=0Example 3: %=：242(surplushypothesis),=>：1-：2=2In general form, we may haveHypothesis testing:Confidence interval:c1：1-C2：2(H。C1-1C2-2=cC1-1'c2-2liesbetween(

17、lowlimit,upperlimit)Procedure:A. Pointestimate:Computec1b1c2b2B. DegreeofImprecisionV(c1b1c2b2)=c2V(b1)c2V(bz)2c1c2Cov(b1,b2)andthentakesquarerootofV(c1b1c2b2).Wewouldneedtoobtainthevariance-covariancematrixoftheparametervectorinordertocarryoutthecalculation.V(b1)andV(b2)areonthediagonal,Cov(b1,b2)i

18、soff-diagonal.Letuslookatthefirsthalftheexample.Computetheconfidenceintervalforthehypothesisthat-1=-2.Step1.b1-b2=0.5732-0.3146=0.2586.Step2:V(b1-b2)=0.0642+0.0310-2*(-0.0227)=0.1406.SD(b1-b2)=0.14061/2=0.3750.學(xué)習(xí)-好資料Step3:Computet2=(0.2586-0)/0.3750=0.6897,insignificant(unsurprisingresult).NoteDF=5-

19、3=2.Iusetwoparametersasanexample.ExamplesofHypothesisTesting:Example 1: ：1=：2(equalityhypothesis),=>：1-：2=0Example 2: 11=10:2(proportionalityhypothesis),=>'-1-10-2=0Example 3: ：1=：2'2(surplushypothesis),=>：1-*2=2Ingeneralform,WemayhaveC1-1c2-2=Hypothesistesting:C1:1c2:2=cConfidencei

20、nterval:C1：1c2-2liesbetween(lowlimit,upperlimit)Procedure:A. Findpointestimate:Computec1b1+c2b2B. FinddegreeofImprecision2.2.Vc1b1c2b2=c1Vb1c2Vb2廣2c1c2Covb1,b2andthentakesquarerootofVc1b1c2b2.Wewouldneedtoobtainthevariance-covariancematrixoftheparametervectorinordertocarryoutthecalculation.V(b1)andV

21、(b2)areonthediagonal,Cov(b1,b2)isoff-diagonal.學(xué)習(xí)-好資料NumericalExample:MultipleregressionsForthefollowingsmalldataset(n=5),usematrixoperationstosolvethefollowingproblems.Youshouldmakeuseoftheinformationbelowasmuchaspossible.Let-1X1X21y'=103241“二102041X2'=02456】Itisknownthat5717X'X=7321. 81一10X'y=：46J.7030-.0239-.1381(XX)、=-.0239.1205-.0426.1381-.0426.0582j更多精品文檔1.WriteoutXandy_infull.1xii1X21X=1X311X411X51X12X22X32X42X521111-1100224,y0546丁0324-5XX = 7：172. WriteXXandXyinfull.717102132,Xy=233281J46學(xué)習(xí)-好資料3.Estimatebwiththehelpofacalculator(bistheleastsquaresestimatorofI',