統(tǒng)計學(xué)方法與應(yīng)用課件lecture4

統(tǒng)計學(xué)方法及其統(tǒng)計學(xué)方法及其應(yīng)StatisticalMethodswithMinistryofEducationKeyLaboratoryofBioinformaticsBioinformaticsDivision,TNLIST/DepartmentofAutomationTsinghuaUniversity,Beijing100084,ChinaRuiJiang,PhDRandom統(tǒng)計學(xué)方法及其Random統(tǒng)計學(xué)方法及其應(yīng)統(tǒng)計學(xué)基隨機(jī)變量的函“Arandomvariableisaquantitywhosevaluesarerandomandtowhichaprobabilitydistribution通過對部分的觀測推斷整通過對部分的觀測推斷整體的性~107測量全國兒童的身全體兒童的集K={k1,k2,…,kN},N:兒童身高的集H={h1,h2,…,hM},M:每次選取一名兒童進(jìn)行觀測得到一個隨機(jī)變X:Hpmf(pdf):fX(x)=f進(jìn)行n次觀測得到n個隨機(jī)變Xi:Hpmf(pdf):fXi(x)=fKfKf(x)研究全中國兒童的身研究降壓藥物的降壓作RandomsamplingwithKRandomsamplingwithK={k1,k2,…,H={h1,h2,…,N:兒童的數(shù)=NM:兒童身高的可能取值的數(shù)量,P(Ki=kj)=1/N,i=1,2,…,n,j=RandomsamplingwithMutuallyIdenticallyRandomsamplingwithoutP(K2RandomsamplingwithoutP(K2=ki|K1=kj)=P(K2 =ki|K1=kj)=1/(N-i=i1RandomsamplingwithoutNotIdenticallyNP(K2=ki)=?P(K2=ki|K1=kj)P(K1=kj=?P(K2=ki|K1=kj)P(K1=kj)+P(K2=ki|K1=ki)P(K1=ki)j1i =(N-1)? 1èN-1N=NRandomsamplingfrominfiniteWhenRandomsamplingfrominfiniteWhenn=1/(N-i+1)?1/N=P(Ki=P(Ki =kk|K1,Ki-￥WhenMutuallyIdenticallyRandomsamplingfrominfiniteRandomRandomTherandomX,arecalledarandom1nareRandomRandomTherandomX,arecalledarandom1naremutuallyofsizenfromthepopulationf(x)ifX,1nindependentrandomvariablesandthemarginalpdforpmfeachisthesamefunctionf(x).Alternatively,X,ncalledindependentandidenticallydistributed(iid)variableswithpdforpmff(x).Arealizationofthese,xn,iscalledanoftheX,XnWhatdoesdescriptivestatisticsreallyofsizethatWhatdoesdescriptivestatisticsreallyofsizethatfromrandomabeX,1ncertainpopulation.Letx1nobservationofX,Xn.DescriptivestatisticsaimsatrepresentingthisobservationbymeansoffiguresandmakingtheinformationcontainedinthesampleWhatdoesinferentialstatisticsreallySincetherandomidenticallydistributed,thegivenbyX,WhatdoesinferentialstatisticsreallySincetherandomidenticallydistributed,thegivenbyX,areindependent1jointnpmfofX,1nnf ,xn)=ff(xn)f(xi).iInparticular,iff(x)isamemberofaparamtricf(x|nf , |q)Inferentialstatisticsthenf(xi|iatusingobservationsofsampletoinferpropertiesassociatedwithqPopulation,sample,descriptivestatistics,Population,sample,descriptivestatistics,inferentialqqsamplingInferRepresentLetXLetX1,,XnbearandomsampleofsizenfromapopulationandletT(x1,,xn)beareal-valuedorvectorvaluedfunctionwhosedomainincludesthesamplespaceof(X1,?,Xn).ThentherandomvariableorrandomvectorT=(X1,?,Xn)iscalledastatistic.TheprobabilitydistributionofastatisticYiscalledthesamplingdistributionofYSampleThesamplemeanisthearithmeticofthevaluesaSampleThesamplemeanisthearithmeticofthevaluesarandomsample.Itisdenoted++X, )=X1nn?X=X(X=Xi1nniSamplevarianceandstandardThesamplevariancethestatisticdefined n-n=S= -X)2Sn,,)ni1innXnn?????=-2 -2SamplevarianceandstandardThesamplevariancethestatisticdefined n-n=S= -X)2Sn,,)ni1innXnn?????=-2 -2i2i22(-2Xi+X=2Xi+2Xii=ni=nXiiiWe??1n? -nX2÷,=say(n-=-n2222i2SXn-1èiiiThesamplestandardderivationthestatisticdefinedSS2andx=n++xn)/benx1,-a)2??-x)(iiaiinn- =-x+andx=n++xn)/benx1,-a)2??-x)(iiaiinn- =-x+x-n2iiiinn=??-x2 -x)(x-iiiiinn=?-x+n(x-+2(x- (x-iiiin?-x)+n(x-22 ii??nini(x-a)2(x-x)2obtainedata=xminimumiibeanyx=(x1+xn)/x1,nn??-x= -nx2(n-1)s2iiibeanyx=(x1+xn)/x1,nn??-x= -nx2(n-1)s2iiiinn?-x)+n(x-- 22iiLeta=0,nn??+2ix2 -xiInothernn??-x)2 -22iiComputationalvariablesXForaseriesofiidk1k==letXkii?k -kX2ComputationalvariablesXForaseriesofiidk1k==letXkii?k -kX2 k-k(X-=Xi.Sk-1èkikkii2kusingS2Canweexpressusingandkkkk? kXki(k+=+iky.kkkXi=(k+kk? kXki(k+=+iky.kkkXi=(k+kik(k- = -k222??kikk+-(k-=-Xi2k2kkkk?kS -(k+22kki?tiExpectationofarandombearandomsamplefromapopulationX,1nletExpectationofarandombearandomsamplefromapopulationX,1nletg(x)afunctionsuchthatEg(X1)andVarg(X1)?=nEg(X?n?Eg(Xè?1ii?=nVarg(Xn?g(X ?1ii?nn?Eè Eg(X)=nEg(Xg(Xi?i1ii2??únnn???=g(X)-g(Xg(X÷iúêèè?ii?iiiénn??ê=?nn?Eè Eg(X)=nEg(Xg(Xi?i1ii2??únnn???=g(X)-g(Xg(X÷iúêèè?ii?iiiénn??ê=g(X) )ú??iii=Eêùù2ég(X)-Eg(X??úii?éù2n(ù?éi))-Eg(X g(X)-Eg +=)-k)?g(?????iin?ù?é+E(g(X)-Eg(X))(g(X)-Eg(X g(X)-???iiiikii=nVarg(X1)+?Cov?g(Xi),g(Xk)i=nVarg(XSamplemeanandsamplebeaapopulationwithX ,SamplemeanandsamplebeaapopulationwithX ,1nands2<mEX=VarX,nES2==E??1nEn1nnn====EEXXiièè11ii?n11nn??Var =Var==nVarX1XX÷÷=E??1nEn1nnn====EEXXiièè11ii?n11nn??Var =Var==nVarX1XX÷÷èi2i2nniié21n? ES = -n?2ê?n-1ii1=(nEX2-nEX1n-1=[n(Var +(EX)2)-n(Var+(EX)n-11+m2= ên(sn-1=èSamplingNormal統(tǒng)計學(xué)方SamplingNormal統(tǒng)計學(xué)方法及其應(yīng)統(tǒng)計學(xué)基隨機(jī)變量的函“ArandomvariableisaquantitywhosevaluesarerandomandtowhichaprobabilitydistributionSamplingfromnormalOne-sampleSamplingfromnormalOne-sample—SamplingfromaunivariatenormalPaired-sampleSamplingfromabivariatenormalTwo-sample—SamplingfromtwounivariatenormalOne-samplemeanandSamplemeanandsampleXarandomsamplefromnormalOne-samplemeanandSamplemeanandsampleXarandomsamplefromnormalrandomX,1nN(m,s2),Sare(X,Y)berandomvectorjointpdforf(x,y).Thenandareindependentrandomandonlyifthereexistfunctionsg(x)andh(y)suchthat,foreveryx??andy??,f(x,y)=g(xnn?(X1-X)=-?(X-X)=0,-X ii1inùéùn1n-1nnêê (Xi-X)2úú2 -X -X?222-X)2(Xi-X(Xin-n-1i1???iiiiisafunctionof(X-X -X,?, -XSnn?(X1-X)=-?(X-X)=0,-X ii1inùéùn1n-1nnêê (Xi-X)2úú2 -X -X?222-X)2(Xi-X(Xin-n-1i1???iiiiisafunctionof(X-X -X,?, -XS23nConsiderthetransformationy1=x,yi=xi-x,i=2,...,n.Theinverseisx1=y1-(y2+ +yn),xi=y1+yi,i=2,?,n,Assumingthatm=0,s2=1,=withJ exp?xinnx22i =è-f(x,?,x)f(x)-i?2(2p)n/1ni2ii? +y) 2y2f(y,?,y)f(x,?,x)|---1 (2p)n/è ?1n1n2ny2?ùùé?n?1??n1)êy2úún= ÷=f(y)fexp(y÷?f(y,?,y÷,yn-è2p 2i2?(2p)(n-1), 1n??n2XisafunctionofonlyY,whileS isafunctionof,.12nDistributionsofsamplebeasamplefromanormalDistributionsofsamplebeasamplefromanormalX,1nN(m,s2),X-snhasastandardnormalSummationoftwoSummationoftwoindependentnormalrandomSummationoftwoSummationoftwoindependentnormalrandom ￥ ?(z-m)2? ?(z-w-u)2òfX(w)fY(z-w)dw= - - ? = ?(z-m-u)2exp 2ps2+ 2(s2+t2) s2+t2ò￥expê-1s2+t2?w-t+(z-u)s2?2 - s2+ ÷ 1LetX N(m,s2)andY N(u,t2)betwoindependentnormalrandomvariables,thenZ=X N(m+u,s2+t2Z=X- N(m-u,s2+t2Summationofmultipleiidn?~N(nm,ns2Summationofmultipleiidn?~N(nm,ns2XiNX1X+X~N(2m,2s2X1+X2(X1+X2+Xn-~N((n-1)m,(n-1)s2+Xn-1)+Xn~N((n-1)m,(n-1)s)+N(m,s~N(nm,ns222One-sample?~N1nn?XXni-(y-nm)2n ?~N(nm,One-sample?~N1nn?XXni-(y-nm)2n ?~N(nm, Y2p(ns2) X2(ns2Yii=nZ,dyZ=1 n??(nz- 2f(z)=f(nz)?=-ZY= exp-(z-è2(s2/n2p(s2/nmgfofthesampleLetX1,?,XnwithmgfMXbemgfofthesampleLetX1,?,XnwithmgfMXbearandomsample(t),thenthemgfofaX=(X++Xn)/MX(t)=[MX(t/n)]nLetY=X1+Xn,M(t)=YXLetX=Y/n,thenthemgfofMX(t)=MY(t/n)=[MX(t/n)]nmgfofthesampleLetX1,?,XnrandomvariableswithmgfMX(t),thenthebeZ=X1 +X éùnntS M(t)= ]=i]=mgfofthesampleLetX1,?,XnrandomvariableswithmgfMX(t),thenthebeZ=X1 +X éùnntS M(t)= ]=i]=E ú ]itXni[M(iZX??ii1LetZ=aY+b,YbearandomfY=Z-bdy=Yaa?z-bf(z)=1ZYaatz???z-b÷d(ay+b￥￥òòM(t)=t(aybeeZZYa--￥ò=(atefY-= YParticularly,ifZ=Y/n,thenMZ(t)=MY(t/Normal-(x-￥ =e-exp(x-1￥-=+ exp-2mx+m2+2s2tx?dx￥xNormal-(x-￥ =e-exp(x-1￥-=+ exp-2mx+m2+2s2tx?dx￥x￥-2(m+s2t?+m2-(m+s2t)2÷dx+(m+s2t 1￥x2é 1()12exp+=x-m+sts2t2÷úm2tè?- exp(x-(m+s2t))2ù￥11ò=-mt s2êúè?s?-222NormalsampleLetX1,?,XaN(m,s2bearandomsampleThen,themgfofthe?ùn???é???ù2+s2n/t?NormalsampleLetX1,?,XaN(m,s2bearandomsampleThen,themgfofthe?ùn???é???ù2+s2n/t??t==exp=exp+ú?ê2M(t èn??÷?÷ènXX?22??aXdistribution.nEquivalently,X-hasastandardnormaldistribution.snSummationofsampleDistributionSummationofsampleDistributionofthesumofsample Xi-misastandardsLetX1, ,Xn bearandomsamplefromanormalpopulationN(m,s2),thenn?X-m?2?èi i=1 hasa distributionwithndegreesofChi-squared-1=x e2Chi-squared-1=x e2,0<x<￥,p>G(p/2)2ppEX=Var =IllustrationofχIllustrationofχ2pdfanddistributioninQuantiledistributioninQuantilefunctionRandomGamma(p/2,scale=2)→2pGamma1f(x|a,q)Gamma(p/2,scale=2)→2pGamma1f(x|a,q)=<￥,a>0,q>0<xa-1e-x/c2ppxx-1-f(x)=<￥,p>e2,0<(p/2)2pAchi-squaredrandomvariablewithpdegreesoffreedomisagammarandomvariablewithshapep/2andscale2Gamma(1/2,scale=2)→21Gamma1<￥,a>Gamma(1/2,scale=2)→21Gamma1<￥,a>0,q>xa-1e-x/0<c211---111f(x)==e2,0<x<xe2(1/xAchi-squaredrandomvariablewith1degreesoffreedomisagammarandomvariablewithshape1/2andscale2Chi-squared mgfcanbecalculated ￥xa-1e-x/M(t)=- Chi-squared mgfcanbecalculated ￥xa-1e-x/M(t)=- ￥ò=1-x(1/ -ùéù111￥ò=?G(a)qa??G(a)[q/(1-qt-￥G(a)[q/(1-qt?? ?a=è1-qtc2,asGamma(p/2,2),thenhasp (t)=? ?p/2Summationofmultipleindependent++~2?npiip? ÷?(t)M÷?è1-2tp?21?n???pi111Summationofmultipleindependent++~2?npiip? ÷?(t)M÷?è1-2tp?21?n???pi111i(t)=?=÷÷÷22M÷÷÷???è1-2tè1-2tè1+++2?npiiIndependentchi-squaredrandomvariablesaddtoachi-squaredrandomvariable,andthedegreesoffreedomalsoadd(Standardnormal)2→21Y=g(X)=2ForarandomvariableN(0,1)andthex?-￥,0),y=g(x)=x,h(y)=-y,211x?(0,+￥),y(Standardnormal)2→21Y=g(X)=2ForarandomvariableN(0,1)andthex?-￥,0),y=g(x)=x,h(y)=-y,211x?(0,+￥),y=g(x)=x,h(y)2y,21=0(withprobabilityxA0?A1?A2=/èA1èA=(-￥,A-(-y2-111111-f(y)=InAee2211122yy2-y-111f(y)=InAee222221y-1f(y)=f(y)+(y)e2121yThesquareofastandardnormalrandomvariableisachi-squaredrandomvariablewith1degreeoffreedom DistributionsofsamplebearandomsamplefromaX,DistributionsofsamplebearandomsamplefromaX,1npopulationN(m,s2),(n-1)S? -Xn?=ièsidistributionwithn-1degreesofhasProof(assumekk+kS -(k- -2kkkkn=1:=1(X-because -Proof(assumekk+kS -(k- -2kkkkn=1:=1(X-because -~S~N(0,221121n=k(k-2 assumptioninkn=k+1:kk+kS =(k- -+~22kkXkkk1k~NN(0,1/-kN(0,1+1/~NXkXk+1-1+1/Thevarianceofthesample(n-1)S~ weé(nùThevarianceofthesample(n-1)S~ weé(nùé(nùThisistosayéù=(n-1)=s2E2???1èn=÷2(n-1)=n2Var.?n-1÷1DistributionsofsamplebearandomDistributionsofsamplebearandomsamplefromaN(m,s2)X,1nX-S hasaStudent'stdistributionwithn-1degreesofWhenvarianceisbearandomsamplefromaN(m,s2)population,LetX,1nX-~NsnWhenvarianceisbearandomsamplefromaN(m,s2)population,LetX,1nX-~Nsninmostcases,thetrueis(n-1)S~n-WeX-X-=X-ms= s Ns(n-1)S/(n-SnS (n-n-s2ThisButwhatisdesirablebecausetheisnotLetN(0,1)andbetwop12e-u/f(u)fLetN(0,1)andbetwop12e-u/f(u)f(v)1vp/2-1e-vG(p/2)2pUT,W=V/ /p,VU=the?w?1(w/0[1/(2p)](w1=èpJStudent’stof(U,V)f(u,v of(T,W)112-u/p/2--v e2pG(p/2)2p?w11Student’stof(U,V)f(u,v of(T,W)112-u/p/2--v e2pG(p/2)2p?w11f(t,w ==÷?è12w(p/2+1/2)-1e-(1/2)(t/pG(p/2)2p/212pG(p/2)2p/2wherea=(p+1)/2,q=2/(1+twa-1e-w//p).theofGamma(a,scale=() =ò+2p11+t2/2()G+()1 2pG(p/2)2p/2 +1 p1G=f(t)=òf(t,w)dwp222.( 1+t2/Gpp(1+t/2Student’st(G+1 |p)2,-￥<xStudent’st(G+1 |p)2,-￥<x<￥,p=f(Gpp(1+xp/p)(p+1)/2EX=0,p>pVarX=p-2,p>t1hasnomean,t2hasnoAtdistributionbecomesaCauchydistributionwhen(samplesize2,illnessappearsinordinarysituation)p=AtdistributionbecomesastandardnormaldistributionwhenthedegreeoffreedomtendstoinfinityStudent’stStudent’stStudent’stdistributioninQuantileStudent’stdistributioninQuantilefunctionRandomBivariatenormalArandomvector(X,Yiftheirjointpdfsaidtohasa1=′ (x,y|m,,22)XBivariatenormalArandomvector(X,Yiftheirjointpdfsaidtohasa1=′ (x,y|m,,22)X 1-2ps ù???x-m??y-m??2 ÷-2rY÷ú.exp÷+?2XXY2(1-r2)??ssssèXXYYMarginalParticularly,WMarginalParticularly,W=X- hasaN(m,s2)distribution, =m-m =s2-2rs +s2 Z=aX+b hasaN(m,s2)distribution, mZ=amX+bmY =a +2abrs +b2s2 If(X,Y Bivariatenormal(m,m,s2,s2,r),the distributionofXisN(m,s2),themarginaldistribution isN(m,s2 Paired-sampleLet(X1,Y1),,(Xn,Yn)bearandomsamplefromanormalpopulationwithparametersmPaired-sampleLet(X1,Y1),,(Xn,Yn)bearandomsamplefromanormalpopulationwithparametersm,m,s2,s2, (X-Y)-(X-mYs/nX-hasastandardnormaldistribution,(X-Y)-(X-mYSX-/nhasastudent'stdistributionwithn-1degreesof= -2rs+s,22X-X Y n-?(-Y)-(X-Y ù2n=Si=1?X-iiTwosamplebearandomsamplefromaN(m,s2)LetTwosamplebearandomsamplefromaN(m,s2)LetX ,1m , bearandomsamplefromanindependentN(m,Y1n =s2=s2XY(X-Y)-(X-mY+s1m1nhasanormaldistribution.TwosamplebearandomsamplefromaN(m,s2)LetX ,1mTwosamplebearandomsamplefromaN(m,s2)LetX ,1m bearandomsamplefromanindependentN(m,Y,1n XY(X-Y)-(X +S1m1nphasastudent'stdistributionwithm+n-2degreesoffreedom.(m- +(n-2Y2=SXm+n-piscalledthepooledvarianceDistributionsofsample(m-21c2X m-m-/S/SSU/m-1=Distributionsofsample(m-21c2X m-m-/S/SSU/m-1=== Y X X (n-/S/pq2V/1n- Y n-n-Y=(U/p)/(V/q),=UConsider LetX1, ,Xm bearandomsamplewithsizenfromanormalpopulationN(m,s2)andletY, ,Y bearandomsamplewithsizemfro aX nornalpopulationN(m,s2 nindependent Then,therandomS2/S S2/ / S2/ hasaSnedecor'sFdistributionwithm-1andn-1degreesoffreedom.Letandbetwopq1f(u)f(v)up/2-1e-u/G(p/2)2pLetandbetwopq1f(u)f(v)up/2-1e-u/G(p/2)2p1vq/2-1e-v/G(q/2)2qthe=/p=UFV/FF+q/(p/1+(p/qq/F+q/W1+(p/q)==U=theJacobianq/w(f+q/p)=w(q/ = (p/q) Jq/(f+q/p) (1+(p/q)2-(f+q/p)ff+q/pq/pf+q/FThef(u,v)Thepdfof(U,V)11up/2-1e-u/2vq/2-1e-vpdfof(F,W)is??q/2- 21+(p/q)?-1(p/(p/q)w1FThef(u,v)Thepdfof(U,V)11up/2-1e-u/2vq/2-1e-vpdfof(F,W)is??q/2- 21+(p/q)?-1(p/(p/q)w1(p/q)÷-÷f(f,w)21+(p/q)??ee((?1+(p/q)f(1+(p/q)f (ppqq)22fp/2-(p/q)p/]=(p+q)/2-1-w()(we (ppqq)22ùéù??p/G(p/2-1f?p p(p+q)/2-1wwe2 ()(G(p ?(1+(p/q)f)(p+q)/2p2q)qpq?2221w(p+q)/2-1e-w/()G(pp2q)q2isap?p?p/ p/2-Gf+qf(f,w)dw÷f(f)=2.()( èq(1+(p/q)f)(p+q)/pq22FGeorgeW.,0￡x<￥,p,q=p/2-xGp??p/=FGeorgeW.,0￡x<￥,p,q=p/2-xGp??p/= f(x|p,((è (1+(p/q)xGGqq22qq-EX,q>? ?p+q-,q>VarX=÷q-p(q-è?IfX~Fp,q,then1~Fq,p;thatis,thereciprocalofanFvariableisagainanFrandomIfX~tq,thenX2~F1,q;thatis,thesquareofatrandomvariableisanFrandomvariableIfX~Fp,q,then(p/q)X/(1+(p/q)X)~Beta(p/2,q/pdfandpdfandFdistributioninQuantilefunctionFdistributioninQuantilefunctionRandomOne- X-Sare~Nsn(n-1)S~;X-S ~Two-)~-One- X-Sare~Nsn(n-1)S~;X-S ~Two-)~-Y)-X-m+n-+S1m1np/S/SS2=~ m-1,n-/S/XYYYSamplingOther統(tǒng)計學(xué)方SamplingOther統(tǒng)計學(xué)方法及其應(yīng)統(tǒng)計學(xué)基隨機(jī)變量的函“ArandomvariableisaquantitywhosevaluesarerandomandtowhichaprobabilitydistributionOrderOrderX(1)OrderOrderX(1)=thesmallest X(1)=min1￡i￡n{Xi}X(2)=thesecondsmallestX(n)=thelargest X(n)=max1￡i￡n{Xi}Theorderstatisticsofarandomsample,X1, ,Xn,arethesamplevaluesplacedinascendingorder.TheyaredenotedbyX(1) ,X(n)SampleSampleThesampleisthedifferencebetweentheSampleSampleThesampleisthedifferencebetweentheandtheminimumvaluesinarandomdenotedX,X1nR=maxXi-minX=-(nSampleSampleThesamplemedian,usuallydenotedbyMisaSampleSampleThesamplemedian,usuallydenotedbyMisasuchthatlessthanapproximatelyone-halfoftheobservationsMandone-?ififisis=í((n(X(nM.?+X(n/2)?AsingleorderI(X ￡x)=Xi￡x>XiP(I(Xi￡x)AsingleorderI(X ￡x)=Xi￡x>XiP(I(Xi￡x)=1)=￡x)=FX(x).I(Xi￡x)inY=?I(Xi￡xYi?nn? (x)=P(X(k(k￡x)=P(Y3k=(x)(1n-iF(x F?iXXèi?nddn? (x)=dx (x)=(k (kF(x)(1-F n-i÷ièXiDistributionofasingleorderX(1),X denotetheorderstatisticsofDistributionofasingleorderX(1),X denotetheorderstatisticsofa)X,Xn,fromacontinuouspopulationcdf(x)andpdffX(x).ThenthepdfX(k(x)= n f(x)F(x)ùk-1-(xù-f.?(k-1)!(n-kX(kXXXOneuniformorderfX(x)=1,FX(=WhenXi~uniform(0,1),thatn(k-1)!(n-k)! (x)OneuniformorderfX(x)=1,FX(=WhenXi~uniform(0,1),thatn(k-1)!(n-k)! (x)k-(1-n-X(k (n = (1-xk(n-k-11G(k)G(n-kSofX(k)(x Beta(k,n-kkE ,n+(k=k(n-k+1)Var(n+1)2(n+(kParticularly,1n+nEmin =,Emax1￡i,n+ii1￡inVarmin =Varmax .(n+1)2(n+ii1￡i1￡iJointdistributionoftwoorderX(1),X denotetheorderstatisticsofa),fromacontinuouspopulationXJointdistributionoftwoorderX(1),X denotetheorderstatisticsofa),fromacontinuouspopulationX,XncdfFX(x)1￡i<j￡fX(x).ThenthejointpdfX(j)n(i-1)!(j-i-1)!(n-j)!=ffXXX(i),X(j(uùiùn-(u)ùj-i-1-F(v)-.?XXXXTwouniformorderLetTwouniformorderLetR= - andV= )/2,(n n- (n fR(r)=n(n- (1-r)=Beta(n-f(v)=né21-vùn1 When uniform(0,1),thatis,f(x)=1,F(x)=x,we (u,v) n ui-1(v-u)j-1-i(1-v)n-X(i),X(j (i-1)!(j-i-1)!(n-j= G(n+ ui-1(v-u)(j-i)-1(1-v)(n+1-j)-1.G(i)G(j-i)G(n+1-So (u,v Dir(i,j-i,n+1-j(i (jJointdistributionofallJointdistributionofallorderFromtheabovepdf,thejointcdf X ,X(n) , ,xn) x2 x (n fX(tndtnúfX(t2)dt2úfX(t1)dt ? tn- LetX(1),,X(n)denotetheorderstatisticsofarandomsample,X1,,Xn,fromacontinuouspopulationwithpdffX(x).ThenthejointpdfofX(1),,X(n)isfX,,X(x1,,xn)=n!fX(x1)fX(x2)fX(xn (nif-￥<x1<<xn<￥and0JointdistributionofuniformJointdistributionofuniformorderFromtheabovepdf,thejointcdfofX(1) ,X(n òx1 , ,xn)=n dtdt (n tn-LetX(1),,X(n)betheorderstatisticsofarandomsample,X1,,Xn,fromauniform(0,1)population.ThenfX(x)=1forx?[0,1]and0otherwise.Therefore,thejointpdfofX(1),,X(n)fX,,X(x1,,xn)=n (nif0￡x<<x￡1and0 ConvergenceinSupposethatConvergenceinSupposethatX1,X2, convergesinprobabilitytoarandomvariableXandthathisacontinuousfunction.Thenh(X1),h(X2),convergesinprobabilitytoh(X).AsequenceofrandomvariablesX ,Xn,convergesprobabilitytoarandomvariableXif,foreverye>0,limP(|X -X| 3e)=0, orlimP(|Xn -X ExampleLetthesampleuniformlydistributed.theclosedinterval[0,1]DefineExampleLetthesampleuniformlydistributed.theclosedinterval[0,1]DefinerandomX(s)=s,s?SDefinerandomXn(s)=s+s,s?SnX(s)-X(s)=s,s?SnnNow,foreverye>limP(|Xn(s)-X(s) < =limP(sn< =P(s?=Therefore, convergesinprobabilitytoXExampleLetthesampleuniformlytheclosedinterval[0,1]withExampleLetthesampleuniformlytheclosedinterval[0,1]withrandomX(s)=s,s?Srandom asnX1(s)=s+I[0,1](sX2(s)=s+I[0,1/2](s),X3(s)=s+I[1/2,1]X4(s)=s+I[0,1/3](s),X5(s)=s+I[1/3,2/3](s),X=+I[2Then,foreverye>limP(|Xn(s)-X(s)<k])< =limP(length(I[0,1 =Therefore, convergesinprobabilitytoXAlmostsure (ConvergencewithprobabilityAlmostsure (Convergencewithprobability AlmostsureAlmostsureconvergeismuchstrongerthanconvergesinprobability.Almostsureconvergeimpliesconvergesinprobability.AsequenceofrandomvariablesX1, ,Xn,convergesalmostsurelytoarandomvariableX if,foreverye>0,P(lim|Xn-X <e)= ExampleLetthesamplespaceuniformlydistributed.Definerandomtheclosed[0,1]withExampleLetthesamplespaceuniformlydistributed.Definerandomtheclosed[0,1]withX(s)=s,s?SDefinerandomXn(s)=s+s,s?SnForeverys?asXn(s s=X(sHowever,fors=X(s)=1+ nnP(s=1)=0andP(s?[,) convergesalmostsurelytoX=ExampleLetthesampleuniformlyExampleLetthesampleuniformly theclosedinterval[0,1]randomX(s)=s,s?SrandomvariablesXn asX1(s)=s+I[0,1](sX2(s)=s+I[0,1/2](s),X3(s)=s+I[1/2,1]X4(s)=s+I[0,1/3](s),X5(s)=s+I[1/3,2/3](s),X6(s)=s+I[2/3,1](sThen,foreverys?S,thevalueXn(s)alternatesbetweenthevaluesands+1infinitelyoften.Therefore,thereisnovalueofs?SforXn s=X(s).InotheraltuhoughXn convergesinprobabilitytoXdoesNOTconvergealmostsurelytoXConvergenceinConvergenceinConvergenceinConvergenceinIfthesequenceofrandomvariables,X1,X2,convergesinprobabilitytoarandomvariableX,thesequencealsoconvergesindistributiontoX.AsequenceofrandomvariablesX1, ,Xn,convergesindistributiontoarandomvariableX foreverye>lim =FX(x atallpoints whereFX(x)iscontinuous.X(n)=LetX1,X2X(nX(n)=LetX1,X2X(niiduniform(0,1)randomvariables.Xigetscloseto1,butmustnecessarilybelessAs-13e)=P)+e)+P(X(n￡1-P(|X(n=P(X(n)￡1-=P ￡1-P(X(n)￡1- =P(Xi￡1-e,i=1,?,n=(1-0,asX(nconvergesto1inprobabilitExampleFurthermore,lete=t/n,PExampleFurthermore,lete=t/n,P(X(n)￡1-t/n)=(1-t/n)thatisne-P(n(1-X(n))￡Recalltheexponentialf(x)=e-1-e-x=1- F(x)=x0-e-tdt=e-n(1-X(n))convergesindistributionanexponentialrandom￥Eg(X￥Eg(X =￥g(x)f{x{x=rP(g(X)3rLetXbearandomvariableandletg(x)beanonnegativefunction.Thenforanyr>0P(g(X)3r)￡Eg(X)rWeaklawoflargenumbersLetX,?,beiidrandomvariableswithE =m =<1nii?niWeaklawoflargenumbersLetX,?,beiidrandomvariableswithE =m =<1nii?niDefine =(1/X.niE( -VarP(| -m3e)=P(|-m|23e2)==n .nnChebychev'sP(|Xn-m<e)=1-P(|Xn-3e)31asInotherlimP(|Xn-m WeaklawoflargeSampleWeaklawoflargeSamplemeanbecomespopulationmeanwhenthesamplesizetendstoLetX1,?,Xn beiidrandomvariableswithEXi=mandVarX=s2<￥.DefineX =(1/n)?n X.Then, i foreverye>limP(| -m thatis, convergesinprobabilitytonAsimulationstudyAsimulationstudyoft<-x<-rnorm(t,0,m<-n<-while(n<m<-c(m,mean(x[1:n]));n=n*2;}MonteCarlo10=1p41ò1-x1-x+arcsin=2011òMonteCarlo10=1p41ò1-x1-x+arcsin=2011ò1-x2dx=1-x2?100p(xf(x1nny1-x?1-x21òi1-x2i0whereeveryxiissampledfromauniform(0,1)? limP?n?f(Xi)-Ep(x)f(X)<e÷ Ep(x)f(X)?n?f(Xi),as i i Ep(x)ê?f(X)ú?=-￥f(x)p(x òh(x)dx=ò-￥f(x)p(x)dx?n?f(x- i=MonteCarloConvergenceofsampleWeaklawoflargenumbersforsampleE = X1,?,XnbeiidrandomandConvergenceofsampleWeaklawoflargenumbersforsampleE = X1,?,Xnbe