Tikhonov吉洪諾夫正則化

TikhonovregularizationFromWikipedia,thefreeencyclopediaTikhonovregularizationisthemostcommonlyusedmethodofofnamedfor.In,themethodisalsoknownasridgeregression.Itisrelatedtothe forproblems.ThestandardapproachtosolveanofgivenasAx=b,isknownasandseekstominimizetheAx一b2where?isthe.However,thematrixmaybeoryieldinganon-uniquesolution.Inordertogivepreferencetoaparticularsolutionwithdesirableproperties,theregularizationtermisincludedinthisminimization:Ax一b2+lirxll2forsomesuitablychosenTikhonovmatrix,r.Inmanycases,thismatrixischosenasther=，,givingpreferencetosolutionswithsmallernorms.Inothercases,operators.,aoraweighted)maybeusedtoenforcesmoothnessiftheunderlyingvectorisbelievedtobemostlycontinuous.Thisregularizationimprovestheconditioningoftheproblem,thusenablinganumericalsolution.Anexplicitsolution,denotedby」,isgivenby:ATbATbTheeffectofregularizationmaybevariedviathescaleofmatrix r.ForraI,whena=Othisreducestotheunregularizedleastsquaressolutionprovidedthat(ATA)-1exists.ContentsBayesianinterpretationAlthoughatfirstthechoiceofthesolutiontothisregularizedproblemmaylookartificial,andindeedthematrixrseemsratherarbitrary,theprocesscanbejustifiedfroma.Notethatforanill-posedproblemonemustnecessarilyintroducesomeadditionalassumptionsinordertogetastablesolution.Statisticallywemightassumethatweknowthatxisarandomvariablewitha.Forsimplicitywetakethemeantobezeroandassumethateachcomponentisindependentwith^x.Ourdataisalsosubjecttoerrors,andwetaketheerrorsinbtobealso withzeromeanandstandarddeviation °”UndertheseassumptionstheTikhonov-regularizedsolutionisthesolutiongiventhedataandtheaprioridistributionof^,accordingto.TheTikhonovmatrixisthen r=a/forTikhonovfactora=°匕/°xIftheassumptionofisreplacedbyassumptionsofanduncorrelatednessof,andstillassumezeromean,thentheentailsthatthesolutionisminimal.GeneralizedTikhonovregularizationForgeneralmultivariatenormaldistributionsforxandthedataerror,onecanapplyatransformationofthevariablestoreducetothecaseabove.Equivalently,onecanseekanxtominimize

Ax-b2+x-xp02Qwherewehaveused||x112tostandfortheweightednormPBayesianinterpretationPistheinverse ofb,x0isthexTPx(cf.the).Intheofx,andQistheinversecovariancematrixofxxTPx(cf.the).Intheofx,andQistheThisgeneralizedproblemcanbesolvedexplicitlyusingtheformula0-Ax)00[]RegularizationinHilbertspaceTypicallydiscretelinearill-conditionedproblemsresultasdiscretizationof,andonecanformulateTikhonovregularizationintheoriginalinfinitedimensionalcontext.IntheabovewecaninterpretAasaon,andxandbaselementsithedomainandrangeof^.TheoperatorA*A+rtristhena boundedinvertibleoperator.RelationtosingularvaluedecompositionandWienerfilterWithr=a',thisleastsquaressolutioncanb(the.GiventhesingularvaluedecompositionofAA=UYVtwithsingularvalues°”theTikhonovregularizedsolutioncanbeexpressedas

x=VDUTbwhereDhasdiagonalvaluesDiib= ib2+a2iandiszeroelsewhere.ThisdemonstratestheeffectoftheTikhonovparameterontheoftheregularizedproblem.Forthegeneralizedcaseasimilarrepresentationcanbederivedusinga.Finally,itisrelatedtothe:uTbi=1ii=1biib2wheretheWienerweightsaref=iandQisthe ofA.ib2+a2iDeterminationoftheTikhonovfactorTheoptimalregularizationparameteraisusuallyunknownandofteninpracticalproblemsisdeterminedbyanadhocmethod.ApossibleapproachreliesontheBayesianinterpretationdescribedabove.Otherapproachesincludethe,,,vedthattheoptimalparameter,inthesenseofminimizes:RSSG= RSSG= T2XGtX+；21)1XTwhereRSSisthe andTistheeffectivenumber.UsingthepreviousSVDdecomposition,wecansimplifytheaboveexpression:

andRSS=F另Cb12+andRSS=F另Cb12+工RSS二RSS0a2Cb)G2+a2iii=1iCb)Eg2… ig2+a2i=1 iEa2± g2+a2i=1 iRelationtoprobabilisticformulationTheprobabilisticformulationofanintroduces(whenalluncertaintiesareGaussian)acovariancematrixCMrepresentingtheaprioriuncertaintiesonthemodelparameters,andacovariancematrixCDrepresentingtheuncertaintieson':-:':-:''[.Jandwhenthesetwomatricesarediagonalandisotropic,equationsabove,withHistoryTikhonovregularizationhasbeeninventedindependentlyinmanydifferentcontexts.ItbecamewidelyknownfromitsapplicationtointegralequationsfromtheworkofandD.L.Phillips.SomeauthorsusethetermTikhonov-Phillipsregularization.ThefinitedimensionalcasewasexpoundedbyA.E.Hoerl,whotookastatisticalapproach,andbyM.Foster,whointerpretedthismethodasa-filter.FollowingHoerl,itisknowninthestatisticalliteratureasridgeregression.[]References(1943)."O6ycTO訪TUBOCTuo6paTHbix3agaq[Onthestabilityofinverseproblems]".39(5):195-198.Tychonoff,A.N.(1963)."OpemeHuuHeKoppeKTHonocTaB“eHHbix3agaquMeTogeperyn刃pu3aquu[Solutionofincorrectlyformulatedproblemsandtheregularizationmethod]".DokladyAkademiiNaukSSSR151:501-504..TranslatedinSovietMathematics4:1035-1038.Tychonoff,A.N.;V.Y.Arsenin(1977).SolutionofIll-posedProblems.Washington:Winston&Sons..Hansen,.,1998,Rank-deficientandDiscreteill-posedproblems,SIAMHoerlAE,1962,Applicationofridgeanalysistoregressionproblems,ChemicalEngineeringProgress,58,54-59.FosterM,1961,AnapplicationoftheWiener-Kolmogorovsmoothingtheorytomatrixinversion,J.SIAM,9,387-392PhillipsDL,1962,At