第四章：信道與信道容量(英文)

Chapter4：ChannelandChannelCapacity

ThegoaltoachieveTheunderstandingofthegoalandcontenthowInfo.TheorystudiesthechannelUnderstandthebasicclassificationofthechannelandgraspthebasicdescriptionmethodofthechannelGrasptheconceptofchannelcapacity/Channelcapacitypricefunction,aswellasrelationsbetweenthisconceptandmutualinfo.,channelinputprobabilitydistribution,channeltransferfunctionCancalculatethechannelcapacity/Thechannelcapacitypricefunction(symmetricaldiscretechannel,additiveGaussnoisechannelwithoutmemory)ofsimplechannelUnderstandthefunctionofthechannelcapacity/CapacitypricefunctionincommunicationssystemresearchDefinitionquestionEntropyEntropyratefunctioninlosslesssourcecodingtheoremMutualinfo.ChannelCapacityfunctioninsourcecodingtheoremReview–natureofmutualinfo.function:Nature.1Relationshipbetweenmutualinfo.andchannelinputprobabilitydistributionNature1:I(X;Y)isaconvexfunctionofthechannelinputprobabilitydistributionp(x).I(X;Y)p(x)Review–natureofmutualinfo.function:Nature.2

RelationshipbetweenInfo.contentandchanneltransitionprobabilitydistribution

Nature2:I(X;Y)isaconcavefunctionofchanneltransitionprobabilitydistributesp(y/X).I(X;Y)p(y/x)Review–natureofmutualinfo.function:Nature.3RelationshipbetweenInfo.contentandchannelinputmarkrelevancy

Nature3:ThechannelinputisdiscreteandwithoutmemoryReview–natureofmutualinfo.function:Nature.4

RelationshipbetweenInfo.contentandchannelinputmarkrelevancyNature4:ThechannelhasnomemoryReview–natureofmutualinfo.function:Nature.5Deductionofnature3andnature4:

Thechannelinputandthechannelthemselvesarealldiscreteandwithoutmemory.ChannelandChannelCapacityOutlineclassificationanddescriptionofthechannelThediscretechannelwithoutmemoryanditsthecapacityContinualchannelanditscapacityCapacitypricefunctionC(F)

§4.1:OutlineThecontentofInfo.TheoryresearchingonthechannelWhat’schannel？ThefunctionofchannelThegoalofresearchingthechannel§5.1:Outline－1ContentthattheInfo.Theorystudiesonthechannel:Channelmodeling:describewithtwoappropriateinput/outputsstochasticprocessesChannelcapacityvariousmeansoftakingfullyadvantageofthechannelcapacityunderdissimilarcondition§5.1:Outline－2Whatischannel? Thechannelisacarrierwhichtransmitsmessages——apassagewhichsignalpassesthrough. Theinformationisabstract,thechannelthenisconcrete.Forinstance:Iftwopeopleconverse,theairisthechannel;Ifthetwocalleachother,thetelephonelineisthechannel;Ifwewatchthetelevision,listentotheradio,thespacetoreceiveandsendisthechannel.§5.1:Outline－3Thefunctionofchannel Thechannelmainlybeusedintransmittingandstoringdatatheinfo.system,butincommunicationssystemmainlybeusedintransmitting.§5.1:Outline－4thegoaltoresearchchannelRealizevalidityandreliabilityofintelligencetransmissionValidity:FullyusethechannelcapacityReliability:Reducestheerrorratethroughthechannelcoding

Studyingthechannelinthecommunicationssystemismainlytodescribe,measure,analyzedifferenttypesofchannel,calculatetheircapacity,namely,thelimittransmissioncapacity,andanalyzetheircharacteristic.Thecommunicationtechnologymainlyresearches－－Thephysicalrulewhichthesignalfollowswhiletransmittinginthechannel,namelytransmissioncharacteristicTheInfo.Theorymainlyresearches－－info.transmissionquestion(hypothesisisthatthetransmissioncharacteristicsareknown)§4.2:ClassificationanddescriptionofthechannelClassificationDescription§4.2:Classificationanddescriptionofchannel－1classification projectphysics——transmissionmediumtype; mathematicsdescriptionway——descriptionwayofsignalanddisturbance;parametertypeofthechannelitself——changeableandpermanentparameter;usertype——singleuserandmulti-user; §4.2:Classificationanddescriptionofchannel－2§4.2:Classificationanddescriptionofchannel－3§4.2:Classificationanddescriptionofchannel－4§4.2:Classificationanddescriptionofchannel－5§4.2:Classificationanddescriptionofchannel－6DescriptionofchannelThechannelmayquotethreegroupsofvariablestodescribe:Channelinputprobabilityspace：Channeloutputprobabilityspace：Channelprobabilityshiftmatrix：PNamely：{P}，Itcanbesimplified：.§4.2:Classificationanddescriptionofchannel－7其中：而 而§4.2:Classificationanddescriptionofchannel－8When

K=1，degeneratestothe

singlemessage(mark)channel;Furtherwhen

n=m=2，degeneratestothebinarysinglemessagechannel.Ifitsatisfiedsymmetry,namelyconstitutesmostcommonlyusedbinarysinglemessagesymmetricalchannelBSC:And:,,§4.3:DiscretechannelwithoutmemoryanditschannelcapacityDiscretechannelwithoutmemoryanditschannelcapacityCalculationofchannelcapacityofthediscretechannelwithoutmemoryChannelcapacitytheoremofthediscretechannelwithoutmemoryChannelcapacityoftheSymmetricdiscretechannelwithoutmemoryPhysicalsignificanceofShannonFirstTheorem§4.3:Discretechannelwithoutmemoryanditschannelcapacity-1Channelofdiscretemessagesequence§4.3:Discretechannelwithoutmemoryanditschannelcapacity-2Discretechannelwithoutmemoryanditschannelcapacity

Accordingtothecharacteristicofthemutualinfo.ofthemessagesequence,tothediscretechannelwithoutmemory:（Nature4）So:

Onlywhenthesourcewithoutmemory,thepreviousformulais“equivalence”（Cor.ofNature3、4）

§4.3:Discretechannelwithoutmemoryanditschannelcapacity-3FurtherunderstandingExistenceofCmaxMutualinfo.nature1，thelimitvalueoftheconvexfunctionexisted.TwoconditionstoreachCmax：ThesourceisdiscreteandwithoutmemoryTheprobabilitydistributionofthechannelinputisthedistributiontomakeI(X,Y)maximum.ThevalueofCisnotdeterminedbytheP(x)ofthesource,butdeterminedbyP.Cistheperformancemeasurementofthechannelwhichisthepassageofinfo.transmission.OnlywhenthesourceX（x1x2…xn)satisfiedcertainconditions，itcanfullyusetheabilityoftheinfo.transmission.§4.3:Discretechannelwithoutmemoryanditschannelcapacity-4ComputationofthechannelcapacityofthediscretechannelwithoutmemoryMentality:Thequestioncanbetransformedto:togettherestraintextremevalueonaclosedregionMethod:1st,togettheextremevalueintheregion2nd,togetextremevalueoftheboundary3rd,togetthemaximumvaluesofprevioustwoConcreterealization：1、solvesinthesimplesituation(forexamplesinglemarkchannel,symmetricalchannel)2、soluteequation3、iterate4、others§4.3:Discretechannelwithoutmemoryanditschannelcapacity-5Channelcapacitytheoremofdiscretechannelwithoutmemory

Theorem5.1：Tothediscretechannelwithoutmemorywhosepre-transitionprobabilitymatrixisQ,theabundantandnecessaryconditionthatitsinputletterprobabilitydistributionp*cancausethemutualinfo.I(p,Q)takethemaximumvalueis:

Note：

istheaveragemutualinfo.whichisthesourceletteraktransmits，Cisthechannelcapacityofthischannel.

§4.3:Discretechannelwithoutmemoryanditschannelcapacity-6UnderstandingofthetheoremUnderthiskindofdistribution,eachletterwhoseprobability>0providesmutualinformation=C,eachletterwhoseprobability=0providesmutuallyinformation≤COnlywhenunderthiskindofdistribution,maycauseI(p,Q)obtainthemaximumvalueCI(X,Y)istheaverageofI(x=ak;Y).Namely:WantstoenhanceI(X,Y)，mayenhancep(ak)Butifenhancep(ak)，mayreduceI(x=ak;Y)Adjustrepeatedlyp(ak),makeI(x=ak;Y)allequaltoCThistime:I(X,Y)＝CThetheoremonlyprovidestheabundantandnecessaryconditionofp(x)tomakeI(X,Y)＝C.ItdoesnothavetheconcretedistributionandtheCvalue,butmayhelptogettheCvalueofpartialchannelsinsimplesituation§4.3:Discretechannelwithoutmemoryanditschannelcapacity-7ChannelcapacityofthesymmetricaldiscretechannelwithoutmemorysymmetricaldiscretechannelwithoutmemoryTheoutputlettersetmaybedividedintocertainsubsets,toeachsubset:Inthematrix,eachlineistherearrangementofthefirstline;

Inthematrix,eachrowistherearrangementofthefirstrow.Theorem5.2：Asforthesymmetricaldiscretechannelwithoutmemory,whenthechannelinputlettersatisfiedequalprobabilitydistribution,itwillachievethechannelcapacity.§4.3:Discretechannelwithoutmemoryanditschannelcapacity-8Symmetricalchannel§4.3:Discretechannelwithoutmemoryanditschannelcapacity-9a1a2b1b2b30。70。10。10。20。7a1a2b1b2b30。20。70。70。10。10。2§4.3:Discretechannelwithoutmemoryanditschannelcapacity-10ComputationoftheBSCchannelcapacitya1a2b1b21-ε1-εεε§4.3:Discretechannelwithoutmemoryanditschannelcapacity-11AccordingtoTheorem5.2，whentheinputsatisfiedequalprobabilitydistribution,themutualinfo.willachievethechannelcapacitynamely：whenp(a1)=p(a2)=1/2；

so：

note：ApplicationExample3.2(18)、3.6(23)§4.3:Discretechannelwithoutmemoryanditschannelcapacity-12computationofthechannelcapacityofdualdeletionchannel

a1a2b1b21-ε1-εεεb3§4.3:Discretechannelwithoutmemoryanditschannelcapacity-13AccordingtoTheorem5.2，whentheinputsatisfiedequalprobabilitydistribution,themutualinfo.willachievethechannelcapacitynamely：whenp(a1)=p(a2)=1/2；

so：

§4.3:Discretechannelwithoutmemoryanditschannelcapacity-140.51.000.51.0cεbaCa=Cb=a:BSC信道的信道容量曲線b:二進(jìn)制刪除信道的信道容量曲線

TogetthechannelcapacityofsymmetricaldiscretematrixP:1/21/31/6P=1/61/21/31/31/61/2

C=logs-H(p1,p2,p3)=log3-H(1/2,1/3,1/6)=log3+1/2log1/2+1/3log1/3+1/6log1/6=1.126bit/sTheresultindicatedthat,Onlywhentheinputsatisfiedequalprobabilitydistribution,thechannelcapacityachievesmaximumvalue.Theaveragegreatestinfo.contentofeachmarktransmissionis1.126bit.1losslesschannel一個(gè)輸入對(duì)多個(gè)互不相交的輸出,因?yàn)閾p失熵H(X/Y)=0故I(X,Y)=H(X)C=logrr為輸入個(gè)數(shù)2definitechannel一個(gè)輸出對(duì)多個(gè)互不相交的輸入,因?yàn)樵肼曥豀(Y/X)=0故I(X,Y)=H(Y)C=logss為輸出個(gè)數(shù)3losslessdefinitechannel一個(gè)輸入對(duì)一個(gè)輸出,因?yàn)閾p失熵H(X/Y)=0噪聲熵H(Y/X)=0故I(X,Y)=H(X)=H(Y)C=logrr為輸入個(gè)數(shù)Discretenoiselesschannel§4.3:Discretechannelwithoutmemoryanditschannelcapacity-15physicssignificanceoftheShannonfirsttheorem:（lengthuncertainlosslesssourcecoding）

（equalsignalestablisheswhenachievedlimit）

astothechannel,thechannelinformationtransmissibility:

（equalsignalestablisheswhenachievedlimit）§4.3:Discretechannelwithoutmemoryanditschannelcapacity-16physicssignificanceoftheShannonfirsttheorem:（lengthuncertainlosslesssourcecoding）Channelcapacityofnoiselessandlosslesschannel：C=logMNowwhentheaveragecodelengthhasextremevalue:ChannelinformationtransmissibilityR=channelcapacityCofthenoiselesschanneltheessenceoflosslesssourcecoding：對(duì)離散信源進(jìn)行適當(dāng)變換，使變換后新的碼符號(hào)信源（信道的輸入信源）盡可能為等概分布，以使新信源的每個(gè)碼符號(hào)平均所含的信息量達(dá)到最大，從而使信道的信息傳輸率R達(dá)到信道容量C，實(shí)現(xiàn)信源與信道理想的統(tǒng)計(jì)匹配。Alsocalled：noiselesschannelcodingtheorem若信道的信息傳輸率R不大于信道容量C，總能對(duì)信源的輸出進(jìn)行適當(dāng)?shù)木幋a，使得在無噪無損信道上能無差錯(cuò)地以最大信息傳輸率C傳輸信息；但要使信道的信息傳輸率R大于C而無差錯(cuò)地傳輸信息則是不可能的?！?.5:信道容量InformationsourceEncoderSXDiscreteCommunicationChannelX={x1,x2,…,xr}DecoderXSInformationreceiverContinuouschannel－－analogchannelContinuouschannel：Char1：thetimeisdiscrete、thescopeiscontinuousChar2：ateachmoment,itisthesinglerandomvariablewhosevalueiscontinuous(vsdiscretesequence)Researchmethod：randomvariableofNfreedomdegree,takestoresearchConaverageeachfreedomdegreeanalogchannel:Char1：thetimeiscontinuous、thescopeiscontinuousChar2：Agroupoftimesamplefunctions,ateachmoment,thetimeandthescopevalueofthesamplefunctionareallcontinualResearchmethod：1、discretewhenthefrequencyandthetimearelimited,canbequantifiedtothediscreterandomvector2、toavoidthedifficultyofresearchingtherandomvectorhavingmemory,findagroupofcompletelyorthogonalfunctionsets,launchesfortheprogression,enabletomaketherandomvectorcomposedbythecoefficientbeindependentorbelinearindependent.Note：1、timelimit--frequencyspectruminfinite,frequencylimit--timeinfinite.2、assumethatthefunctionvalueisverysmalloutsideForT,andthatthetimeandthefrequencyislimitedwillnotcauseseriousdistortionAnalogsourceAnalogcommunicationsystemSourcecodingChannelcodingAnalogchannelChanneldecodingSourcedecodingDestinationA/DconverterModulationDemodulationD/Aconverter01101…01110010…01111010…01101…§5.4:Continuouschannelanditscapacity－1－ReviewEntropyofcontinuousrandomvariable－differentialentropy（VSdiscreterandomvariable）Thecontinuousrandomvariablebiggestentropydistribution--reliesontheconstraintcondition(VSdiscreterandomvariable)Thepeakpowerislimited--theuniformdistributionrandomvariablewillhavethebiggestdifferentialentropyTheaveragepowerislimited--theGaussdistributionrandomvariablewillhavethebiggestdifferentialentropyThevalueterritoryofthecontinualchannelinputisinsufficienttoexpressthelimittothechannelalsotheconstraintconditionC＝max[h(Y)-h(n)]C取決于信道的統(tǒng)計(jì)特性（加性信道即噪聲的統(tǒng)計(jì)特性）輸入隨機(jī)矢量X所受的限制條件（一般考慮平均功率受限時(shí)）Unit：bit/N個(gè)自由度Channelcapacityofthecontinualchannel--capacityexpensefunctiondescription§5.4:Continuouschannelanditscapacity－2C.FWu&Zhu&Fu－－channelcapacityWu：Channelcapacity：discrete、continuousCapacitycostfunction：discretechannel、continuouschannelZhu：Channelcapacity：discreteCapacitycostfunction：continuouschannel&continuouschannelFu：Channelcapacity：discrete、continuous§5.4:Continuouschannelanditscapacity－3Methodstoresearchcontinuouschannelcapacitybasic、simplechannel：addablenoisechannelwithoutmemoryWhenthechannelnoiseisGaussWhichdistributioninputwilltakefullyadvantageofthechannelWhenthechannelinputisGaussWhichdistributionnoisewillaffectthechannelinfo.transmissionmost§5.4:Continuouschannelanditscapacity－4Basicknowledge：toaddablechannelY=X+NX：channelinputN：channelnoiseY：channeloutputThechanneltransitionprobabilitydistributionfunctionistheNdistributionfunctionb(x)isthecorrespondexpensewhenthechannelinputisxIftwoofX,Y,NistheGaussdistribution,thenthenotheralsoistheGaussdistributionDifferentialentropyoftherandomvariablesatisfiesGaussdistributionh(XG)＝Thevalueofdifferentialentropyh(XG)onlyconcernswiththevariance ,hasnothingtodowiththeaveragevalueAddablechannelErrorSource+EXOutputInput§5.5:AnalogchannelanditscapacityChannelcapacityexpensefunctionoftheanalogchannelanditscomputation:Generalizedsteadylimitedfrequency(F),limitedtime(T),limitedpower(P)whiteGausschannelanditscapacityCShannon

formula

PhysicalsignificanceofShannon

formulaUsageofShannon

formula§5.5:Analogchannelanditscapacity－4Generalizedsteadylimitedfrequency(F),limitedtime(T),limitedpower(P)whiteGausschannelanditscapacityC

Tothecontinuousprocesssourcewhichhaslimitedfrequency(F),limitedtime(T)maybeunfoldedthefollowingsamplingfunctionsequence:Nowletthese2FTsamplesvaluesequencepassthroughthewhiteGausschannelwhichhaspowerlimited(P)andgetitscapacityC.

§5.5:Analogchannelanditscapacity－Shannon

formula1ShannonformulaTheorem5.3：WhenageneralsteadyrandomprocessX(t,w)sourcewhichsatisfieslimitedfrequency(F)andtime(T)passesthroughawhiteGausschannelwhichhaslimitedpower(P),itscapacityis:ThisisthefamousShannonformula.WhenT=1,thecapacityis:§5.5:Analogchannelanditscapacity－Shannonformula2prove：wehadgotthecapacityofthesinglecontinuousmessage(NO.k)whenpassingthroughtheGausschannel:Meanwhile，wehadprovedthatwhenthesourceandthechannelhasnomemoryinthemutualinfo.,thefollowingformulawillbetenable:Accordingtothedef.ofthechannelcapacity:§5.5:Analogchannelanditscapacity－Shannon

formula3PhysicalsignificanceoftheShannonformulaItpresentsthedialecticalrelationsamongthethreesignalphysicsparameterswhichdeterminedthechannelcapacityC:F、T、。Productofthethreeisa“moldable"volume(threedimensional).Thethreemayexchange.§5.5:Analogchannelanditscapacity－Shannonformula4－usageofShannonformula:1

Usethefrequencyband

inexchangeforthesignalnoiseratio:Widenfrequencycorrespondenceprinciple.Intheradarsignaldesign,thereislinearlyfrequencymodulatedpulse.Intheanalogcommunications,thefrequencymodulationsurpassestheamplitudemodulation,andthewiderthefrequencybandis,thestrongertheanti-disturbanceis.Inthedigitalcommunication,thepseudo-code(PN)straightlyexpandswiththetimefrequencycode.Thewiderthebandwidthis,themorethewidenfrequencyincreases,andthestrongertheanti-disturbanceis.Inthedeepspacecorrespondence(powerenergyislimited,frequencyspectrumresourcesisrelativelyrich),weusetwolevelsdigitalcommunicationwaytoeffectivelyusethechannelcapacity.

Attention：thereislimit歸一化信道容量關(guān)于帶寬W的關(guān)系圖歸一化信道帶寬關(guān)于信噪比SNR的關(guān)系圖

－anotherformofShannonformula：

其中，為噪聲密度，即單位帶寬的噪聲強(qiáng)度，σ2=N0F;

Eb

表示單位符號(hào)信號(hào)的能量，Eb=STb=S/F；Eb/N0

稱為歸一化信噪比.也稱為能量信噪比.WhenEb/N0<<1，

≈Eb/N0

(nat)＝(bit)Conclusion：whenthesignalnoiseratioislow,thechannelcapacityisapproximatelydeterminedbythepowersignalnoiseratio.§5.5:Analogchannelanditscapacity－Shannonformula

5－usageofShannonformula:2usethesignalnoiseratioinexchangeforthefrequencybrandbasicprincipleofmultiplesystem,multi-levels,multi-dimensionalconstellationmodulationwayThatbeusedInthesatellite,thedigitalmicrowaveoftenincluding:

themulti-levelsmodulation,theheterogeneitymodulation,thehighdimensionalconstellationmodulation(M-QAM)andsoon.Itusethewealthysignalnoiseratiointhehighqualitychanneltoexchangeforthefrequencyband,thustoenhancethetransmissionvalidity.

§5.5:Analogchannelanditscapacity－Shannonformula6usageofShannonformula:3usethetimetoexchangeforsignalnoiseratio重傳、弱信號(hào)累積接收基于這一原理。t=T0istheboundary.信號(hào)功率S有規(guī)律隨時(shí)間線性增長，噪聲功率σ2無規(guī)律，隨時(shí)間呈均方根增長?！?.5:Analogchannelanditscapacity－Shannonformula7usageofShannonformula:4

usethetimetoexchangeforthefrequencybandorusethefrequencybandtoexchangeforthetime:

Thewidenfrequency--reducesthetime:electroniccountermeasureofcorrespondence,submarinecorrespondenceThenarrowband--increasesthetime:Thetelephonechannelpassesontheaccuratemovingpicture§5.5:Analogchannelanditscapacity－Shannonformula

8Thegoaltodiscusschannelcapacityandthechannelexpensefunction：Nottorealizethereliabletransmission（whichisthegoalofthechannelcoding）OnlytorealizethemosttransmissionabilityofthechannelMaypresenttheboundaryofthechannelcodingAbouttheShannon

formulathechannelisaddablewhiteGaussnoise（AWGN)PresenttherelationshipoftheS、N、Wandthechannelcapacity（thebiggesttransmitspeed）NotpresenttherelationshipofS、N、Wandthedifferenceprobability－1.59dB例2、在圖片傳輸中，每幀約為2.25×106個(gè)像素，為了能很好地重現(xiàn)圖像，需分16個(gè)亮度電平，并假設(shè)亮度電平等概率分布。試計(jì)算每秒鐘傳送30幀圖片所需信道的帶寬(信噪功率比為30dB)。信噪比（S/N）通常用分貝（dB）表示，分貝數(shù)=10×log10（S/N）

解：高斯白噪聲加性信道單位時(shí)間的信道容量：(比特/秒）

要求的信息傳輸率為：Ct=2.25×106×log16×30=2.7×108（bit/s)=Wlog(1+S/N)而：10lg(S/N)=30dBS/N=103W=(2.7×108)/log(1+103)≈2.7×107(HZ)

貓速度和寬帶的解釋舉例3.8(26)曹志剛《現(xiàn)代通信原理》關(guān)于香農(nóng)公式的一些結(jié)論Channelcapacity：Thegreatestinformationcontentthatcanbetransmittedin