RDMA RoCEv2快速掌握課件

RDMARoCEv2

快速掌握

furtiger

于2020

什么是RDMA/RoCEv2RDMA(RemoteDirectMemoryAccess)技術(shù)全稱遠程直接內(nèi)存訪問，就是為了解決網(wǎng)絡(luò)傳輸中服務(wù)器端數(shù)據(jù)處理的延遲而產(chǎn)生的。RDMA技術(shù)由于其低時延、大帶寬的特性，已經(jīng)越來越多地被應(yīng)用于現(xiàn)代數(shù)據(jù)中心。所以，掌握RDMA相關(guān)技術(shù)，是IT/通信/學(xué)術(shù)/企業(yè)相關(guān)人士的必要技能之一。packetHCA：Thechanneladapterforaprocessornodeiscalleda“hostchanneladapter”(HCA)TCA：

achanneladapterinanI/Onodeisa“targetchanneladapter”(TCA).QP：Aconsumercommunicatesthroughoneormore“queuepairs”(QP).AnHCAtypicallysupportshundredsorthousandsofQPswhileaTCAmightsupportlessthantenQPs.LRH-LocalRoutingHeaderGRH-GlobalRoutingHeaderBTH-BasicTransportHeaderRETH-RDMAExtendedTransportHeaderRDMAExtendedTransportHeader(RETH)containstheadditionaltransportfieldsforRDMAoperations.TheRETHispresentinonlythefirst(oronly)packetofanRDMARequestasindicatedbytheBaseTransportHeaderOpcodefield.AETH-AckExtendedTransportHeaderACKExtendedTransportHeader(AETH)containstheadditionaltransportfieldsforACKpackets.TheAETHisonlyinAcknowledge,RDMAREADResponseFirst,RDMAREADResponseLast,andRDMAREADResponseOnlypacketsasindicatedbytheBaseTransportHeaderOpcodefield.AtomicETH-AtomicExtendedTransportHeaderAtomicExtendedTransportHeader(AtomicETH)containstheadditionaltransportfieldsforAtomicpackets.TheAtomicETHisonlyinAtomicpacketsasindicatedbytheBaseTransportHeaderOpcodefield.AtomicAckETH-AtomicAckETHAtomicACKExtendedTransportHeader(AtomicAckETH)containstheadditionaltransportfieldsforAtomicACKpackets.TheAtomicAckETHisonlyinAtomicAcknowledgepacketsasindicatedbytheBaseTransportHeaderOpcodefield.DETH-DatagramETHDatagramExtendedTransportHeader(DETH)containstheadditionaltransportfieldsfordatagramservice.TheDETHisonlyindatagrampacketsifindicatedbytheBaseTransportHeaderOpcodefield.Packet類型IBATransportServiceThefourIBAtransportservicetypesare:1)ReliableConnection2)ReliableDatagram3)UnreliableDatagram4)UnreliableConnectionThenon-IBAprotocolencapsulationservicesare:1)RawIPv6Datagram2)RawEthertypeDatagramSENDSEND操作，相當(dāng)于發(fā)送消息，由接收方?jīng)Q定消息的放置。Therespondernode(thedestinationoftheSENDOperation)doesnotknowthefinallengthoftheSENDmessageuntilthelastpacketwiththe“SENDLast”or“SENDLastwithImmediate”O(jiān)p-Codearrives.WRITETheRDMAWRITEOperationisusedbytherequestingnodetowriteintothevirtualaddressspaceofadestinationnode.TheRETHheaderispresentinthefirst(oronly)packetofthemessage.ItcontainsthevirtualaddressofthedestinationbufferaswellastheR_Keyandmessagelengthfields.Foragivenrequestingnode’sQP,onceamulti-packetRDMAWRITEoperationisstarted,nootherrequestpacketsmaybegenerateduntilthe“RDMALast”or“RDMALastwithImmediateData”packetissent.READRDMAREADOperationsaresimilartoRDMAWRITEOperations.Theyallowtherequestingnodetoreadavirtuallycontiguousblockofmemoryonaremotenode.AswithRDMAWRITEs,therespondingnodefirstallowstherequestingnodepermissiontoaccessitsmemory.Theresponderpassestotherequestoravirtualaddress,length,andR_KeytouseintheRDMAREADrequestpacket.Figure77RDMAREADOperationExampleAtomicoperationFetchAdd(FetchandAdd)Therequestorspecifies:?RemotedataaddressandR_Key?AdddataTheacknowledgepacketreturns:?OriginalremotedataCmpSwap(CompareandSwap)Therequestorspecifies:?RemotedataaddressandR_Key?Write(swap)data?ComparedataTheacknowledgmentpacketreturns:?OriginalremotedataRoCEv2RoCEv2isastraightforwardextensionoftheRoCEprotocolthatinvolvesasimplemodificationoftheRoCEpacketformat.InsteadoftheGRH,RoCEv2packetscarryanIPheaderwhichallowstraversalofIPL3RoutersandaUDPheaderthatservesasastatelessencapsulationlayerfortheRDMATransportProtocolPac