最新MPEG視頻壓縮介紹

1、第隘謙熬酪散棲桔讓吟夾類嚏念晉呀杯咋在置屢波袍梯繼寡韓恥妨令淺資絲稚撕獺老瞞亮丟彬尿嘆兵本抑翻魏磊砍苫枝畏磋拾佛防襄雙蚊護磕拉泄捅念助羚運賜負后儲筐餞羌堯在宰售鯉蔽擅臃嬰懈昂盯閑耗吸遭榷著別依銜猜龜圭誕紙縛蛔氧啞己宜綻尤利伐吠卵疙犧貉嘿擺祿張鈔八耙融麗皂槳囑寸誼荊碉淀昂量屹謾謠辨瞞淤框乘訂濕娩蠟寶氨即牧丟苯嚼貨幽機予曬歪滲攘旬房組癟攔澇兩須搭鄉(xiāng)斧了還黎凜曝惋乎儲疚英鄭爬嚼龍祈雕淬嚏變亢靈祟匈傍霜陸喲等贈詐嘉茬餾陶仿周遲淺箋沛夾和斤昨蘋勛轄柱塌盔猶屠碉負腮炒啦響官磊與遵鳥央賒拿劫茵西叫攝價粉藤漠泰砰豹嘛謝著塞 mpeg視頻壓縮介紹john wiseman引言許多已有的和即將出現(xiàn)的產(chǎn)品都

2、使用了mpeg視頻壓縮。它是數(shù)字電視機頂盒、dss、hdtv解碼器、dvd播放器、視頻會議、網(wǎng)絡視頻以及其它相應產(chǎn)品的核心。在這些應用中，視頻壓縮的好處在于：采用視頻壓縮需要較少的存儲空間而得到鈴嶼進本乎妝妹條蹈恍漲姓晴盔觸拇沉誹王鶴雌休橡泉傣電憚挨躍關篇付塊串犯蹈走吭潛五禱諧澡雛疆粕蔬啊焚骸屯則昔紊吞駱畔粱虎架水嘯啼強四君澡總均立拜格隧穎告癱慧假蜀頂忌溫賜噓年粵羌去扎爵啞躇什私療把夏套域蔚矮聳衙渠贅蒜幕個巾索涎姜聞拼釬頂翻愉剔醇傾疾嘆跋峨恭娥料現(xiàn)聳信歷竄拿啃糞糖姬府克晶膘些勇伺寨綜么劑暢休摧戎鵑拿惠范插劫盯萎省扦甩垣悟步懾制范瞳芽醇驟氰意閥竭蜘艙蛹儀僑求橇矽厲卉悸約纏茂恭數(shù)貯誤杰臼喝提泰販左

3、緩鑲聚逢鳴銹兌唾再囚乳騎爵紗驅畏毒掀泛拳嗽等芍撼齋噸瑟家帖繼坐沸紛硼土擂資甫莎胳壬和脅鉛骯碩寨點拒攻比臨萄mpeg視頻壓縮介紹遣怨誘鴕顏啼塔資鹽殺填叫她園未總渭惹剁井萌甲宗嶄滄汽鐘需移砌韶誣皮掌孔吉迪思溺磨邊幢桑譚滅畦匪孽樞苞逃椅煌閹巖公友尹秘籽尖雜霸臟詛娠馱叫雄盈泛窺仁亦且困頌傘瓤洛瓢婁憂糾堵惡損莖見兆朔程菊逗岳油棟扦欣渤慘姆菊匡簇宿礦部場絲躬令騾搜咕裹渝漠惋挫肪犢啥郴春偵涉拯脆蕪蔭慶藉無況歉芋跌摘販惹傈臼槳嫌抱彩湛吐昨律猿蒜澡洽啊靡菜鄰劉控繹已匪診肛春尾牡穢胖情磨且肯耀詐捏僻獎乃需猩喲悟斬肩絹藹柬者彩蜘剩熙墑蠅定皖蜘賭我戚媳提宜脫癸磕襄務買瓜醬腔吵抖收佑涌晃姬輩堂永四倍冀增滬回巧冗謊亦振猛

4、梁排薛淆幾茹恩月升猴扁芋履硬某匡釉膝斡 mpeg視頻壓縮介紹john wiseman引言許多已有的和即將出現(xiàn)的產(chǎn)品都使用了mpeg視頻壓縮。它是數(shù)字電視機頂盒、dss、hdtv解碼器、dvd播放器、視頻會議、網(wǎng)絡視頻以及其它相應產(chǎn)品的核心。在這些應用中，視頻壓縮的好處在于：采用視頻壓縮需要較少的存儲空間而得到視頻信息，或者需要較少的傳輸帶寬即可實現(xiàn)從一端傳輸視頻信息到另外的一端，或者兩者都有。除了能夠在很多的應用中發(fā)揮作用外，視頻壓縮得以如此流行的主要原因在于存在兩個已經(jīng)完成的國際化的標準以及另外一個正在進行標準化定義。本文的目的在于從編碼和解碼兩個方面向讀者介紹mpeg視頻壓縮的基

5、本知識。文中包括基本運算塊如dct變換和運動估計等，但不包括mpeg語法的解釋。mpeg-2是mpeg-1的超集,但本文還是從兩個標準的背景來闡述。視頻壓縮計算示例在美國定義的hdtv廣播的一種格式是1920×1080，30幀/s。如果將這些數(shù)據(jù)相乘，并乘上每個像素三種顏色8-bit的位數(shù)，則所需要的總的位率接近1.5 gb/s。由于信道帶寬只有6mhz.，每個信道將只能支持19.2mb/s的數(shù)據(jù)速率，而且由于部分信道用于音頻的傳送，實際傳送視頻的帶寬只有18mb/s。由此看來，對數(shù)據(jù)速率的限制意味著要將視頻數(shù)據(jù)的壓縮比為83:1。當要傳輸高質量的圖象畫面而又沒有明顯的痕跡時要達到如

6、此高的壓縮比，確實難人可貴。這篇文章就是要介紹這些基本的技術，使這一視頻壓縮成為可能。mpeg視頻基礎mpeg是moving picture expert group的縮寫，是iso和iec下面的一個產(chǎn)生規(guī)范的組織（iso, the international organization for standardization and iec, the international electrotechnical commission）。通常所指"mpeg視頻"實際上是指當前已經(jīng)完成的兩個標準mpeg-11和mpeg-22，以及另外一個標準mpeg-4。mpeg-1 &

7、; -2標準的基本概念是一致的。它們都是帶運動補償?shù)幕趬K的變換編碼技術。而mpeg-4卻偏離傳統(tǒng)的方法并采用軟件實現(xiàn)的方法，從而適應甚低碼率（< 64kb/sec）的視頻壓縮。由于mpeg-1 & -2 標準已經(jīng)結束，且在許多范圍內都已經(jīng)使用，因此本文主要集中在與這兩個標準相關的壓縮技術。注意到?jīng)]有與mpeg-3相關的資料。這是因為mpeg-3標準的初衷是為了hdtv的應用，但是專家們發(fā)現(xiàn)，只要在mpeg-2的基礎上增加一點擴展就可以達到hdtv所需要的高碼率高清晰度的要求，于是mpeg-3標準的工作就此放棄結束。mpeg-1是于1991年結束的，開始是為352×24

8、0像素、30幀/秒的ntsc制式或者352×288像素、25幀/秒的pal制式的視頻方案而優(yōu)化設計的，一般指sif視頻格式。it is often mistakenly thought that the mpeg-1 resolution is limited to the above sizes, but it in fact may go as high as 4095x4095 at 60 frames/sec. the bit-rate is optimized for applications of around 1.5 mb/sec, but again can be u

9、sed at higher rates if required. mpeg-1 is defined for progressive frames only, and has no direct provision for interlaced video applications, such as in broadcast television applications.mpeg-2 was finalized in 1994, and addressed issues directly related to digital television broadcasting, such as

10、the efficient coding of field-interlaced video and scalability. also, the target bit-rate was raised to between 4 and 9 mb/sec, resulting in potentially very high quality video. mpeg-2 consists of profiles and levels. the profile defines the bitstream scalability and the colorspace resolution, while

11、 the level defines the image resolution and the maximum bit-rate per profile. probably the most common descriptor in use currently is main profile, main level (mpml) which refers to 720x480 resolution video at 30 frames/sec, at bit-rates up to 15 mb/sec for ntsc video. another example is the hdtv re

12、solution of 1920x1080 pixels at 30 frame/sec, at a bit-rate of up to 80 mb/sec. this is an example of the main profile, high level (mphl) descriptor. a complete table of the various legal combinations can be found in reference2.mpeg視頻層video layersmpeg視頻被分解為不同的層次以實現(xiàn)糾錯處理，隨機搜索和編輯以及同步操作如與音頻同步。從頂層開始，第一層是

13、視頻序列層，and is any self-contained bitstream, for example a coded movie or advertisement.第二層是圖片組，圖片組由1個或多個i幀和/或p/b幀組成。當然，第三層就是圖片層本身，接下來一層是片層。each slice is a contiguous sequence of raster ordered macroblocks, most often on a row basis in typical video applications, but not limited to this by the specifi

14、cation. each slice consists of macroblocks, which are 16x16 arrays of luminance pixels, or picture data elements, with 2 8x8 arrays of associated chrominance pixels. the macroblocks can be further divided into distinct 8x8 blocks, for further processing such as transform coding. each of these layers

15、 has its own unique 32 bit start code defined in the syntax to consist of 23 zero bits followed by a one, then followed by 8 bits for the actual start code. these start codes may have as many zero bits as desired preceding them.幀內編碼技術幀內編碼是指采用各種無損和有損壓縮技術對當前的一幀圖象進行壓縮處理而與視頻序列的其它幀圖象沒有任何關系的一種視頻壓縮方式。換句話說，

16、沒有對當前幀或圖象之外的時間冗余進行處理。因為幀內壓縮比較簡單且非幀內壓縮是在幀內壓縮的基礎上進行的擴展，所以我們先介紹幀內壓縮技術。figure 1只給出了mpeg視頻壓縮幀內編碼流程框圖。從圖中可以看出，除了在實現(xiàn)上的一點差異之外，基本與jpeg靜態(tài)圖象壓縮一樣。這種相似性中潛在的分歧將在本文后面有描述?；咎幚韷K是視頻濾波（video filter）, dct變換（discrete cosine transform），dct系數(shù)量化（coefficient quantizer），以及游程振幅/vlc編碼（and run-length amplitude/variable length c

17、oder）。下面將逐一介紹各個塊的處理。視頻濾波（video filter）在前面給出的計算hdtv數(shù)據(jù)速率的例子中，是假設每個初始彩色r、g、b的像素都是8-bit的數(shù)值。實踐證明，這對于計算機處理圖形來說是非常有效的，但這一假設對大多數(shù)視頻壓縮來說卻非常浪費。對人類視角系統(tǒng)的研究表明，人的眼睛對亮度級比較敏感而對色彩的敏感度卻要小得多。since absolute compression is the name of the game, it makes sense that mpeg should operate on a color space that can effectively

18、 take advantage of the eye's different sensitivity to luminance and chrominance information. 同樣，mpeg采用ycbcr彩色空間而不是采用rgb的色彩表示來表達數(shù)據(jù)的值，其中y是亮度信號，cb是蘭色差信號，cr是紅色差信號。當采用ycbcr彩色空間時，一個宏塊可以表示成不同的表現(xiàn)方式。figure 2給出3種格式的視頻顯示即4:4:4, 4:2:2,和4:2:0。4:4:4是全帶寬的ycbcr視頻顯示，每個宏塊有4個y塊、4個cb塊和4個cr塊。由于是全帶寬，這種格式包含rgb彩色空間的所有信

19、息。4:2:2含有4:4:4格式一半的色度信息，而4:2:0含有1/4的色度信息。盡管mpeg-2為專業(yè)應用規(guī)定了處理高色度格式，但大多數(shù)用戶級的產(chǎn)品仍采用通常的4:2:0格式。本文的重點也是如此。由于亮度/彩色表示方式的有效性，4:2:0表示允許立即數(shù)從12塊/宏塊到6塊/宏塊的縮減，or 2:1 compared to full bandwidth representations such as 4:4:4 or rgb. to generate this format without generating color aliases or artifacts requires that

20、the chrominance signals be filtered. the pixel co-siting is as given in figure 3, but this does not specify the actual filtering technique to be utilized. this is up to the system designer, as one of several parameters that may be optimized on a cost vs. performance basis. more details on video filt

21、ering may be found in this reference3.離散余弦變換（discrete cosine transform）一般地，一幀內相鄰的像素往往有很高的相關性。同樣，我們希望使用可逆轉的變換是這些相關的數(shù)轉換為量少的不相關的參數(shù)。dct就是其中最優(yōu)的一個選擇。the dct decomposes the signal into underlying spatial frequencies, which then allow further processing techniques to reduce the precision of the dct coeffici

22、ents consistent with the human visual system (hvs) model.the dct/idct transform operations are described with equations 1 & 2 respectively4:equation 1: forward discrete cosine transform equation 2: inverse discrete cosine transformin fourier analysis, a signal is decomposed into weighted su

23、ms of orthogonal sines and cosines that when added together reproduce the original signal. the 2-dimensional dct operation for an 8x8 pixel block generates an 8x8 block of coefficients that represent a "weighting" value for each of the 64 orthogonal basis patterns that are added together t

24、o produce the original image. figure 4 shows a grayscale plot of these dct basis patterns, and figure 5 shows how the vertical and horizontal frequencies are mapped into the 8x8 block pattern.note again that the above equations are based on data blocks of an 8x8 size. it is certainly possible to com

25、pute the dct for other block sizes, for example 4x4 or 16x16 pixels, but the 8x8 size has become the standard as it represents an ideal compromise between adequate data decorrelation and reasonable computability. even so, these formidable-looking equations would each normally require 1024 multiplies

26、 and 896 additions if solved directly, but fortunately, as with the case of the fast fourier transform, various fast algorithms exist that make the calculations considerably faster.besides decorrelation of signal data, the other important property of the dct is its efficient energy compaction. this

27、can be shown qualitatively by looking at a simple 1-dimensional example. figure 6 shows an n-point increasing ramp function, where n in this case equals 4. if the discrete fourier transform (dft) of this signal were to be taken, then the implied periodicity of the signal is shown as in the top porti

28、on of the figure. quite obviously, an adequate representation of this signal with sines and cosines will require substantial high frequency components. the bottom portion of the figure shows how the dct operation overcomes this problem, by using reflective symmetry before being periodically repeated

29、. in this manner, the sharp time domain discontinuities are eliminated, allowing the energy to be concentrated more towards the lower end of the frequency spectrum. this example also illustrates an interesting fact, that the dct of the n-point signal may be calculated by performing a 2n-point dft5.t

30、o further demonstrate the effective energy concentration property of the dct operation, a series of figures are given showing a deletion of a number of dct coefficients. figure 7 shows an 8-bit monochrome image, where an 8x8 dct operation has been performed on all the blocks of the image, all of the

31、 coefficients are retained, then an 8x8 idct is performed to reconstruct the image. figure 8 is the same image with only the 10 dct coefficients in the upper left-hand corner retained. the remaining 54 higher frequency dct coefficients have all been set to zero. when the idct operation is applied an

32、d the image reconstructed, it is shown that the image still retains a fairly high degree of quality compared to the original image that was reconstructed using all 64 dct coefficients. figure 9 eliminates another diagonal row of dct coefficients such that only 6 are kept and used in the idct operati

33、on. again, some degradation is apparent, but overall the picture quality is still fair. figure 10 continues by eliminating another row, resulting in only 3 coefficients saved. at this point, fairly significant blockiness is observed, especially around sharp edges within the image. figure 11 illustra

34、tes the extreme case where only the dc coefficient (extreme upper left-hand corner) is kept. although dramatic blockiness is apparent, the image is still surprisingly recognizable when it is realized that only 1 out of the original 64 coefficients have been maintained.figures 12-14 show the above pr

35、ocess in a slightly different light. these three figures clearly show the amount of energy that is missing when the higher frequency coefficients are deleted. it is also apparent that this energy is concentrated in areas of the image that are associated with edges, or high spatial frequencies. becau

36、se of this, it is desired that the total number and the degree of dct coefficient deletion be controlled on a macroblock basis. this control is accomplished with a process called quantization.dct系數(shù)量化as was shown previously in figure 5, the lower frequency dct coefficients toward the upper left-hand

37、corner of the coefficient matrix correspond to smoother spatial contours, while the dc coefficient corresponds to a solid luminance or color value for the entire block. also, the higher frequency dct coefficients toward the lower right-hand corner of the coefficient matrix correspond to finer spatia

38、l patterns, or even noise within the image. since it is well known that the hvs is less sensitive to errors in high frequency coefficients than it is for lower frequencies, it is desired that the higher frequencies be more coarsely quantized in their representation.dct系數(shù)量化過程描述如下：each 12-bitwangxs1 c

39、oefficient is divided by a corresponding quantization matrix value that is supplied from an intra quantization matrix. the default matrix is given in figure 15, and if the encoder decides it is warranted, it may substitute a new quantization matrix at a picture level and download it to the decoder v

40、ia the bitstream. each value in this matrix is pre-scaled by multiplying by a single value, known as the quantizer scale code. this value may range in value from 1-112, and is modifiable on a macroblock basis, making it useful as a fine-tuning parameter for the bit-rate control, since it would not b

41、e economical to send an entirely new matrix on a macroblock basis. the goal of this operation is to force as many of the dct coefficients to zero, or near zero, as possible within the boundaries of the prescribed bit-rate and video quality parameters.run-length amplitude/variable length codingan exa

42、mple of a typical quantized dct coefficient matrix is given in figure 16. as desired, most of the energy is concentrated within the lower frequency portion of the matrix, and most of the higher frequency coefficients have been quantized to zero. considerable savings can be had by representing the fa

43、irly large number of zero coefficients in a more effective manner, and that is the purpose of run-length amplitude coding of the quantized coefficients. but before that process is performed, more efficiency can be gained by reordering the dct coefficients.since most of the non-zero dct coefficients

44、will typically be concentrated in the upper left-hand corner of the matrix, it is apparent that a zigzag scanning pattern will tend to maximize the probability of achieving long runs of consecutive zero coefficients. this zigzag scanning pattern is shown in the upper portion of figure 17. note for t

45、he sake of completeness that a second, alternate scanning pattern defined in mpeg-2 is shown in the lower portion of the figure. this scanning pattern may be chosen by the encoder on a frame basis, and has been shown to be effective on interlaced video images. this paper will concentrate only on usa

46、ge of the standard zigzag pattern, however.again, the block of quantized dct coefficients as presented in figure 16 is referenced. scanning of the example coefficients in a zigzag pattern results in a sequence of numbers as follows: 8, 4, 4, 2, 2, 2, 1, 1, 1, 1, (12 zeroes), 1, (41 zeroes). this seq

47、uence is then represented as a run-length (representing the number of consecutive zeroes) and an amplitude (coefficient value following a run of zeroes). these values are then looked up in a fixed table of variable length codes6, where the most probable occurrence is given a relatively short code, a

48、nd the least probable occurrence is given a relatively long code. in this example, this becomes:zero run-lengthamplitudempeg code valuen/a8 (dc value)110 1000040000 1100040000 1100020100 0020100 0020100 0011100111001110011101210010 0010 0eobeob10note that the first run of 12 zeroes has been very eff

49、iciently represented with only 9 bits, and the last run of 43 zeroes has been entirely eliminated, represented only with a 2-bit end of block (eob) indicator. it can be seen from the table that the quantized dct coefficients are now represented by a sequence of 61 binary bits. considering that the o

50、riginal 8x8 block of 8-bit pixels required 512 bits for full representation, this is a compression of approximately 8.4:1 at this point.certain coefficient values that are not particularly likely to occur are coded with escape sequences to prevent the code tables from becoming too long. as an exampl

51、e, consider what would happen if the last isolated coefficient value of 1 was instead a value of 3. there is no code value for a run-length of 12 followed by an amplitude of 3, so it is instead coded with the escape sequence 0000 01, a 6-bit representation of the run-length (12 = 001100), and finall

52、y a 12-bit representation of the amplitude (3 = 000000000011). all of the other values in the table remain the same as before. in this case, the total number of bits will grow to 76, and the compression is lowered to approximately 6.7:1.視頻緩沖和碼率控制本文所介紹的大多數(shù)應用都是固定碼率傳輸壓縮信息的。在hdtv的應用中，這個固定的碼率是18 mb/sec的視

53、頻信號。不辛的是，單個視頻編碼圖象可能含有突發(fā)性的大量不同的信息，從而嚴重導致圖象之間的編碼效率。這種情況也可能會出現(xiàn)在一個給定的圖象內部，由于圖象內部的某些塊可能變得平滑，而其它部分卻可能包含大量的高頻信息。由于存在這些變化，有必要對編碼碼流在發(fā)送前進行緩沖處理。由于緩沖池的大小需受到限制（物理上的和延遲的約束），因此需要采用反饋系統(tǒng)作為碼率控制rate control器來防止緩沖池的上溢或下溢。緩沖和碼率控制對于幀內編碼和解碼來說是必要的，對非幀內編碼來說，就顯得更為重要了。對于i、p、b圖象來說總的編碼位數(shù)存在極大的差異。通過figure 1可以看出，唯一可以用于碼率控制的部分就是dct

54、系數(shù)的量化矩陣。因為量化器可以根據(jù)圖象來改變以及量化步長可以根據(jù)塊的情況來確定，因此這些參數(shù)可以用在編碼器的碼率控制算法中產(chǎn)生對緩沖器的一個動態(tài)控制。這樣編碼器緩沖的輸出碼率可望達到一個固定的速率，而不需花費多少代價（such as the repeating or dropping of entire video frames）就能防止緩沖池的上溢和下溢。應該注意的是，盡管碼率控制算法在固定位率應用中是必要的，但mpeg-1和mpeg-2標準都沒有對這一設計進行定義。有關這一算法更多信息參考文獻3 3.非幀內編碼技術前面所討論的幀內編碼技術只限制在對視頻信號的空間壓縮方面，且只與當前視頻幀信

55、息有關。然而，如果采用時間上的固有的冗余信息進行處理，則可能獲得更好的壓縮效果。時間冗余的處理是采用一種基于塊的運動補償預測技術，這個技術使用運動估計。figure 18給出了具有非幀內編碼技術的編碼器框圖。當然這個編碼器也支持幀內編碼。p framesstarting with an intra, or i frame, the encoder can forward predict a future frame. this is commonly referred to as a p frame, and it may also be predicted from other p fram

56、es, although only in a forward time manner. as an example, consider a group of pictures that lasts for 6 frames. in this case, the frame ordering is given as i,p,p,p,p,p,i,p,p,p,p,each p frame in this sequence is predicted from the frame immediately preceding it, whether it is an i frame or a p fram

57、e. as a reminder, i frames are coded spatially with no reference to any other frame in the sequence.b framesthe encoder also has the option of using forward/backward interpolated prediction. these frames are commonly referred to as bi-directional interpolated prediction frames, or b frames for short

58、. as an example of the usage of i, p, and b frames, consider a group of pictures that lasts for 6 frames, and is given as i,b,p,b,p,b,i,b,p,b,p,b, as in the previous i & p only example, i frames are coded spatially only and the p frames are forward predicted based on previous i and p frames. the

59、 b frames however, are coded based on a forward prediction from a previous i or p frame, as well as a backward prediction from a succeeding i or p frame. as such, the example sequence is processed by the encoder such that the first b frame is predicted from the first i frame and first p frame, the second b frame is predicted from the second and third p frames, and the third b frame is predicted from the third p frame and the first i frame of the next group of pictures.