學(xué)習(xí)TI DVSDK過程記錄(一)：視頻數(shù)據(jù)從哪里來到哪里去

1、本文討論的是基于DVSDK3,硬件平臺(tái)是勤澤的ZMV6467。 DVSDK TI為達(dá)芬奇平臺(tái)的開發(fā)者提供了一套完善的視頻軟件開發(fā)套件。DVSDK功能非常強(qiáng)大，它將很多視頻編解碼的細(xì)節(jié)都隱藏起來了，使用者只需要關(guān)系應(yīng)用就可以，但是DVSDK的強(qiáng)大又使得它變得異常的龐大，讓使用者摸不著頭腦， 或者根本不知道怎樣開始開發(fā)。接下來的TI DVSDK學(xué)習(xí)記錄文章將會(huì)一步步分析DVSDK，將我學(xué)習(xí)使用DVSDK進(jìn)行開發(fā)的過程中碰到的問題或者相關(guān)的經(jīng)驗(yàn)一一寫出,文章中不會(huì)涉及到如何構(gòu)建編譯環(huán)境，如何編譯DVSDK的問題，如需了解這方面的問題可以參考TI官方文檔或聯(lián)系勤澤。文章如有錯(cuò)誤敬請(qǐng)?zhí)岢觥?#

2、160;以視頻采集應(yīng)用的開發(fā)過程，提出以下問題：1.視頻怎樣來，來自哪里？2.采集到視頻數(shù)據(jù)要怎樣處理？（顯示/壓縮/存放/發(fā)送）3.視頻數(shù)據(jù)怎樣直接顯示出來？4.視頻數(shù)據(jù)怎樣壓縮？5.視頻數(shù)據(jù)怎樣存放？6.視頻數(shù)據(jù)如何發(fā)送？  下面就開始一一來解決這些問題： 先從硬件功能框圖看一下視頻的輸入通道，紅色框部分：  DM6467支持BT.656/BT.1120/RAW的數(shù)據(jù)輸入，最高可以支持1080P30FPS， 如果需要支持1080P60fps，可以使用1GHz版本的DM6467T。1）模擬高清分量輸入，經(jīng)過視頻解碼器后輸出符合BT.1120的

3、數(shù)據(jù)，然后再通過VPIF接口進(jìn)入CPU。2）模擬標(biāo)清信號(hào)輸入，經(jīng)過視頻解碼器后輸出符合BT.656的數(shù)據(jù)，然后再通過VPIF接口進(jìn)入CPU。3）數(shù)字?jǐn)z像頭(eg.cmos)，直接通過VPIF接口進(jìn)入CPU。這里要注意，因?yàn)镈M6467只支持YUVPSemi4:2:0格式的數(shù)據(jù)顯示和編碼(H.264),所以數(shù)字?jǐn)z像頭輸出的YCBCR或RGB數(shù)據(jù)再進(jìn)入CPU后需要進(jìn)行轉(zhuǎn)換(顏色空間的問題，可以參考本博客前面的文章)，這個(gè)轉(zhuǎn)換可以通過ARM，也可以通過DSP來進(jìn)行，勤澤的軟件已經(jīng)附帶了一個(gè)DSP的轉(zhuǎn)換核，可以直接轉(zhuǎn)換cmos的YCBCR數(shù)據(jù)進(jìn)行顯示和編碼。 數(shù)據(jù)進(jìn)入CPU后，接下來就是要獲

4、取數(shù)據(jù)了，這部分的只使用到ARM端。數(shù)據(jù)流向：video decoder driver -> vpif_caputre driver ->V4L2 driver->應(yīng)用層這里我們先不去關(guān)心video decoder driver 和  vpif_caputre driver，因?yàn)檫@兩個(gè)不影響我們分析DVSDK。 這里的V4L2 driver和我們平常使用的V4L2驅(qū)動(dòng)沒什么區(qū)別，就是提供了一些訪問視頻設(shè)備的接口，但是TI為了簡化對(duì)V4L2的操作，方便用戶更快的編寫出視頻應(yīng)用程序，所以他們在V4L2的基礎(chǔ)上又封裝了一層叫做DMAI的東西。DMAI是

5、什么呢？因?yàn)榻酉聛頃?huì)有更多的新名詞冒出，所以這里我們先看一下DVSDK的一個(gè)簡單的構(gòu)成框圖，這個(gè)框圖在后面會(huì)不斷擴(kuò)大：  從框圖可以看出DMAI將視頻編解碼器(CE)、內(nèi)存管理(CMEM)和V4L2的訪問操作都統(tǒng)一到一套接口上了，這樣視頻編解碼應(yīng)用程序的開發(fā)者只需要關(guān)系數(shù)據(jù)的業(yè)務(wù)應(yīng)用而不需要去理會(huì)邏輯層的東西。 但是作為底層的開發(fā)人員，我們還是有必要去看看DMAI到底是怎樣和CE/V4L2/CMEM進(jìn)行交互，并且把它們封裝起來的，然后到最后再看怎樣使用DMAI進(jìn)行應(yīng)用程序的開發(fā)。 1.DMAI與VL42的交互(DMAI版本:2_20_00_15)。在dm

6、ai_2_20_00_15/packages/ti/sdo/dmai/linux/dm6467目錄下有一個(gè)Capture.c的文件，Capture.c包含了所有對(duì)V4L2的操作封裝?，F(xiàn)在來看一下文件中的一個(gè)函數(shù)，這個(gè)函數(shù)用于創(chuàng)建一個(gè) 視頻捕獲設(shè)備的操作句柄。View Code 1 /* 2 * Capture_create 3 */ 4 Capture_Handle Capture_create(BufTab_Handle hBufTab, Capture_Attrs *attrs) 5 6 struct v4l2_capability cap; 7 struct v4l2_cropcap c

7、ropCap; 8 struct v4l2_crop crop; 9 struct v4l2_format fmt; 10 enum v4l2_buf_type type; 11 Capture_Handle hCapture; 12 VideoStd_Type videoStd; 13 Int32 width, height; 14 Uint32 pixelFormat; 15 16 assert(attrs); 17 Dmai_clear(fmt); 18 19 /* Allocate space for state object */ 20 hCapture = calloc(1, si

8、zeof(Capture_Object); 21 22 if (hCapture = NULL) 23 Dmai_err0("Failed to allocate space for Capture Objectn"); 24 return NULL; 25 26 27 /* User allocated buffers by default */ 28 hCapture->userAlloc = TRUE; 29 30 /* Open video capture device */ 31 /* 打開V4L2視頻輸入設(shè)備 */ 32 hCapture->fd =

9、 open(attrs->captureDevice, O_RDWR, 0); 33 34 if (hCapture->fd = -1) 35 Dmai_err2("Cannot open %s (%s)n", attrs->captureDevice, 36 strerror(errno); 37 cleanup(hCapture); 38 return NULL; 39 40 41 /* See if an input is connected, and if so which standard */ 42 /* 檢測V4L2設(shè)備當(dāng)前是否有視頻信號(hào)輸入

10、，輸入信號(hào)的標(biāo)準(zhǔn) */ 43 if (Capture_detectVideoStd(hCapture, &videoStd, attrs) < 0) 44 cleanup(hCapture); 45 return NULL; 46 47 48 hCapture->videoStd = videoStd; 49 50 if (VideoStd_getResolution(videoStd, &width, &height) < 0) 51 cleanup(hCapture); 52 Dmai_err0("Failed to get resolu

11、tion of capture video standardn"); 53 return NULL; 54 55 56 /* Query for capture device capabilities */ 57 /* 這里就是調(diào)用ioctl直接操作v4l2設(shè)備了，查詢設(shè)備的特性*/ 58 if (ioctl(hCapture->fd, VIDIOC_QUERYCAP, &cap) = -1) 59 cleanup(hCapture); 60 if (errno = EINVAL) 61 Dmai_err1("%s is no V4L2 devicen&quo

12、t;, attrs->captureDevice); 62 cleanup(hCapture); 63 return NULL; 64 65 Dmai_err2("Failed VIDIOC_QUERYCAP on %s (%s)n", attrs->captureDevice, 66 strerror(errno); 67 cleanup(hCapture); 68 return NULL; 69 70 71 if (!(cap.capabilities & V4L2_CAP_VIDEO_CAPTURE) 72 Dmai_err1("%s i

13、s not a video capture devicen", attrs->captureDevice); 73 cleanup(hCapture); 74 return NULL; 75 76 77 if (!(cap.capabilities & V4L2_CAP_STREAMING) 78 Dmai_err1("%s does not support streaming i/on", attrs->captureDevice); 79 cleanup(hCapture); 80 return NULL; 81 82 83 fmt.typ

14、e = V4L2_BUF_TYPE_VIDEO_CAPTURE; 84 /* 獲取V4L2設(shè)備的幀格式 */ 85 if (ioctl(hCapture->fd, VIDIOC_G_FMT, &fmt) = -1) 86 Dmai_err2("Failed VIDIOC_G_FMT on %s (%s)n", attrs->captureDevice, 87 strerror(errno); 88 cleanup(hCapture); 89 return NULL; 90 91 92 fmt.fmt.pix.width = width; 93 fmt.f

15、mt.pix.height = height; 94 fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE; 95 96 switch(attrs->colorSpace) 97 case ColorSpace_UYVY: 98 fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_UYVY; 99 break;100 case ColorSpace_YUV420PSEMI:101 fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_NV12;102 break;103 case ColorSpace_YUV422

16、PSEMI:104 fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_NV16;105 break;106 default:107 Dmai_err1("Unsupported color format %gn", attrs->colorSpace); 108 cleanup(hCapture);109 return NULL;110 111 112 if (videoStd = VideoStd_BAYER_CIF) | (videoStd = VideoStd_BAYER_VGA) | 113 (videoStd = VideoStd

17、_BAYER_1280) 114 fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_SBGGR8;115 116 117 fmt.fmt.pix.bytesperline = BufferGfx_calcLineLength(fmt.fmt.pix.width, 118 attrs->colorSpace);119 fmt.fmt.pix.sizeimage = BufferGfx_calcSize(attrs->videoStd, attrs->colorSpace);120 121 /printf("DMAI: pix.bytespe

18、rline= %d, pix.sizeimage= %drn",fmt.fmt.pix.bytesperline,fmt.fmt.pix.sizeimage);122 pixelFormat = fmt.fmt.pix.pixelformat;123 124 if (videoStd = VideoStd_CIF) | (videoStd = VideoStd_SIF_PAL) | 125 (videoStd = VideoStd_SIF_NTSC) | (videoStd = VideoStd_D1_PAL) |126 (videoStd = VideoStd_D1_NTSC) |

19、 (videoStd = VideoStd_1080I_30) |127 (videoStd = VideoStd_1080I_25) 128 fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;129 else 130 fmt.fmt.pix.field = V4L2_FIELD_NONE;131 132 133 /* 設(shè)置V4L2輸入設(shè)備的幀格式 */134 if (ioctl(hCapture->fd, VIDIOC_S_FMT, &fmt) = -1) 135 printf("Failed VIDIOC_S_FMT on %s (

20、%s)n", attrs->captureDevice,136 strerror(errno);137 cleanup(hCapture);138 return NULL;139 140 141 if (fmt.fmt.pix.width != width) | (fmt.fmt.pix.height != height) 142 Dmai_err4("Failed to set resolution %d x %d (%d x %d)n", width,143 height, fmt.fmt.pix.width, fmt.fmt.pix.height);1

21、44 cleanup(hCapture);145 return NULL;146 147 148 if (pixelFormat != fmt.fmt.pix.pixelformat) 149 Dmai_err2("Pixel format 0x%x not supported. Received 0x%xn", 150 pixelFormat, fmt.fmt.pix.pixelformat);151 cleanup(hCapture);152 return NULL; 153 154 155 Dmai_dbg3("Video input connected s

22、ize %dx%d pitch %dn",156 fmt.fmt.pix.width, fmt.fmt.pix.height, fmt.fmt.pix.bytesperline);157 158 /* Query for video input cropping capability */159 160 if (attrs->cropWidth > 0 && attrs->cropHeight > 0) 161 162 cropCap.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;163 if (ioctl(hCaptu

23、re->fd, VIDIOC_CROPCAP, &cropCap) = -1) 164 Dmai_err2("VIDIOC_CROPCAP failed on %s (%s)n", attrs->captureDevice,165 strerror(errno);166 cleanup(hCapture);167 return NULL;168 169 170 if (attrs->cropX & 0x1) 171 Dmai_err1("Crop width (%ld) needs to be evenn", attrs

24、->cropX);172 cleanup(hCapture);173 return NULL;174 175 176 crop.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;177 crop.c.left = attrs->cropX;178 crop.c.top = attrs->cropY;179 crop.c.width = attrs->cropWidth;180 crop.c.height = hCapture->topOffset ? attrs->cropHeight + 4 + 2 :181 attrs->cro

25、pHeight;182 183 Dmai_dbg4("Setting capture cropping at %dx%d size %dx%dn",184 crop.c.left, crop.c.top, crop.c.width, crop.c.height);185 186 /* Crop the image depending on requested image size */187 if (ioctl(hCapture->fd, VIDIOC_S_CROP, &crop) = -1) 188 Dmai_err2("VIDIOC_S_CROP

26、 failed on %s (%s)n", attrs->captureDevice,189 strerror(errno);190 cleanup(hCapture);191 return NULL;192 193 194 195 if (hBufTab = NULL) 196 hCapture->userAlloc = FALSE;197 198 /* The driver allocates the buffers */199 /* 調(diào)用CMEM創(chuàng)建要用到的視頻緩沖區(qū) */200 if (_Dmai_v4l2DriverAlloc(hCapture->fd,2

27、01 attrs->numBufs,202 V4L2_BUF_TYPE_VIDEO_CAPTURE,203 &hCapture->bufDescs,204 &hBufTab,205 hCapture->topOffset,206 attrs->colorSpace) < 0) 207 Dmai_err1("Failed to allocate capture driver buffers on %sn",208 attrs->captureDevice);209 cleanup(hCapture);210 return N

28、ULL;211 212 213 else 214 /* Make the driver use the user supplied buffers */215 /* 如果調(diào)用者已經(jīng)創(chuàng)建好緩沖區(qū)，那只需要加入相應(yīng)的隊(duì)列管理就可以 */216 if (_Dmai_v4l2UserAlloc(hCapture->fd,217 attrs->numBufs,218 V4L2_BUF_TYPE_VIDEO_CAPTURE,219 &hCapture->bufDescs,220 hBufTab,221 0, attrs->colorSpace) < 0) 222 Dm

29、ai_err1("Failed to intialize capture driver buffers on %sn",223 attrs->captureDevice);224 cleanup(hCapture);225 return NULL;226 227 228 229 hCapture->hBufTab = hBufTab;230 231 /* Start the video streaming */232 type = V4L2_BUF_TYPE_VIDEO_CAPTURE;233 234 /* 配置完后，啟動(dòng)V4L2設(shè)備 */235 if (ioc

30、tl(hCapture->fd, VIDIOC_STREAMON, &type) = -1) 236 Dmai_err2("VIDIOC_STREAMON failed on %s (%s)n", attrs->captureDevice,237 strerror(errno);238 cleanup(hCapture);239 return NULL;240 241 242 hCapture->started = TRUE;243 244 return hCapture;245  從上面的代碼可以看出，DMAI的函數(shù)將大量的對(duì)V4L2的

31、控制操作都封裝起來了，使用者只需要傳入緩沖區(qū)列表和視頻采集參數(shù)就可以創(chuàng)建一個(gè)視頻采集設(shè)備。設(shè)備采集到數(shù)據(jù)后，會(huì)將數(shù)據(jù)填充到使用者提供的緩沖區(qū)，使用者只需要取出相應(yīng)的緩沖區(qū)就可以得到視頻數(shù)據(jù)。在上面的函數(shù)中，調(diào)用了一個(gè)_Dmai_v4l2DriverAlloc()函數(shù)，這個(gè)函數(shù)的作用是分配視頻緩沖區(qū)，它最終是調(diào)用CMEM進(jìn)行內(nèi)存分配的，下面一起看看這個(gè)函數(shù)的具體實(shí)現(xiàn)。 2.DMAI與CMEM的交互。查看dmai_2_20_00_15/packages/ti/sdo/dmai/linux/dm6467/_VideoBuf.c_Dmai_v4l2DriverAlloc() 1 /* 2 *

32、 _Dmai_v4l2DriverAlloc 3 */ 4 Int _Dmai_v4l2DriverAlloc(Int fd, Int numBufs, enum v4l2_buf_type type, 5 struct _VideoBufDesc *bufDescsPtr, 6 BufTab_Handle *hBufTabPtr, Int topOffset, 7 ColorSpace_Type colorSpace) 8 9 BufferGfx_Attrs gfxAttrs = BufferGfx_Attrs_DEFAULT; 10 struct v4l2_requestbuffers r

33、eq; 11 struct v4l2_format fmt; 12 _VideoBufDesc *bufDesc; 13 Buffer_Handle hBuf; 14 Int bufIdx; 15 Int8 *virtPtr; 16 17 Dmai_clear(fmt); 18 fmt.type = type; 19 20 if (ioctl(fd, VIDIOC_G_FMT, &fmt) = -1) 21 Dmai_err1("VIDIOC_G_FMT failed (%s)n", strerror(errno); 22 return Dmai_EFAIL; 23

34、 24 25 Dmai_clear(req); 26 req.count = numBufs; 27 req.type = type; 28 req.memory = V4L2_MEMORY_MMAP; 29 30 /* Allocate buffers in the capture device driver */ 31 /* 申請(qǐng)建立V4L2的視頻緩沖區(qū)管理隊(duì)列*/ 32 if (ioctl(fd, VIDIOC_REQBUFS, &req) = -1) 33 Dmai_err1("VIDIOC_REQBUFS failed (%s)n", strerror(e

35、rrno); 34 return Dmai_ENOMEM; 35 36 37 if (req.count < numBufs | !req.count) 38 Dmai_err0("Insufficient device driver buffer memoryn"); 39 return Dmai_ENOMEM; 40 41 42 /* Allocate space for buffer descriptors */ 43 *bufDescsPtr = calloc(numBufs, sizeof(_VideoBufDesc); 44 45 if (*bufDesc

36、sPtr = NULL) 46 Dmai_err0("Failed to allocate space for buffer descriptorsn"); 47 return Dmai_ENOMEM; 48 49 50 gfxAttrs.dim.width = fmt.fmt.pix.width; 51 gfxAttrs.dim.height = fmt.fmt.pix.height; 52 gfxAttrs.dim.lineLength = fmt.fmt.pix.bytesperline; 53 gfxAttrs.colorSpace = colorSpace; 54

37、 gfxAttrs.bAttrs.reference = TRUE; 55 56 /* 調(diào)用CMEM建立緩沖區(qū) */ 57 *hBufTabPtr = BufTab_create(numBufs, fmt.fmt.pix.sizeimage, 58 BufferGfx_getBufferAttrs(&gfxAttrs); 59 60 if (*hBufTabPtr = NULL) 61 return Dmai_ENOMEM; 62 63 64 /* 將建立好的緩沖區(qū)放到隊(duì)列中并且配置好相關(guān)屬性 */ 65 for (bufIdx = 0; bufIdx < numBufs; bu

38、fIdx+) 66 bufDesc = &(*bufDescsPtr)bufIdx; 67 68 /* Ask for information about the driver buffer */ 69 Dmai_clear(bufDesc->v4l2buf); 70 bufDesc->v4l2buf.type = type; 71 bufDesc->v4l2buf.memory = V4L2_MEMORY_MMAP; 72 bufDesc->v4l2buf.index = bufIdx; 73 74 /* 查找隊(duì)列中的緩沖區(qū) */ 75 if (ioctl(f

39、d, VIDIOC_QUERYBUF, &bufDesc->v4l2buf) = -1) 76 Dmai_err1("Failed VIDIOC_QUERYBUF (%s)n", strerror(errno); 77 return Dmai_EFAIL; 78 79 80 /* 修改緩沖區(qū)的屬性 */ 81 82 /* Map the driver buffer to user space */ 83 virtPtr = mmap(NULL, 84 bufDesc->v4l2buf.length, 85 PROT_READ | PROT_WRITE,

40、86 MAP_SHARED, 87 fd, 88 bufDesc->v4l2buf.m.offset) + topOffset; 89 90 if (virtPtr = MAP_FAILED) 91 Dmai_err1("Failed to mmap buffer (%s)n", strerror(errno); 92 return Dmai_EFAIL; 93 94 95 /* Initialize the Buffer with driver buffer information */ 96 hBuf = BufTab_getBuf(*hBufTabPtr, bu

41、fIdx); 97 98 Buffer_setNumBytesUsed(hBuf, fmt.fmt.pix.bytesperline * 99 fmt.fmt.pix.height);100 Buffer_setUseMask(hBuf, gfxAttrs.bAttrs.useMask);101 Buffer_setUserPtr(hBuf, virtPtr);102 103 /* Initialize buffer to black */104 _Dmai_blackFill(hBuf);105 106 Dmai_dbg3("Driver buffer %d mapped to %

42、#x has physical address "107 "%#lxn", bufIdx, (Int) virtPtr, Buffer_getPhysicalPtr(hBuf);108 109 bufDesc->hBuf = hBuf;110 111 /* Queue buffer in device driver */112 /* 將緩沖區(qū)放回隊(duì)列中 */113 if (ioctl(fd, VIDIOC_QBUF, &bufDesc->v4l2buf) = -1) 114 Dmai_err1("VIODIOC_QBUF failed

43、 (%s)n", strerror(errno);115 return Dmai_EFAIL;116 117 118 119 return Dmai_EOK;120 繼續(xù)分析Buffer_create()函數(shù)查看dmai_2_20_00_15/packages/ti/sdo/dmai/Buffer.c。Buffer_create() 1 /* 2 * Buffer_create 3 */ 4 Buffer_Handle Buffer_create(Int32 size, Buffer_Attrs *attrs) 5 6 Buffer_Handle hBuf; 7 UInt32 obj

44、Size; 8 9 if (attrs = NULL) 10 Dmai_err0("Must provide attrsn");11 return NULL;12 13 14 if (attrs->type != Buffer_Type_BASIC &&15 attrs->type != Buffer_Type_GRAPHICS) 16 17 Dmai_err1("Unknown Buffer type (%d)n", attrs->type);18 return NULL;19 20 21 objSize = attr

45、s->type = Buffer_Type_GRAPHICS ? sizeof(_BufferGfx_Object) :22 sizeof(_Buffer_Object);23 24 hBuf = (Buffer_Handle) calloc(1, objSize);25 26 if (hBuf = NULL) 27 Dmai_err0("Failed to allocate space for Buffer Objectn");28 return NULL;29 30 31 _Buffer_init(hBuf, size, attrs);32 33 if (!attrs->referenc