gstreamer的簡易音視頻同步方法

1、- 1 -Gstreamer 的簡易音同步方法 概述Gstreamer 的音頻同步，概括起來是一個比較大，因為在網(wǎng)上可以看到很多音同步的辦法。這里只看最普通的一種。以音頻時鐘做為參考時鐘(要求參考時鐘上的時間是線性遞增的)；生成數(shù)據(jù)流時依據(jù)參考時鐘上的時間給每個數(shù)據(jù)塊都打上時間戳(一般包括開始時間和結(jié)束時間)；在時，數(shù)據(jù)上的時間戳，同時參考當(dāng)前參考時鐘上的時間來安排(如果數(shù)據(jù)塊上的時間大于參考時鐘的時間，則不急于，直到參考時鐘達(dá)到數(shù)據(jù)塊的開始時間；如果數(shù)據(jù)塊上的時間小于參考時鐘的時間，丟棄該數(shù)據(jù)塊，以使得趕上進(jìn)度。)Gstreamer 的因分離器如下圖：demux element 將音頻，分離

2、后，給各自的器進(jìn)行。1.+-+二提供時鐘默認(rèn)情況下，是由 Audio Sink 來提供參考時鐘的。下面開始代碼之旅：1./* gst-plugins-base-0.10.32/gst-libs/gst/audio/gstbaseaudiosink.c */：TCL Multimedia Technology Holdings Limited Multimedia Building, TCL ernational E City,#1001 Zhongshanyuan Road, Nanshan District, Shenzhen, Guangdong China2./*默認(rèn)的情況下是由這個 e

3、lement 來提供 clock 的。*/3. #define DEFAULT_PROVIDE_CLOCKTRUE4. sic void gst_base_audio_sink_init (GstBaseAudioSink * baseaudiosink, GstBaseAudioSinkClass * g_class 5.baseaudiosink-provide_clock = DEFAULT_PROVIDE_CLOCK6./* 這里在 clock 類里面新建了一個時鐘 */7.baseaudiosink-provided_clock = gst_audio_clock_new (GstA

4、udioSinkClock,8.(GstAudioClockGetTimeFunc) gst_base_audio_sink_get_time, baseaudiosink);2.|Audio|3.+-|4./+-+5.+-+ /6.| demux|/7.|8.+-+ 9.+-+10.+-|11.|12.+-+則應(yīng)盡快或者干脆- 2 -9.三如何同步？以我實驗的為例，使用的是 xvimagesink element 它的繼承關(guān)系如下：1.GObject從 element 的 chain func 開始(PS:為什么從 chain 開始，參考Gstreamer 初見)1./*：TCL Multi

5、media Technology Holdings Limited Multimedia Building, TCL ernational E City,#1001 Zhongshanyuan Road, Nanshan District, Shenzhen, Guangdong China2. * gst-plugins-base/sys/xvimage/xvimagesink.c3. * gst-plugins-base/gst-libs/gst/gstsink.c4. * 這兩個文件里都沒有 chain 函數(shù).5. * 在 gstreamer-0.10.32/libs/gst/base/

6、gstbasesink.c 中 chain 函數(shù)為6. */7. sic GstFlowReturn gst_base_sink_chain (GstPad * pad, GstBuffer * buf) 8.basesink = GST_BASE_SINK (GST_OBJECT_PARENT (pad);2.+GstObject3.+-Gsement4.+GstBaseSink5.+-GstSink6.+GstXvImageSink10.11. /*12. * 查詢是否 sink 將提供 clock13. */14. ggst_base_audio_sink_get_provide_clo

7、ck (GstBaseAudioSink * sink) 15.gresult;16.result = sink-provide_clock;17.return result;18. 19.20. /* 查詢 clock 的時間21. * 如果將這里的返回結(jié)果變慢，那么就會變慢。當(dāng)然和音頻就不同步了。22. */23. sic GstClockTime gst_base_audio_sink_get_time (GstClock * clock, GstBaseAudioSink * sink) 24.result = gst_util_u64_scale_(sles, GST_SECOND,

8、 sink-ringbuffer-spec.rate);25.return result;26. - 3 -9.10.return gst_base_sink_chain_main (basesink, pad, _PR_IS_BUFFER, buf);11.12.13.sic GstFlowReturn gst_base_sink_chain_main (GstBaseSink * basesink, GstPad * pad,14.gu8 obj_type, gpoer obj) 15.result = gst_base_sink_chain_unlocked (basesink, pad

9、, obj_type, obj);16.17.18.sic GstFlowReturn gst_base_sink_chain_unlocked (GstBaseSink * basesink, GstPad * pad,19.gu8 obj_type, gpoer obj) 20.result = gst_base_sink_queue_object_unlocked (basesink, pad,obj_type, obj, TRUE);21.22.23.sicGstFlowReturn gst_base_sink_queue_object_unlocked (GstBaseSink *

10、basesink,24.GstPad* pad,25.gu8 obj_type, gpoer obj, gprerollable) 26.while (G_UNLIKELY (!g_queue_is_empty (q) 27.ret = gst_base_sink_render_object (basesink, pad, ot, o);28.29.30.31./*gstreamer-0.10.32/libs/gst/base/gstbasesink.c */32.sic GstFlowReturn gst_base_sink_render_object (GstBaseSink * base

11、sink, GstPad * pad,33.gu8 obj_type, gpoer obj) 34./* 這里開始做同步，同步成功后，才開始*/35.ret = gst_base_sink_do_sync (basesink, pad, sync_obj, &late, &step_end, obj_type);36.if(G_UNLIKELY (ret != GST_FLOW_OK)37.goto sync_failed;38.39.if(!OBJ_IS_BUFFERLIST (obj_type) 40.ret = bclass-render (basesink, buf);41. else

12、 42.ret = bclass-render_list (basesink, buflist);43.44.45.46.sic GstFlowReturn gst_base_sink_do_sync (GstBaseSink * basesink, GstPad * pad,47.GstMiniObject * obj, g* late, g* step_end, gu8 obj_type) ：TCL Multimedia Technology Holdings Limited Multimedia Building, TCL ernational E City,#1001 Zhongsha

13、nyuan Road, Nanshan District, Shenzhen, Guangdong China- 4 -這里同步完成，其實這里還有最后一個小問題，那么就是 AudioClock 是以什么為時鐘的呢。其實就是以聲卡的時鐘為時鐘的。因為聲卡有時鐘同步功能。所以時間。計算一共了多少個 sle，就可以計算出當(dāng)前了多長的：TCL Multimedia Technology Holdings Limited Multimedia Building, TCL ernational E City,#1001 Zhongshanyuan Road, Nanshan District, Shenz

14、hen, Guangdong China48.sus = gst_base_sink_wait_clock (basesink, stime, &jitter);49.return GST_FLOW_OK;50. 51. /*52. * time: the running_time to be reached* jitter: (out) (allow-none): the jitter to be filled with time diff, or NULL* This function will block until time is reached. It is usually called by55. * subclassest use their ownernal synchronisation.56. */57. GstClockReturn gst_base_sink_wait_clock (GstBaseSink * sink, GstClockTime time,58.GstClockTimeDiff * jitter) 59.if (G_UNLIKELY (clock = GST_ELEMENT_CLOCK (sink) = NULL)60.goto