Gstreamer：暂停/恢复 RTP 流中的视频

发布于 2024-10-12 12:34:13 字数 1269 浏览 1 评论 0原文

我正在构建一个 gstreamer 管道，该管道从网络源接收两个 RTP 流：

ILBC 音频流 + 相应的 RTCP 流
H263 视频流 + 相应的 RTCP 流

所有内容都放入一个 gstreamer 管道中，因此它将使用两个流中的 RTCP 来同步音频/视频。到目前为止，我已经想出了这个（使用 gst-launch 进行原型设计）：

gst-launch -vvv  gstrtpbin name=rtpbin
  udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H263-2000" port=40000 ! rtpbin.recv_rtp_sink_0
  rtpbin. ! rtph263pdepay ! ffdec_h263 ! xvimagesink
  udpsrc port=40001 ! rtpbin.recv_rtcp_sink_0
  rtpbin.send_rtcp_src_0 ! udpsink port=40002 sync=false async=false

  udpsrc caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)PCMU,encoding-params=(string)1,octet-align=(string)1" port=60000 rtpbin.recv_rtp_sink_1
  rtpbin. ! rtppcmudepay ! autoaudiosink
  udpsrc port=60001 ! rtpbin.recv_rtcp_sink_1 
  rtpbin.send_rtcp_src_1 ! udpsink port=60002 sync=false async=false

如果网络源开始发送视频和音频，则此管道效果很好。如果视频流稍后暂停，gstreamer 仍将播放音频，甚至在网络源恢复视频流时开始播放视频。

然而，我的问题是，如果网络源仅以音频流开始（稍后可能会添加视频），则管道似乎会暂停/冻结，直到视频流也开始。

由于视频在我的应用程序中是可选的（并且可以由用户随意添加/删除），因此有什么方法可以连接例如“videotestsrc”，它将提供某种后备视频数据以保持管道在以下情况下运行：有没有网络视频数据？

我尝试过使用“videotestsrc”和一个名为“videomixer”的东西进行实验，但我认为混合器仍然需要两个流都处于活动状态。非常感谢任何反馈！

原文

I'm constructing a gstreamer pipeline that receives two RTP streams from an networked source:

ILBC Audio stream + corresponding RTCP stream
H263 Video stream + corresponding RTCP stream

Everything is put into one gstreamer pipeline so it will use the RTCP from both streams to synchronize audio/video. So far I've come up with this (using gst-launch for prototyping):

gst-launch -vvv  gstrtpbin name=rtpbin
  udpsrc caps="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H263-2000" port=40000 ! rtpbin.recv_rtp_sink_0
  rtpbin. ! rtph263pdepay ! ffdec_h263 ! xvimagesink
  udpsrc port=40001 ! rtpbin.recv_rtcp_sink_0
  rtpbin.send_rtcp_src_0 ! udpsink port=40002 sync=false async=false

  udpsrc caps="application/x-rtp,media=(string)audio,clock-rate=(int)8000,encoding-name=(string)PCMU,encoding-params=(string)1,octet-align=(string)1" port=60000 rtpbin.recv_rtp_sink_1
  rtpbin. ! rtppcmudepay ! autoaudiosink
  udpsrc port=60001 ! rtpbin.recv_rtcp_sink_1 
  rtpbin.send_rtcp_src_1 ! udpsink port=60002 sync=false async=false

This pipeline works well if the networked source starts out with sending both video and audio. If the videostream is paused later on, gstreamer will still playback audio and even will start playing back the video when the networked source resumes the video stream.

My problem is however that if the networked source starts out with only an audio stream (video might be added later on), the pipeline seems to pause/freeze until the video stream starts as well.

Since video is optional (and can be added/removed at will by the user) in my application, is there any way I can hook up for instance an 'videotestsrc' that will provide some kind of fallback video data to keep the pipeline running when there is no networked video data?

I've tried experimenting with 'videotestsrc' and a thing called 'videomixer' but I think that mixer still requires both streams to be alive. Any feedback is greatly appreciated!

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

与酒说心事 2024-10-19 12:34:13

我提出了一个简单的功能，可以通过更改垃圾箱来暂停恢复。在下面的示例中，我提供了动态更改目标 bin 的逻辑。我相信这不会完全阻止您所寻求的管道。类似的逻辑可用于 src bin。在这里，您可以删除网络源 bin 和相关的解码器/解复用器 bin，并添加 videotestsrc bin。

private static void dynamic_bin_replacement(Pipeline pipe, Element src_bin, Element dst_bin_new, Element dst_bin_old) {
pipe.pause();
src_bin.unlink(dst_bin_old);                     
pipe.remove(dst_bin_old);
pipe.add(dst_bin_new);
dst_bin_new.syncStateWithParent();
src_bin.link(dst_bin_new);
pipe.ready();                    
pipe.play();
}

您可能想尝试的另一个逻辑是“挂锁”。请查看以下帖子

http:// cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-block.txt

和

http://web.archiveorange.com/archive/v/8yxpz7FmOlGqxVYtkPb4

和

在 GStreamer 管道中添加和删除音频源

更新

尝试 输出选择器和输入选择器箱，因为它们似乎是更好的选择。我发现它们是最可靠的，并且与它们一起拥有巨大的运气。我分别使用fakesink或fakesrc作为选择器的另一端。
valve bin 是另一种替代方案，我发现它甚至不需要 fakesink 或 fakesrc bin。它也非常可靠。

还有媒体文件源的正确状态转换顺序

NULL ->准备好 ->暂停 ->播放（向上）

播放 ->暂停 ->准备好 -> NULL（向下）

上例中的顺序应该更正，其中ready() 应该在pause() 之前。另外，我倾向于认为取消链接应该在 null() 状态之后执行，而不是在 Pause() 之后执行。我还没有尝试过这些改变，但理论上它们应该有效。

有关详细信息，请参阅以下链接

http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-states.txt?h=BRANCH-RELEASE-0_10_19

I present a simple function for pause resume by changing bins. In the following example I provide the logic to change destination bin on the fly dynamically. This shall not completely stop the pipeline which is what you seek I believe. A similar logic could be used for src bins. Here you may remove your network source bin and related decoder/demux bins and add videotestsrc bins.

private static void dynamic_bin_replacement(Pipeline pipe, Element src_bin, Element dst_bin_new, Element dst_bin_old) {
pipe.pause();
src_bin.unlink(dst_bin_old);                     
pipe.remove(dst_bin_old);
pipe.add(dst_bin_new);
dst_bin_new.syncStateWithParent();
src_bin.link(dst_bin_new);
pipe.ready();                    
pipe.play();
}

The other logic you may want to try is "PADLOCKING". Please take a look at the following posts

http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-block.txt

and

http://web.archiveorange.com/archive/v/8yxpz7FmOlGqxVYtkPb4

and

Adding and removing audio sources to/from GStreamer pipeline on-the-go

UPDATE

Try output-selector and input-selector bins as they seem to be better alternative. I found them most reliable and have had immense luck with them. I use fakesink or fakesrc respectively as the other end of the selector.
valve bin is another alternative that I found doesn't even need fakesink or fakesrc bins. It is also extremely reliable.

Also the correct state transition order for media file source

NULL -> READY -> PAUSED -> PLAYING (Upwards)

PLAYING -> PAUSED -> READY -> NULL (Downwards)

My order in the above example should be corrected where ready() should come before pause(). Also I would tend to think un-linking should be performed after null() state and not after pause(). I haven't tried these changes but theoretically they should work.

See the following link for detailed info

http://cgit.freedesktop.org/gstreamer/gstreamer/tree/docs/design/part-states.txt?h=BRANCH-RELEASE-0_10_19

回复收藏 0 原文

~没有更多了~