将视频流式传输到多个源或从多个源传输视频
我想得到一些想法，比如你们中的一些人将如何解决这个问题。 我有一个机器人，它运行 Linux 并使用网络摄像头（带有 v4l2 驱动程序）作为其传感器之一。我用gtkmm写了一个控制面板。服务器和客户端都是用C++编写的。服务器是机器人，客户端是“控制面板”。图像分析正在机器人上进行，我想将视频从相机流回控制面板，原因有两个： A）为了好玩 B）覆盖图像分析结果
所以我的问题是，有哪些好的方法可以将视频从网络摄像头流式传输到控制面板以及优先考虑机器人代码来处理它？我对编写自己的视频压缩方案并将其通过现有的网络端口不感兴趣，我认为新的网络端口（专用于视频数据）将是最好的。问题的第二部分是如何在 gtkmm 中显示视频？视频数据异步到达，我无法控制 gtkmm 中的 main()，所以我认为这会很棘手。
我愿意使用 vlc、gstreamer 或任何其他我不知道的通用压缩库。
谢谢！
编辑： 该机器人拥有 1GHz 处理器，运行类似桌面版本的 Linux，但没有 X11。
I wanted to get some ideas one how some of you would approach this problem.
I've got a robot, that is running linux and uses a webcam (with a v4l2 driver) as one of its sensors. I've written a control panel with gtkmm. Both the server and client are written in C++. The server is the robot, client is the "control panel". The image analysis is happening on the robot, and I'd like to stream back the video from the camera to the control panel for two reasons:
A) for fun
B) to overlay image analysis results
So my question is, what are some good ways to stream video from the webcam to the control panel as well as giving priority to the robot code to process it? I'm not interested it writing my own video compression scheme and putting it through the existing networking port, a new network port (dedicated to video data) would be best I think. The second part of the problem is how do I display video in gtkmm? The video data arrives asynchronously and I don't have control over main() in gtkmm so I think that would be tricky.
I'm open to using things like vlc, gstreamer or any other general compression libraries I don't know about.
thanks!
EDIT:
The robot has a 1GHz processor, running a desktop like version of linux, but no X11.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
Gstreamer 几乎可以为您解决所有这些问题，只需很少的努力，并且还与 Glib 事件系统很好地集成。 GStreamer 包括 V4L 源插件、gtk+ 输出小部件、用于调整视频大小/编码/解码的各种过滤器，最重要的是，用于在机器之间移动数据的网络接收器和源。
对于原型，您可以使用“gst-launch”工具来组装视频管道并测试它们，然后在代码中以编程方式创建管道就相当简单了。搜索“GStreamer 网络流”即可查看人们使用网络摄像头等执行此操作的示例。
Gstreamer solves nearly all of this for you, with very little effort, and also integrates nicely with the Glib event system. GStreamer includes V4L source plugins, gtk+ output widgets, various filters to resize / encode / decode the video, and best of all, network sink and sources to move the data between machines.
For prototype, you can use the 'gst-launch' tool to assemble video pipelines and test them, then it's fairly simply to create pipelines programatically in your code. Search for 'GStreamer network streaming' to see examples of people doing this with webcams and the like.
我不确定所使用的实际技术，但如果您想避免丢帧，这最终可能会成为巨大的同步*****。我同时将视频流式传输到文件和网络。我最终所做的是使用一个带有三个指针的大循环缓冲区：一个写入和两个读取。有三个控制线程（以及一些额外的编码线程）：一个写入缓冲区，如果到达缓冲区中其他两个线程都没有读取的点，则会暂停；两个读取线程将从缓冲区读取并写入文件/网络（如果他们领先于生产者，则暂停）。由于所有内容都是作为帧写入和读取的，因此同步开销可以保持在最低限度。
我的制作人是一个转码器（来自另一个文件源），但在您的情况下，您可能希望相机以通常的任何格式生成整个帧，并且只为服务器进行转码（使用 ffmpeg 之类的东西），而机器人处理图像。
不过，您的问题有点复杂，因为机器人需要实时反馈，因此无法暂停并等待流媒体服务器赶上。因此，您可能希望尽快将帧发送到控制系统，并将一些帧分别缓冲在循环缓冲区中，以便流式传输到“控制面板”。某些编解码器比其他编解码器更好地处理丢帧，因此如果网络落后，您可以开始覆盖缓冲区末尾的帧（注意它们不会被读取）。
I'm not sure about the actual technologies used, but this can end up being a huge synchronization ***** if you want to avoid dropped frames. I was streaming a video to a file and network at the same time. What I eventually ended up doing was using a big circular buffer with three pointers: one write and two read. There were three control threads (and some additional encoding threads): one writing to the buffer which would pause if it reached a point in the buffer not read by both of the others, and two reader threads that would read from the buffer and write to the file/network (and pause if they got ahead of the producer). Since everything was written and read as frames, sync overhead could be kept to a minimum.
My producer was a transcoder (from another file source), but in your case, you may want the camera to produce whole frames in whatever format it normally does and only do the transcoding (with something like ffmpeg) for the server, while the robot processes the image.
Your problem is a bit more complex, though, since the robot needs real-time feedback so can't pause and wait for the streaming server to catch up. So you might want to get frames to the control system as fast as possible and buffer some up in a circular buffer separately for streaming to the "control panel". Certain codecs handle dropped frames better than others, so if the network gets behind you can start overwriting frames at the end of the buffer (taking care they're not being read).
当你说“一个新的视频端口”然后开始谈论 vlc/gstreaming 时，我发现很难弄清楚你想要什么。显然，这些软件包将有助于通过多种协议进行流式传输和压缩，但显然您需要一个“网络端口”而不是“视频端口”来发送流。
如果您真正的意思是通过无线视频/电视源发送显示输出，那就是另一回事了，但是您需要硬件专家而不是软件专家的建议。
继续前进。我已经通过 MMS/UDP 协议进行了大量的流传输，vlc 处理得很好（作为服务器和客户端）。然而，它是为台式机设计的，可能没有您想要的那么轻量级。我认为像 gstreamer、mencoder 或 ffmpeg 这样的东西会更好。机器人有什么样的CPU？如果您计划进行实时压缩，您将需要一些麻烦。
在客户端，我想您会发现许多用于处理 GTK 中视频的小部件。在担心接口细节之前我会先研究一下。
When you say 'a new video port' and then start talking about vlc/gstreaming i'm finding it hard to work out what you want. Obviously these software packages will assist in streaming and compressing via a number of protocols but clearly you'll need a 'network port' not a 'video port' to send the stream.
If what you really mean is sending display output via wireless video/tv feed that's another matter, however you'll need advice from hardware experts rather than software experts on that.
Moving on. I've done plenty of streaming over MMS/UDP protocols and vlc handles it very well (as server and client). However it's designed for desktops and may not be as lightweight as you want. Something like gstreamer, mencoder or ffmpeg on the over hand is going to be better I think. What kind of CPU does the robot have? You'll need a bit of grunt if you're planning real-time compression.
On the client side I think you'll find a number of widgets to handle video in GTK. I would look into that before worrying about interface details.