如何渲染“视频”在嵌入式设备上的 Qt / X11 中
我目前正在尝试编写一个应用程序来显示来自专用 USB 设备的实时“视频”。 USB 设备提供大约。 10 帧/秒的 320x240 视频，采用灰度级数组。该应用程序是在运行 Linux 和 X11 的 iMx51 (800Mhz arm) 平台上用 Qt 编写的。我的应用程序需要显示此视频，缩放至 2 倍（即 640x480）并使用尽可能少的 CPU。
目前，图像显示为一个简单的 QLabel，每次从设备传输新的视频帧时，它都会更新其像素图。当 QLabel 以 320x240 显示时，视频播放流畅，应用程序使用 20% 的 CPU。
当我们尝试将 QLabel 的大小扩展到 640x480 时，CPU 使用率会达到峰值，以 10 帧/秒的速度消耗所有可用资源。我们减少 CPU 使用率的唯一方法是丢帧。
我们尝试编写一个带有自定义绘制事件的小部件来自己绘制视频图像的内容，但没有成功，这种技术比 QLabel 使用的 CPU 略多。
我们执行此操作的其他产品能够将图像直接传输到原始帧缓冲区（无 x11），尽管它们的 CPU 速度慢得多，但它们可以毫无问题地处理此任务。看起来 X11 和 Qt 给我们增加了巨大的开销。
必须有一种正确的方法来通过 Qt 和 X11 显示不断更新的图像，并且资源不那么密集。我们正在探索在该设备上启动并运行 openGL ES 的可能性，但这暂时可能是不可能的。 Qt 框架中是否缺少一些东西可以让我完成这个相对简单的任务？
I am currently trying to write an application that displays live "video" from a specialized USB device. The USB device provides approx. 10 frames/sec of 320x240 video in an array of gray levels. The application is written in Qt on an iMx51 (800Mhz arm) platform running linux and X11. My application needs to display this video, scaled by a factor of 2 (so 640x480) and use as little CPU as possible.
Currently, the images are displayed as a simple QLabel which updates it's pixmap every time a new video frame has been transmitted from the device. When the QLabel is being displayed at 320x240 the video plays smoothly and the application uses 20% of the CPU.
When we attempt to scale the QLabel to 640x480 in size the CPU usage spikes, consuming all available resources at 10frames/sec. Our only means of reducing the CPU usage is to drop frames.
We have tried writing a widget with a custom paint event to paint the contents of the video image ourselves to no avail, this technique used slightly more CPU than the QLabel.
Our other products that do this have the capability to blit the image directly to a raw frame buffer (no x11) and despite their much slower CPU's they can handle this task without any problem. It seems that X11 and Qt is adding an immense amount of overhead for us.
There must be a PROPER way to display a constantly updating image through Qt and X11 that is not so resource intensive. We are exploring the possibility of getting openGL ES up and running on the unit but that may be out of the question for the time being. Is there something I'm missing in the Qt Framework that will allow me to accomplish this relatively simple task?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
下载并阅读 VLC 源代码后，我决定使用 X11 的 Xvideo 扩展。 VLC 源代码包含一个 xvideo 输出“引擎”，使用 xcb 与 xserver 进行通信。我以此为例来编写我自己的解决方案。
在我们的平台上，xvideo 扩展的缩放和显示功能是硬件加速的。
我打算在时间允许的情况下尽快提供教程和示例代码。
After downloading and reading through the VLC source code I settled on using the Xvideo extension for X11. The VLC source code contains an xvideo output "engine" using xcb to communicated with the xserver. I used this as an example to code my own solution.
On our platform the scaling and display features of the xvideo extension are hardware accelerated.
I intend to provide a tutorial and example code as soon as time allows.
你的分辨率问题是 CPU 比 GPU 慢得多，因此渲染带有标签的帧确实是一个坏主意。当您提高分辨率时，CPU 必须处理的数据大小会呈指数级增加。处理视频时，您必须使用运行速度更快的 GPU。
我建议您使用 Qt OpenGL 模块中的 QGL 小部件 ( http://doc.qt .nokia.com/4.7/qtopengl.html）。
您可以从设备读取帧，将它们移动到视频内存，使用 GPU 渲染它们（如果您学习/知道如何操作，还可以应用滤镜和效果），从而节省大量 CPU 时间并减少解决问题。
Your problem with resolution is that the CPU is far much slower than the GPU, therefore rendering the frames with labels is really a bad idea. When you increase the resolution, you exponentially increase the size of the data that the CPU will have to work on.. When working with video you have to use the GPU which runs much faster.
I suggest you to use the QGL widget from the Qt OpenGL module ( http://doc.qt.nokia.com/4.7/qtopengl.html ).
You can read the frames from your device, move them to the video memory, use the GPU to render them (you can also apply filters and effects if you learn/know how to do it), saving a lot of CPU time and decreasing your resolution problems.
不要使用 QLabel 来显示移动内容。相反，子类化 QWidget 并实现您自己的小部件。另外，您可能想尝试直接绘画（QWidget 的标志）以避免双缓冲等。
当然，使用 GPU 进行缩放等很好，但在我看来，您仍然需要移动所有数据使用 CPU 从设备到 GPU。也就是说，您仍然会使用相当多的周期来移动 1 和 0。
Do not use a QLabel to display moving contents. Instead, sub-class QWidget and implement your own widget. Also, you might want to experiement with direct painting (flags for QWidget) to avoid double buffering, etc.
Of course, using a GPU for scaling and such is good, but it sounds to me as if you still have to move all the data from the device to the GPU using the CPU. I.e. you will still be using quite a few cycles just to move ones and zeros.
“Plain”X11 由于其客户端-服务器架构，位图渲染效率非常低，您需要使用 MIT-SHM 扩展。根据 Qt 版本，它可能包括本地使用 MIT-SHM 的“光栅”图形引擎。
另请参阅在没有OpenGL的情况下在X11上的QT中显示图像的最快方法是什么？ - 最终我设法变得非常不错Qt 的性能。诀窍是观察您复制位图的次数并尝试将其最小化。
请注意，您实际上并不需要 OpenGL 来处理位图，如果您有适合您的硬件的驱动程序，您可以尝试 2D 加速。您拥有共享内存，因此对于位图渲染，您的 GPU 与 CPU 一样快，如果您使用其他硬件进行位图渲染（本质上是内存到内存的传输），它只会节省一些 CPU 周期。
"Plain" X11 is very inefficient for bitmap rendering due to its client-server architecture, you need to use MIT-SHM extension. Depending on the Qt version, it may include "raster" graphic engine which is using MIT-SHM natively.
See also What is the fastest way to display an image in QT on X11 without OpenGL? - eventually I managed to get very decent performance from Qt. The trick is to watch how many times you copy your bitmap and try to minimize it.
Note that you don't really need OpenGL for bitmaps, you could try 2D acceleration if you have a suitable driver for your hardware. You have shared memory, so for bitmap rendering your GPU is as fast as CPU, it would just save you some CPU cycles if you do bitmap rendering (essentially, a memory to memory transfer) using some other hardware.