OpenGL 视频播放器的纹理缓冲区
我正在使用 OpenGL、Ffmpeg 和 SDL 来播放视频，目前正在优化获取帧、解码帧、将帧从 YUV 转换为 RGB、将帧上传到纹理并在四边形上显示纹理的过程。每个阶段都由单独的线程执行，并写入由 SDL 互斥体和条件控制的共享缓冲区（纹理的上传和显示除外，因为它们需要位于同一上下文中）。
我让播放器在单独的线程上很好地处理解码、转换和 OpenGL 上下文，但意识到因为视频是每秒 25 帧，所以我只能从缓冲区获取转换后的帧，将其上传到 OpenGL 并绑定/显示它OpenGL 线程中的时间为 40 毫秒。由于存在 40 毫秒的间隙，渲染循环会循环大约 6-10 次，不会为显示的每一帧显示下一帧。
因此，我认为最好也为纹理提供一个缓冲区，并设置一个使用 glGenTextures() 和我需要的 glParameters 等创建和初始化的纹理数组。
当自上次帧刷新以来尚未经过 40 毫秒时，运行一个方法，该方法从转换缓冲区中获取下一个转换后的帧，并通过绑定它然后调用 glTexSubImage2D() 将其上传到纹理缓冲区中的下一个空闲纹理。当距离上一帧刷新已过去 40 毫秒时，会运行一个单独的方法，从纹理缓冲区中获取下一个 GLuint 纹理，并将其与 glBindTexture() 绑定。因此，我只是将之前所做的事情（从转换缓冲区抓取、上传、显示）分解为单独的方法（从转换缓冲区抓取、上传到纹理缓冲区 | 和 | 从纹理缓冲区抓取、显示）以利用40ms 刷新之间浪费的时间。
这听起来合理吗？因为运行时，视频总是以零星的方式停止，有时在应该播放的时候播放大约 4 帧（每 40 毫秒），但随后出现 2 秒的间隙，然后显示 1 帧，然后出现 3 秒的间隙而且视频完全无法观看。
该代码与我管理转换线程的方式几乎相同，从解码缓冲区抓取解码帧，将它们从 YUV 转换为 RGB，然后将它们放入转换缓冲区，因此看不到巨大的瓶颈可能在哪里。
瓶颈可能出在 OpenGL 方面吗？事实上，我将新图像数据存储到 10 个不同的纹理，问题是当从纹理缓冲区抓取新纹理时，就视频内存上的内存位置而言，原始数据可能与上一个纹理相距一百万英里？这是我唯一的答案尝试，但我不太了解 OpenGL 内部是如何工作的，所以这就是我在这里发帖的原因。
有人有什么想法吗？
I am using OpenGL, Ffmpeg and SDL to play videos and am currently optimizing the process of getting frames, decoding them, converting them from YUV to RGB, uploading them to texture and displaying the texture on a quad. Each of these stages is performed by a seperate thread and they written to shared buffers which are controlled by SDL mutexes and conditions (except for the upload and display of the textures as they need to be in the same context).
I have the player working fine with the decode, convert and OpenGL context on seperate threads but realised that because the video is 25 frames per second, I only get a converted frame from the buffer, upload it to OpenGL and bind it/display it every 40 milliseconds in the OpenGL thread. The render loop goes round about 6-10 times not showing the next frame for every frame it shows, due to this 40ms gap.
Therefore I decided it might be a good idea to have a buffer for the textures too and set up an array of textures created and initialised with glGenTextures() and the glParameters I needed etc.
When it hasn't been 40ms since the last frame refresh, a method is ran which grabs the next converted frame from the convert buffer and uploads it to the next free texture in the texture buffer by binding it then calling glTexSubImage2D(). When it has been 40ms since the last frame refresh, a seperate method is ran which grabs the next GLuint texture from the texture buffer and binds it with glBindTexture(). So effectively, I am just splitting up what was being done before (grab from convert buffer, upload, display) into seperate methods (grab from convert buffer, upload to texture buffer | and | grab from texture buffer, display) to make use of the wasted time between 40ms refreshes.
Does this sound reasonable? Because when ran, the video halts all the time in a sporadic manner, sometimes about 4 frames are played when they are supposed to (every 40ms) but then there is a 2 second gap, then 1 frame is shown, then a 3 second gap and the video is totally unwatchable.
The code is near identical to how I manage the convert thread grabbing decoded frames from the decode buffer, converting them from YUV to RGB and then putting them into the convert buffer so can't see where the massive bottlenecking could be.
Could the bottlenecking be on the OpenGL side of things? Is the fact that I am storing new image data to 10 different textures the problem as when a new texture is grabbed from the texture buffer, the raw data could be a million miles away from the last one in terms of memory location on the video memory? That's my only attempt at an answer, but I don't know much about how OpenGL works internally so that's why I am posting here.
Anybody have any ideas?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
我不是 OpenGL 专家，但我对瓶颈的猜测是纹理在系统内存中正确初始化，但在“错误”的时间发送到视频内存（就像一次全部而不是尽快），导致停滞管道。使用 glTexSubImage2D 时，在绑定纹理之前，您无法保证纹理何时到达视频内存。
谷歌搜索似乎像素缓冲区对象可以让您更好地控制它们何时位于视频内存中： http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&Number=262523
I'm no OpenGL expert, but my guess of the bottleneck is that the textures are intialized properly in system memory but are sent to the video memory at the 'wrong' time (like all at once instead of as soon as possible), stalling the pipeline. When using glTexSubImage2D you have no guarantees about when a texture arrives in video memory until you bind it.
Googling around it seems pixelbuffer objects give you more control about when they are in video memory: http://www.opengl.org/discussion_boards/ubbthreads.php?ubb=showflat&Number=262523