如何在 RTSP 源过滤器中生成样本时间戳？
实际上，我已经能够生成时间戳并且它可以与一些过滤器（复用器）一起使用，但由于我希望能够使用 GDCL MP4 Multiplexer Filter ，所以我想讨论一下计算采样时间的方法价值观。
奇怪的是，如果我将 Smart Tee Filter 放在 RTSP Source Filter 和 GDCL MP4 Multiplexer Filter 之间，一切似乎都能正常工作。我能够正确捕获视频流。但如果没有智能 T 恤过滤器，我仍然能够捕获视频，但这次会出现周期性故障。据我所知，Smart Tee Filter 除了提供双输出引脚之外没有太多作用，一个带有时间戳（捕获引脚），一个没有时间戳（预览引脚），并且两个引脚共享相同的流缓冲区。因此，我想到的事情是，Smart Tee Filter 正在以某种方式重新组织时间戳等内容。但如果我不生成时间戳，Smart Tee Filter 也不会生成这些值。我当前的猜测是，我正确地计算了帧开始时间，但错误地计算了帧停止时间，并且 Smart Tee Filter 正在重新计算帧停止时间，因为它们应该是这样的（当然，这只是一个猜测）。
我曾经像下面的公式一样计算开始和停止时间。
startTime = now
timeDelta = now - previousFrame
endTime = startTime + timeDelta
这不是确切的公式，但很接近。结果如下。
Media Time/Time: 0-1 0-0
Media Time/Time: 1-2 500028-1000056
Media Time/Time: 2-3 930053-1360078
Media Time/Time: 3-4 1610092-2290131
Media Time/Time: 4-5 2200126-2790160
Media Time/Time: 5-6 2900166-3600206
Media Time/Time: 6-7 3500200-4100234
Media Time/Time: 7-8 4240242-4980284
Media Time/Time: 8-9 4720270-5200298
Media Time/Time: 9-10 5350306-5980342
Media Time/Time: 10-11 5980342-6610378
Media Time/Time: 11-12 6610378-7240414
Media Time/Time: 12-13 7250414-7890450
Media Time/Time: 13-14 7880450-8510486
Media Time/Time: 14-15 8510486-9140522
Media Time/Time: 15-16 9140522-9770558
Media Time/Time: 16-17 9780559-10420596
Media Time/Time: 17-18 10410595-11040631
Media Time/Time: 18-19 11040631-11670667
Media Time/Time: 19-20 11680668-12320705
Media Time/Time: 20-21 12310704-12940740
Media Time/Time: 21-22 12940740-13570776
Media Time/Time: 22-23 13600778-14260816
Media Time/Time: 23-24 14220813-14840848
Media Time/Time: 24-25 14840849-15460885
Media Time/Time: 25-26 15480885-16120921
Media Time/Time: 26-27 16110921-16740957
Media Time/Time: 27-28 16740957-17370993
Media Time/Time: 28-29 17380994-18021031
Media Time/Time: 29-30 18011030-18641066
Media Time/Time: 30-31 18631065-19251100
这些是通过 SetMediaTime() 和 SetTime() 方法设置的媒体时间和时间值（均为开始-停止格式）。我看到的问题是一些开始/结束时间在连续帧中重叠。当然，这是由抖动因素等引起的。如果我根据前一帧计算时间增量，并且如果下一帧比预期早到达，则会发生重叠。所以，我对我的代码做了一些改变。我的最终代码如下所示。
_FILETIME fileTime;
GetSystemTimeAsFileTime(&fileTime);
now = ((((__int64)fileTime.dwHighDateTime << 32) + fileTime.dwLowDateTime) - streamReader->rtpProtocolHandler->m_iReferenceTime);
timeDelta = now - m_iFrameTimePrevious;
m_iFrameTime = max(now, m_iFrameTime);
m_iFrameTimePrevious = m_iFrameTime;
REFERENCE_TIME rtStart = m_iFrameTime;
REFERENCE_TIME rtStop;
if(timeDelta > 0) {
m_iFrameTime += timeDelta;
rtStop = m_iFrameTime;
} else {
rtStop = rtStart;
}
pSample->SetTime(&rtStart, &rtStop);
pSample->SetMediaTime(&m_iMediaTime, &(++m_iMediaTime));
结果是这样的：
Media Time/Time: 0-1 0-0
Media Time/Time: 1-2 470027-940054
Media Time/Time: 2-3 940054-1380079
Media Time/Time: 3-4 1580091-2220128
Media Time/Time: 4-5 2220128-2810161
Media Time/Time: 5-6 2870164-3520200
Media Time/Time: 6-7 3520200-4110234
Media Time/Time: 7-8 4170239-4820278
Media Time/Time: 8-9 4820278-5420312
Media Time/Time: 9-10 5470313-6120348
Media Time/Time: 10-11 6120348-6720382
Media Time/Time: 11-12 6770387-7420426
Media Time/Time: 12-13 7420426-8060463
Media Time/Time: 13-14 8060463-8630494
Media Time/Time: 14-15 8630494-9140521
Media Time/Time: 15-16 9260530-9890566
Media Time/Time: 16-17 9890566-10490600
Media Time/Time: 17-18 10560604-11230642
Media Time/Time: 18-19 11230642-11840677
Media Time/Time: 19-20 11860679-12490716
Media Time/Time: 20-21 12490716-13090750
Media Time/Time: 21-22 13170754-13850792
Media Time/Time: 22-23 13850792-14450826
Media Time/Time: 23-24 14460827-15070862
Media Time/Time: 24-25 15070862-15680897
Media Time/Time: 25-26 15760902-16450942
Media Time/Time: 26-27 16450942-17060977
Media Time/Time: 27-28 17100978-17751014
Media Time/Time: 28-29 17751014-18171038
Media Time/Time: 29-30 18171040-18591066
Media Time/Time: 30-31 18771074-19371108
这次没有重叠，但所有的故障仍然存在。我做错了什么？
请记住，我的源过滤器可与其他 mp4 复用器配合使用，如果我将 Smart Tee 过滤器放在中间，它也可与 GDCL MP4 复用器配合使用。
Actually, I'm already able to generate timestamps and it works with some filters (muxers) but since I want to be able to use GDCL MP4 Multiplexer Filter, I want to discuss my method of calculating sample time values.
Weird thing is, if I put Smart Tee Filter between my RTSP Source Filter and GDCL MP4 Multiplexer Filter everything seems to by working. I'm able to capture video streams correctly. But without Smart Tee Filter, i'm still able to capture video but this time with periodic glitches. As far as I know, Smart Tee Filter doesn't do much except giving double output pins, one with timestamps (capture pin) and one without timestamps (preview pin) and both pins share the same stream buffer. So, the thing came to my mind was somehow Smart Tee Filter is reorganizing things like timestamps. But if I don't generate timestamps, Smart Tee Filter doesn't generate those values, either. My current guess is, I'm calculating frame start times correctly but frame stop times incorrectly and Smart Tee Filter is recalculating frame stop times as they should be (this is just a guess of course).
I used to calculate start and stop times like the formula below.
startTime = now
timeDelta = now - previousFrame
endTime = startTime + timeDelta
This is not the exact formula but it was something close. And results were like below.
Media Time/Time: 0-1 0-0
Media Time/Time: 1-2 500028-1000056
Media Time/Time: 2-3 930053-1360078
Media Time/Time: 3-4 1610092-2290131
Media Time/Time: 4-5 2200126-2790160
Media Time/Time: 5-6 2900166-3600206
Media Time/Time: 6-7 3500200-4100234
Media Time/Time: 7-8 4240242-4980284
Media Time/Time: 8-9 4720270-5200298
Media Time/Time: 9-10 5350306-5980342
Media Time/Time: 10-11 5980342-6610378
Media Time/Time: 11-12 6610378-7240414
Media Time/Time: 12-13 7250414-7890450
Media Time/Time: 13-14 7880450-8510486
Media Time/Time: 14-15 8510486-9140522
Media Time/Time: 15-16 9140522-9770558
Media Time/Time: 16-17 9780559-10420596
Media Time/Time: 17-18 10410595-11040631
Media Time/Time: 18-19 11040631-11670667
Media Time/Time: 19-20 11680668-12320705
Media Time/Time: 20-21 12310704-12940740
Media Time/Time: 21-22 12940740-13570776
Media Time/Time: 22-23 13600778-14260816
Media Time/Time: 23-24 14220813-14840848
Media Time/Time: 24-25 14840849-15460885
Media Time/Time: 25-26 15480885-16120921
Media Time/Time: 26-27 16110921-16740957
Media Time/Time: 27-28 16740957-17370993
Media Time/Time: 28-29 17380994-18021031
Media Time/Time: 29-30 18011030-18641066
Media Time/Time: 30-31 18631065-19251100
These are Media Time and Time values (both in start-stop format) which are set by SetMediaTime() and SetTime() methods. The problem I've seen was some start/end times were overlapping in sequential frames. Of course that is caused by jitter factors etc. If I calculate the time delta depending on the previous frame and if next frame arrives earlier than expected, overlapping occurs. so, i made a little change in my code. my final code is like below.
_FILETIME fileTime;
GetSystemTimeAsFileTime(&fileTime);
now = ((((__int64)fileTime.dwHighDateTime << 32) + fileTime.dwLowDateTime) - streamReader->rtpProtocolHandler->m_iReferenceTime);
timeDelta = now - m_iFrameTimePrevious;
m_iFrameTime = max(now, m_iFrameTime);
m_iFrameTimePrevious = m_iFrameTime;
REFERENCE_TIME rtStart = m_iFrameTime;
REFERENCE_TIME rtStop;
if(timeDelta > 0) {
m_iFrameTime += timeDelta;
rtStop = m_iFrameTime;
} else {
rtStop = rtStart;
}
pSample->SetTime(&rtStart, &rtStop);
pSample->SetMediaTime(&m_iMediaTime, &(++m_iMediaTime));
And results are like this:
Media Time/Time: 0-1 0-0
Media Time/Time: 1-2 470027-940054
Media Time/Time: 2-3 940054-1380079
Media Time/Time: 3-4 1580091-2220128
Media Time/Time: 4-5 2220128-2810161
Media Time/Time: 5-6 2870164-3520200
Media Time/Time: 6-7 3520200-4110234
Media Time/Time: 7-8 4170239-4820278
Media Time/Time: 8-9 4820278-5420312
Media Time/Time: 9-10 5470313-6120348
Media Time/Time: 10-11 6120348-6720382
Media Time/Time: 11-12 6770387-7420426
Media Time/Time: 12-13 7420426-8060463
Media Time/Time: 13-14 8060463-8630494
Media Time/Time: 14-15 8630494-9140521
Media Time/Time: 15-16 9260530-9890566
Media Time/Time: 16-17 9890566-10490600
Media Time/Time: 17-18 10560604-11230642
Media Time/Time: 18-19 11230642-11840677
Media Time/Time: 19-20 11860679-12490716
Media Time/Time: 20-21 12490716-13090750
Media Time/Time: 21-22 13170754-13850792
Media Time/Time: 22-23 13850792-14450826
Media Time/Time: 23-24 14460827-15070862
Media Time/Time: 24-25 15070862-15680897
Media Time/Time: 25-26 15760902-16450942
Media Time/Time: 26-27 16450942-17060977
Media Time/Time: 27-28 17100978-17751014
Media Time/Time: 28-29 17751014-18171038
Media Time/Time: 29-30 18171040-18591066
Media Time/Time: 30-31 18771074-19371108
No overlapping this time but all the glitches remains. what am I doing wrong?
Please remember that my source filter works with other mp4 muxers and it also works with GDCL MP4 Muxer if I put Smart Tee filter in the middle.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
当使用 RTSP/RTP/RTCP 时，我建议使用 RTP 时间戳作为示例演示时间（偏移为零）。
如何确定每一帧的持续时间完全相同？
根据我的经验，无论是使用实时源还是基于文件的媒体，情况都从未如此。您的用例当然可能有所不同。
此外，通过计算时间戳，您无法同步多个 RTP 流（例如音频和视频）。这是有 RTCP 发送方报告的原因之一。
When using RTSP/RTP/RTCP I would recommend using the RTP timestamps for sample presentation times (offset to zero).
How can you be sure that each frame is exactly the same duration?
In my experience, this has never been the case, whether using live sources or file-based media. Your use-case may of course differ.
Also, by calculating the timestamps you have no way of synchronising multiple RTP streams, say audio and video. That is one of the reasons for having RTCP sender reports.
问题不是由时间戳引起的，而是某种缓冲区损坏引起的。 Smart Tee 过滤器可能通过丢弃无效帧或提供源过滤器中缺少的帧队列来解决问题。当我使用适当的队列机制更改有缺陷的共享缓冲区时，一切都开始正常工作。
感谢 Geraint Davies 的大力支持和他的调试工具来解决我的问题。
Problem was not caused by the timestamps but some kind of a buffer corruption. Smart Tee filter was fixing the problem probably by dropping invalid frames or supplying a frames queue which was missing at my source filter. When I changed my buggy shared buffer with a proper queue mechanism, everything started to work correctly.
Thanks to Geraint Davies for his great support and his debugging tools for solving my problem.