为什么视频播放器会丢失实时 RTP 流的一些前帧？
我写了一个RTSP服务器。它通过 RTP/UDP 发送 h246/AAC 流数据。视频的 RTP 发送数据包间隔为 30 毫秒，音频的 RTP 发送数据包间隔为 20 毫秒。时间戳是从 flv 标签中提取的（我的服务器从 flv 文件获取视频和音频数据）。视频播放器丢失前几个视频帧。因此，音频比视频领先几秒钟。这是为什么呢？在服务器端流式传输之前我应该暂停一下吗？
I have written an RTSP server. It sends an h246/AAC stream data over RTP/UDP. The RTP send packet interval for video is 30ms, for audio is 20 ms. Timestamps are extracted from the flv-tag (My server gets video and audio data from flv file). The video player looses the first few video frames. As a result audio is ahead of video by a few seconds. Why is this? Should i do any pause before streaming on server side?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
有几种可能性：
UDP 是一种不可靠的协议。您可以检查 RTP 序列号以查看是否属于这种情况以及丢弃了多少/哪些帧。可能有助于最大程度地减少 UDP 数据包丢失的方法是增加客户端上的 UDP 接收缓冲区大小。 这里是如何执行此操作的示例linux。当然，您可以在 Windows 上执行相同的操作。
客户端只有收到 IDR 帧后才能正确解码视频。在此之前，它无法正确解码视频。您流式传输到新客户端的第一帧是否是 IDR 帧（请记住它仍然可能丢失）。
无论如何，在我看来，您的视频播放器应用程序还存在另一个问题：即使丢帧，播放器也负责缓冲和同步音频和视频，并且应该能够做到这一点，而不管数据包丢失。
纯粹为了提供信息，您还可以通过 RTSP（因此通过 TCP）实现 RTP/RTCP 交错。这样您就不必担心丢帧。类似 live555 媒体流库和 VLC 支持此功能。
回答你关于暂停的最后一个问题：不，这与它无关。 RTSP 纯粹是信令协议。数据包丢失将发生在传输（UDP）层。
There are a couple of possibilities:
UDP is an unreliable protocol. You can check the RTP sequence numbers to see if this is the case and how many/what frames are dropped. What might help to minimise the UDP packet loss is to increase the UDP receive buffer size on the client. Here is an example of how to do it on linux. You can do the same thing on windows of course.
The client can only decode the video correctly once it has received the IDR frame. Until that point it can not correctly decode the video. Is the first frame you stream to a new client an IDR-frame (keeping in mind that it can still be lost).
In any case, is seems to me like there is also another problem with your video player application: even if the frames are dropped the player is responsible for buffering and synchronising audio and video and should be able to do this irrespective of packet loss.
On a purely informative note, you could also implement RTP/RTCP interleaved over RTSP (and hence over TCP). That way you don't have to worry about dropped frames. Libraries like the live555 media streaming lib and VLC support this.
To answer your last question about the pause: no, that has nothing to do with it. RTSP is purely the signaling protocol. Packet loss will be occurring in the transport (UDP) layer.