使用 gstreamer 和 ffmpeg 进行 H.264 解码
我正在使用 OPAL voip SIP 堆栈开发一个 voip 应用程序。
我正在重写一个名为 OpalLocalEndpoint 的类，并在 gstreamer 管道中读取/写入编码数据。为了读取，我从 appsink 获取 rtp 有效负载数据，为了写入，我将有效负载数据推送到 appsrc。
我从wireshark捕获了SDP文件。
这是该应用程序的客户端。
v=0
o=- 1319058426 1 IN IP4 192.168.0.71
s=Opal SIP Session
c=IN IP4 192.168.0.71
t=0 0
m=audio 5086 RTP/AVP 125 0 8 124 101
a=sendrecv
a=rtpmap:125 Speex/16000/1
a=fmtp:125 sr=16000,mode=any
a=rtpmap:0 PCMU/8000/1
a=rtpmap:8 PCMA/8000/1
a=rtpmap:124 Speex/8000/1
a=fmtp:124 sr=8000,mode=any
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16,32,36
m=video 5088 RTP/AVP 109 108 34 114
b=AS:4096
b=TIAS:4096000
a=sendrecv
a=rtpmap:109 h264/90000
a=fmtp:109 packetization-mode=1;profile-level-id=42C01E
a=rtpmap:108 h263-1998/90000
a=fmtp:108 D=1;F=1;I=1;J=1;CIF=1;CIF4=1;QCIF=1;CUSTOM=320,240,1;CUSTOM=640,480,1
a=rtpmap:34 h263/90000
a=fmtp:34 F=1;CIF=1;CIF4=1;QCIF=1
a=rtpmap:114 MP4V-ES/90000
a=fmtp:114 profile-level-id=5
这是回复客户端的服务器，
v=0
o=- 1319058099 1 IN IP4 192.168.0.215
s=HHP Video Codec/1.0
c=IN IP4 192.168.0.215
t=0 0
m=audio 5006 RTP/AVP 125 0 8 124
a=inactive
a=rtpmap:125 Speex/16000/1
a=rtpmap:0 PCMU/8000/1
a=rtpmap:8 PCMA/8000/1
a=rtpmap:124 Speex/8000/1
a=maxptime:20
m=video 5004 RTP/AVP 109
b=AS:2048
b=TIAS:2048000
a=sendrecv
a=rtpmap:109 h264/90000
a=fmtp:109 packetization-mode=1;profile-level-id=42c01e
我使用以下方法对数据进行编码：
v4l2src name=videoSrc ! video/x-raw-yuv, format=(fourcc)I420, width=352, height=288, framerate=(fraction)30/1 ! videobalance name=VideoBalance ! textoverlay name=chanNameFilter ! textoverlay name=osdMessageFilter ! textoverlay name=sessionTimerOverlay ! x264enc byte-stream=true bframes=0 b-adapt=0 tune=0x4 speed-preset=3 bitrate=256 sliced-threads=false profile=0 ! rtph264pay mtu=1412 ! appsink name=videoAppSink sync=false
并尝试使用以下方法对传入数据进行解码
appsrc is-live=true do-timestamp=false typefind=true name=videoAppSrc ! application/x-rtp, media=video, payload=109, clock-rate=90000, encoding-type=H264, byte-stream=true, access-unit=true ! rtph264depay ! ffdec_h264 ! xvimagesink name=videoOutputSink
但是，虽然编码数据显示在客户端上（一开始没有，但我必须添加所有这些属性直到它最终正确显示），我一直无法让解码端工作。
它显示的屏幕大部分是灰色的，带有粉红色、黄色和绿色的光点。有时我会得到更多一些合适的颜色，但大多数时候只有灰色。
如果我使用同样的管道与 VLC 交互，它工作得很好。我的猜测是我在某个地方搞砸了帽子。谁能就我应该寻找什么提供任何想法？
我对其他每个编码器也有同样的问题，即 theora、h263 等......尽管每个编码器的方式不同。
I have a voip application I am working on using the OPAL voip SIP stack.
I am overriding a class called OpalLocalEndpoint and reading/writing encoded data to and from my gstreamer pipelines. For reading, I grab the rtp payloaded data from an appsink, and for writing I push the payloaded data to the appsrc.
I captured the SDP files from wireshark.
Here is a client to the App.
v=0
o=- 1319058426 1 IN IP4 192.168.0.71
s=Opal SIP Session
c=IN IP4 192.168.0.71
t=0 0
m=audio 5086 RTP/AVP 125 0 8 124 101
a=sendrecv
a=rtpmap:125 Speex/16000/1
a=fmtp:125 sr=16000,mode=any
a=rtpmap:0 PCMU/8000/1
a=rtpmap:8 PCMA/8000/1
a=rtpmap:124 Speex/8000/1
a=fmtp:124 sr=8000,mode=any
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16,32,36
m=video 5088 RTP/AVP 109 108 34 114
b=AS:4096
b=TIAS:4096000
a=sendrecv
a=rtpmap:109 h264/90000
a=fmtp:109 packetization-mode=1;profile-level-id=42C01E
a=rtpmap:108 h263-1998/90000
a=fmtp:108 D=1;F=1;I=1;J=1;CIF=1;CIF4=1;QCIF=1;CUSTOM=320,240,1;CUSTOM=640,480,1
a=rtpmap:34 h263/90000
a=fmtp:34 F=1;CIF=1;CIF4=1;QCIF=1
a=rtpmap:114 MP4V-ES/90000
a=fmtp:114 profile-level-id=5
Here is the server replying back to the client
v=0
o=- 1319058099 1 IN IP4 192.168.0.215
s=HHP Video Codec/1.0
c=IN IP4 192.168.0.215
t=0 0
m=audio 5006 RTP/AVP 125 0 8 124
a=inactive
a=rtpmap:125 Speex/16000/1
a=rtpmap:0 PCMU/8000/1
a=rtpmap:8 PCMA/8000/1
a=rtpmap:124 Speex/8000/1
a=maxptime:20
m=video 5004 RTP/AVP 109
b=AS:2048
b=TIAS:2048000
a=sendrecv
a=rtpmap:109 h264/90000
a=fmtp:109 packetization-mode=1;profile-level-id=42c01e
I encode the data with:
v4l2src name=videoSrc ! video/x-raw-yuv, format=(fourcc)I420, width=352, height=288, framerate=(fraction)30/1 ! videobalance name=VideoBalance ! textoverlay name=chanNameFilter ! textoverlay name=osdMessageFilter ! textoverlay name=sessionTimerOverlay ! x264enc byte-stream=true bframes=0 b-adapt=0 tune=0x4 speed-preset=3 bitrate=256 sliced-threads=false profile=0 ! rtph264pay mtu=1412 ! appsink name=videoAppSink sync=false
And attempt to decode the incoming data with
appsrc is-live=true do-timestamp=false typefind=true name=videoAppSrc ! application/x-rtp, media=video, payload=109, clock-rate=90000, encoding-type=H264, byte-stream=true, access-unit=true ! rtph264depay ! ffdec_h264 ! xvimagesink name=videoOutputSink
However, while the encoded data shows up on the client (it didn't at first, I had to add all of those properties until it finally showed up correctly), I have not been able to get the decoding end to work.
It shows a mostly gray screen with blips of pink, yellow, and green. Sometimes I get a little more of the right colors and most of the time just gray.
If I use this same exact pipeline to interact with VLC it works fine. My guess is that I am goofing up the caps somewhere. Can anyone offer any ideas on what I should be looking for?
I am having the same troubles with each of my other encoders as well, i.e. theora, h263 etc... though each in a different way.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
事实证明，VOIP 堆栈（本来是一个优秀的堆栈）要么有错误，要么我不理解它打包和传输 RTP 数据包的方式。我绕过它并通过 gstreamer udpsink 和 udpsrc 发送数据，它工作正常。现在我唯一剩下的问题将直接向堆栈的开发团队提出。感谢您的帮助。
It turns out the VOIP stack, which is otherwise an excellent stack, either has a bug or I do not understand the way it packages and transmits the RTP packets. I bypassed it and sent the data via gstreamer udpsink and udpsrc and it works fine. Now my only remaining questions will be directed to the dev team of the stack. Thanks for your help.