Why RTP/RTSP meddle with my H.264 NALs?

Question

I looked in The RFC and noting could explain why the following happens(Though the decoder can still produce the original movie).

I transmitted the H.264/AVC nals using VSS h.264 encoder, the byte stream looked something like this E5 46 0E 4F FF A0 23...

when I read the movie data one the receiver side after the RTP Broadcaster/RTSP receiver, I get extra unknown data but always in the same places, 8 bytes are added before Start Code prefix (0x00000001), and 2 bytes are added after Start Code prefix it looks something like this.

XX XX XX XX XX XX XX XX 00 00 00 01 XX XX, then I look in the Wireshark and i could see that the RTP adds the bytes to the data payload.

Why does it happens why? and why the decoder seems to cope well with those extra bytes?!

Answer 1

Thats some messed up stream... And you can mess it up even more, and it will still work, because decoder parses it for 0x000001 start code, skipping the bytes that are added at the beginning. Those two new bytes at the end must be H264 fragmentation bytes... or something H264 related since they work.

So basically, this is due to defective packetizer/RTSP source filter. My guess is that if you ASCII encode those 8 bytes you will get the vendor name of the RTSP source filter... xD

Answer 2

As I mentioned in another post Changing NALU h.264/avc, for RTP encupsulation, H.264 is transmitted over RTP as defined in RFC 3984. This in particular defines how exactly large NAL units are broken into smaller parts that fit smaller message sizes, suchas UDP datagram size. That is, fragmentation.

Receiver depacketizes the data and restores NALUs, and it uses this extra information to do the job.

So what you essentially need is to compare raw data you have against RFC 3984 format. Also, Wireshark already does partially this for you by dissecting traffic into readable items.