qgroundcontrol | platform ground control station for drones ( Android , iOS | Robotics library

After much head banging, I finally figured out the root cause of this. And it's a bit obscure..

The wireless video transmitter in the drone is able to dynamically change the video bitrate depending on the radio link's available bandwidth. Or put in another way: When the drone is too far away or there is strong interference, the video quality degrades.

When this happens, video frames (contained in only one slice within a single NAL) start to become significantly smaller. Since I'm reading 512-byte chunks of an h264 stream with no particular alignment and forwarding them to GStreamer as GstBuffers, if the size of data needed for one frame is lower than 512 bytes, there is a possibility that a buffer contains multiple frames. In this case, h264parse sees this as N different buffers with identical timestamps. The default behavior then seems to be to ignore both upstream PTS and DTS and try to compute the timestamp based on the duration of the frame by reading the VUI from the SPS, which is not present in my stream. Therefore, the buffer leaving the source pad of h264parse will have no PTS and no DTS, thus making mp4mux complain.

As I've previously mentioned, my stream is quite simple so I wrote a simple parser to detect the beginning of each NAL. This way I can 'unpack' the stream coming from the USB hardware and make sure that every buffer pushed into my pipeline will contain only one NAL (therefore, as much as one frame), independently timestamped.

And for redundancy, I added a probe attached to the sink pad of my tee element to make sure I have correct timestamps in every buffer through it. Otherwise, they're forced to the running time of the element like this.