byte-stream | A non-blocking stream abstraction for PHP based on Amp | Reactive Programming library

I was able to extract the codec_data by passing the x264enc output into h264parse.

I think I finally figured this out.

The first thing is that I need to include removing the AVCVIDEOPACKET headers (packet type and composition time fields). These are not part of the h264 format and thus cause parsing errors.

The second thing I needed to do was to not pass the sequence header as a buffer to the source. Instead the sequence header bytes need to be set as the codec_data field for the appsrc's caps. This now allows for no parsing errors when passing the video data to h264parse, and even gives me a correctly sized window.

The third thing I was missing is the correct dts and pts values. It turns out the RTMP timestamp I'm given is the dts, and pts = AVCVIDEOPACKET.CompositionTime + dts.

Your problem is that decodebin selects nvv4l2decoder that outputs into NVMM memory. videoconvert cannot read from NVMM memory. You would use nvvidconv instead that can read from NVMM and output into system memory.

However, it is not mandatory to decode h264 for reencoding into h264. This simple pipeline should do the job:

You may try adding rtpjitterbuffer in receiver:

First I see a syntax error:

• missing argument in method-call: ByteArrayInputStream stream = reportPDF.generateReportDocument(dtos, ); (after the comma)

With this syntax error you most likely receive a compilation-error on console.

Assume this is a lapse and you can fix it to something like ByteArrayInputStream stream = reportPDF.generateReportDocument(dtos); then it should compile.

Assume further your server application boots and runs without errors, then you could test the HTTP-endpoint with a HTTP-call.

You can test using a HTTP-client like CURL, postman or maybe even a browser.

Then you should receive a response with HTTP status code 200 and the body containing the PDF-file as binary with MIME-type application/pdf and specified header Content-Dispositon.

The browser is expected to prompt you with a download-dialogue.

Your InputStreamResource is a valid way, but you should be confident when using it.

In a Spring controller method, you can return the binary data in different types:

• ResponseEntity as byte-array
• ResponseEntity as stream (not input-stream for reading input)
• ResponseEntity as abstract binary content, see Spring's Resource
• ResponseEntity as entire file

See also

• Spring boot Angular2 file download not working
• PDF Blob is not showing content, Angular 2
• Return generated pdf using spring MVC

There are also some response-directed ways especially in Spring:

• return a InputStreamResource as you did
• return a StreamingResponseBody is very convenient
• write to a HttpServletResponse, probably the oldest way

See: How To Download A File Directly From URL In Spring Boot

Remember: Input is for reading (e.g. from a request), output is for writing (e.g. to a response). So you need an output type, like byte[] or ByteArrayOutputStream etc for your response-body.

When reading input into ByteArrayInputStream stream you could copy from that input to an output-stream with e.g. Apache-Commons IOUtils: IOUtils.copy(in, out);. Or simply return the byte-array: byte[] data = stream.readAllBytes();

See: Java InputStream to Byte Array and ByteBuffer | Baeldung

You can install a GstPadProbe at the the funnel input pad and check for the EOS event on the callback.

After much head banging, I finally figured out the root cause of this. And it's a bit obscure..

The wireless video transmitter in the drone is able to dynamically change the video bitrate depending on the radio link's available bandwidth. Or put in another way: When the drone is too far away or there is strong interference, the video quality degrades.

When this happens, video frames (contained in only one slice within a single NAL) start to become significantly smaller. Since I'm reading 512-byte chunks of an h264 stream with no particular alignment and forwarding them to GStreamer as GstBuffers, if the size of data needed for one frame is lower than 512 bytes, there is a possibility that a buffer contains multiple frames. In this case, h264parse sees this as N different buffers with identical timestamps. The default behavior then seems to be to ignore both upstream PTS and DTS and try to compute the timestamp based on the duration of the frame by reading the VUI from the SPS, which is not present in my stream. Therefore, the buffer leaving the source pad of h264parse will have no PTS and no DTS, thus making mp4mux complain.

As I've previously mentioned, my stream is quite simple so I wrote a simple parser to detect the beginning of each NAL. This way I can 'unpack' the stream coming from the USB hardware and make sure that every buffer pushed into my pipeline will contain only one NAL (therefore, as much as one frame), independently timestamped.

And for redundancy, I added a probe attached to the sink pad of my tee element to make sure I have correct timestamps in every buffer through it. Otherwise, they're forced to the running time of the element like this.

I found the solution. A videoconvert element is needed in the playing pipeline.
The working playing pipeline is:

udpsrc port=2705 ! application/x-rtp,width=400,height=400,encoding-name=H264,payload=96,framerate=7/1 ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! autovideosink

1. There is a way using compact regions, but there is a big restriction:

Our binary representation contains direct pointers to the info tables of objects in the region. This means that the info tables of the receiving process must be laid out in exactly the same way as from the original process; in practice, this means using static linking, using the exact same binary and turning off ASLR. This API does NOT do any safety checking and will probably segfault if you get it wrong. DO NOT run this on untrusted input.

This also gives insight into universal serialization is not possible currently. Data structures contain very specific pointers which can differ if you're using different binaries. Reading in the raw bytes into another binary will result in invalid pointers.

There is some discussion in this GitHub issue about weakening this requirement.

1. I think the proper way is to open an issue or pull request upstream to export the data constructors in the internal module. That is what happened with HashMap which is now fully accessible in its internal module.

Update: it seems there is already a similar open issue about this.