gSTREAM | GPU eSTREAM framework | GPU library

Sorry I can't make a comment yet, but I did kind of a similar integration and I managed to use omxplayer by following this guys blog. Hope that helps

https://www.codedesigner.de/articles/omxplayer-kivy-overlay/

The problem is about using GStreamer library as a plugin of OpenCV. OpenCV doesn't throw exception even you build source code without GStreamer support. (In default, GStreamer library was directly found by Ubuntu, conversely Raspberry Pi 4 couldn't find it.)

Firstly I check build information of OpenCV with std::cout< in Ubuntu 18.04 machine and found that:

GStreamer: YES (1.14.5)

Also I just check this on Raspberry Pi 4 side and build information was:

GStreamer:NO

Before the build OpenCV I just compare GStreamer plugins with gst-inspect-1.0 command for both of them and I just install some missing plugins like gstreamer1.0-tools . Also I wasn't know the problem, before the checking build information, so I installed some other GStreamer plugins that currently I don't remember.

Lastly, I build system by adding -D WITH_GSTREAMER=ON flag. And now it works well.

I'll edit answer if the problem related to missing plugins those are installed later. For this, I'll check this issue with clean Buster OS image.

It is not necessary anymore to convert cv::Mat* to cv::Mat. Furthermore, it's not logical.

Here is current pipeline that working without error in Gazebo Simulation:

Ok after several tries I found, I guess, the issue: QT was built statically in my case so it needs to import explicitly the plugins used. After checking qt multimedia installation prerequisites

https://doc.qt.io/qt-5/linux-requirements.html#multimedia-dependencies

And all the gstreaming libraries needed

I added explicitly on my .pro file the multimedia plugins (I was thinking QTCreator already adds all the needed references but obviously I was wrong...)

So, after an extensive dialog with Azure Media Services team from Microsoft, it turns out that the Azure Media Player, requires a sound track to be present for it to be able to play back.

Changing the GST_PIPE to:

I managed to solve this by installing almost everything I could find which was remotely connected with gstream and qtmultimedia. I'm not sure which are necessary but the current list is below.

For testing it's good to clean cached python executable so that it doesn't use older version. This is not only in the same dir but also ~/.local/lib/python3.6/site-packages/.

The cherry on this package cake was libqt5multimedia5-plugins.

Installed list for gstream

You can register a producer on the Request object. It will have its pauseProducing method called when the write buffer of that Request is full. When room becomes available, it will have its resumeProducing method call.

You can use this information to drop frames that may not be delivered in a timely fashion. However, you will have to actually identify frames in your server (currently you only have a dataReceived method that passes data through as a stream with no idea where frames begin or end). This also has the problem that buffer fullness is possibly a very lagging indicator of delays in the stream. And if the bottleneck in the system isn't in between reading data from gstreamer and writing it to the requests, adding backpressure sensitivity to this part of the program isn't going to help.

There is no specific error state in GStreamer (GStreamer states are: NULL, READY, PAUSED, PLAYING (*)). Thus, you need to watch and check if any error occurs. You should check, at least:

• the return value of the function for changing the state (gst_element_set_state)
• if any error message is sent to the bus:

  Errors can be received by listening to the GstBus of the element/pipeline for GstMessage objects with the type GST_MESSAGE_ERROR or GST_MESSAGE_WARNING. The thrown errors should be inspected, and filtered if appropriate.

(*) Note: actually, the complete list of states in the API includes also a pending state that may be relevant in some cases (see e.g. How to resume playing after paused using gstreamer?)