Demux | A gateway to facilitate decentralised streaming ecosystem | REST library

Your issue stems from an improper understanding of delayed linking.

Pads in gstreamer can be linked if and only if they agree on their format. For some elements, all possible pad formats are known ahead-of-time. This means they can be linked ahead of time, with functions like link_may and link. For example, from gst-inspect-1.0 oggdemux In the pad section, we see this:

The answer is #EXTVLCOPT:image-duration=100 after each image file. (change 100 for the number of seconds.

Found in https://forum.videolan.org/viewtopic.php?t=148945

You may just use video-sink property of playbin:

Sounds like a stereo audio interlacing problem, where every other sample is being skipped. Your provided output sample is a stereo MP3, yet both channels are identical.

Try using channels=1 or playing with or removing demux processing.

Okay so I've figured out where the problem was.

I had built ffmpeg from source (master), and I must've messed up the config because something was missing. The linker very helpfully found the missing library in my regular ffmpeg from my package manager (stable 4.4.1). So my program was linking against half master and half 4.4.1, resulting in undefined behaviour.

Turns out the random values I was getting came from that undefined behaviour. Linking with the stable 4.4.1 build directly fixed the issue and now I get_fps returns the expected values.

I'm surprised I got this far without noticing. Not even a segfault.

Because you are doing a new Media for each frame, you won't be able to stream it as a single stream.

What you could do is create a MJPEG stream : Put .jpg images one after one in a single stream, and use that stream with LibVLCSharp to stream it.

However, if LibVLCSharp is faster to read the data from your memory stream than you are writing data to it, it will detect the end of the file, and will stop the playback / streaming (A Read() call that returns no data is considered as the end of the file). To avoid that, the key is to "block" the Read() call until there is actually data to read. This is not a problem to block the call as this happen on the VLC thread.

The default MemoryStream/StreamMediaInput won't let you block the Read() call, and you would need to write your own Stream implementation or write your own MediaInput implementation.

Here are a few ideas to block the Read call:

• Use a BlockingCollection to push Mat instances to the stream input. BlockingCollection has a Take() method that blocks until there is actually some data to read
• Use a ManualResetEvent to signal when data is available to be read (it has a Wait() method)

It would be easier to talk about that on The LibVLC discord, feel free to join !

If you manage to do that, please share your code as a new libvlcsharp-sample project!

Don't forget that Play() is not a synchronous method as you might expect. It is a method that posts a stop message to a background thread, and only then starts to play the media.

When you're executing your IsStartingOrPlaying() method right after, chances are that the state is not the one that you might have expected, thus calling the second Play()

So what happened is that the library in Debian providing support for live555 was removed in February of this year, this affects all downstream distros including but not limited to RPi OS and Ubuntu:

https://askubuntu.com/a/1363113

The 2 active versions were 2020.01.19-1 and 2018.11.26-1.1, Live555 has since added GPL license headers to the offending files, however the RFC issue remains.

Now you may be tempted to just download the latest Live555 source code and compile it... it does not work. There have been changes to function names and structures referenced by VLC, and as such VLC will not compile against the source. You need to get an older version, I specifically used this one, which is a tweaked snapshot from 2020 prior to the modifications that prevent VLC compilation:

https://github.com/rgaufman/live555

The configuration you want is ./genMakefiles linux-with-shared-libraries, I do not know if it is required but since my system is x86-64-bit I added -m64 to the compiler options first

After compilation and install, I went on to compile VLC, adding '--enable-live555' and '--with-live555-tree=extras/live555-master' after placing the root Live555 folder in the VLC extras folder, however VLC failed to compile, it turns out the Live555's make install does not copy all the header files needed to where VLC is looking. They were dropped as 4 subfolders into /usr/local/include/, and the actual libs into /usr/local/lib/. Adding the correct CXX/CPP flags will make it look where they were put, however I put them all in a single folder and used 1 flag.

I also had to '--disable-mod' to work around a dependency version issue that I had no interest in fixing, since I do not use modplug or any mod files.

50 minutes later... VLC successfully compiled! However it was expecting the libraries for Live555 to be in /usr/lib/ not /usr/local/lib/, since it took so long to compile I was just fine with linking or copying the libraries into the expected folder, and after that VLC works with RTSP when linked to the new file. Or you can choose to maintain the original VLC and run the new file directly if you need to load the camera feeds.

This is very similar to How to compile ffmpeg to get only mp3 and mp4 support, but with a few different compilation options:

./configure --disable-everything --disable-network --disable-autodetect --enable-small --enable-demuxer=mpegts --enable-muxer=mp4 --enable-parser=aac,h264 --enable-decoder=aac,h264 --enable-protocol=file

The the link above for more details.