mediamux | writing responsive React components in a concise | Frontend Utils library

The following code is an attempt to use AAC on Android to encode a floating point sine tone signal into an mp4 file. However, it fails. Either I get a distorted tone or nothing at all. The length of the mp4 file is also wrong.

I would appreciate some help solving this problem. Thanks!

I’m muxing an audio and video file. I have the urls where both files are hosted. I use android.media.MediaMuxer and android.media.MediaExtractor to mux them. During this muxing, those classes download the files, but I want to show progress, so I need to know the total file size of the audio file and the video files.

Does anyone know how to get the file size using those Android API’s? Or do I have to just hit the URL’s myself (again) to get the content length?

Example files

Video url: https://v.redd.it/832kzf6e23h71/DASH_1080.mp4

Audio url: https://v.redd.it/832kzf6e23h71/DASH_audio.mp4

My Android app creates an MP4 using the MediaCodec and the MediaMuxer. I use the MediaPlayer to play back the video. While the video plays, it isn't possible to seek to any location in the video using Android's MediaPlayer. More specifically, the seekTo function will not work. Using other apps to play the video and seek is somewhat sketchy. Some apps seem to work while others do not.

I have swapped my mp4 with a video that I recorded on the camera as well as various videos I've found on the Internet and none of them have the problem seeking. The fact that the stock camera app can generate an MP4 and let you seek clearly indicates a problem in the way the codec is being setup. This leads me to believe that the problem is most likely in the format settings used to create the video. I have tried modifying a number of settings without any success including the profile (used both baseline and main), the profile level, the I-frame interval (GOP) as well as the bitrate and video size. I also made sure the presentation time for each frame matched the frame rate exactly. Here is the info I am getting for both the video that doesn't support seek and one that does (the camera video). Is there anything in these settings that could be causing the problem?

A short test file can be downloaded here. The seek works if you play this in QuickTime or VLC:

https://drive.google.com/file/d/15QiDPYdPd_tVQTkqXuP0v2L7eKoMbWQo/view?usp=sharing

Video that doesn't support seek:

I used MediaMuxer and MediaCodec to generate a mp4 video.

The video is playble after I call mMediaMuxer.stop()

However, when the user quit the app before I get the change to call the stop() method, I am left with a big mp4 file that is not playable.

Is there anyway to repair this mp4 file to make it playable?

Here is one example of a corrupted mp4 file

And I was able to repair the file using this online tool but this tool asked to upload a non-corrupted video as reference.

Here is the non-corrupted mp4 video that I used as reference. When I uploaded this video, the tool repaired my broken mp4 file.

So it is possible to repair the file but how did they do it?

If useful, here is the code I used to generate both corrupted and non corrupted

I have a long running video task in a Xamarin.Android app; it decodes a video file using a MediaPlayer into a custom OpenGL ES Surface then queues another Surface, encodes the data using a MediaCodec and drains into a ByteBuffer, which then is passed into a MediaMuxer based upon encoder output ByteBuffer availability. The operation works well and fast, until the video file written byte total is over ~1.3GB, at which point the video (but not audio) will lockup.

The app seems to have too many GREFs, as I'm watching them go up and down in realtime, until they're finally well above 46000 GREFs. It seems like the operating system (or app?) is having trouble dumping all of the GREFs via GC, which causes the app to get stuck in the middle of video processing. I'm monitoring the android resources and the total available memory never changes very much at the OS level; the cpu also seems to always have plenty of idle headroom (~28%).

I am outputting to the system console and watching the gref output logging using:

adb shell setprop debug.mono.log gref

The garbage collection seems to not be able to keep up after about 14 minutes. The GREF count goes up, then down, up, then down; eventually, it goes so high that the GREF count stays above 46k, with the following message looping:

I'm trying to make some deep learning experiments on android on video samples. And I've got stuck into remuxing videos. I have a couple of questions to arrange information in my head:) I have read some pages: https://vec.io/posts/android-hardware-decoding-with-mediacodec and https://bigflake.com/mediacodec/#ExtractMpegFramesTest but still I have a mess.

My questions:

1. Can I read video with MediaExtractor and then pass data to MediaMuxer to save video in another file? Without using MediaCodec?
2. If I want to modify frames before saving, can I do that without using Surface? Just by modifying ByteBuffer? I assume that I need to decode data from MediaExtractor, then modify content, then encode it to MediaMuxer.
3. Does sample is the same as frame in context of method MediaExtractor::readSampleData ?
4. Do I need to decode sample?

I am trying to merge audio and video, but I am getting java.lang.IllegalStateException: Failed to add the track to the muxer. I think the problem is I can not add .weba audio to the muxer. If that the case how can I go about merging them?