resampler | A Simple and Efficient Audio Resampler Implementation in C | Speech library

I would like to calculate an aggregated performance measure (precision) for all iterations of a leave-one-out resampling.

For a single iteration, the result for thie measure can only be 0, 1 (if positive class is predicted) or NaN (if negative class is predicted.

I want to aggregate this over the existing values of the whole resampling, but the aggregation result is always NaN (naturally, it will be NaN for many iterations). I could not figure out (from the help page for ResampleResult$aggregate()) how to do this:

I have C++ files in my iOS Xcode project. Those files uses the next libraries that I'm calling via HomeBrew:

• mpg123/1.26.5
• libgcrypt
• ffmpeg
• libgpg-error
• fftw
• libsndfile

The way I'm including them in the project is by setting the HeaderSearch Paths:

And the Library Search Paths:

That is all I'm doing to call those libraries. The error that I'm getting when I compile the project is the next one:

Here's my small CSV file, which is totally encoded in UTF-8, and the date is totally correct.

I repaired the erros from:

• Is there a function to get the difference between two values on a pandas dataframe timeseries?
• I can't convert the Pandas Dataframe type to datetime
• Pandas Python: KeyError Date

I'm having an issue with a BufferedWaveProvider from NAudio library. I'm recording 2 audio devices (a microphone and a speaker), merge them into 1 stream and send it to an encoder (for a video).

To do this, I do the following:

1. Create a thread where I'll record the microphone using WasapiCapture.
2. Create a thread where I'll record the speakers audio using WasapiLookbackCapture. (I also use a SilenceProvider so I don't have gaps in what I record).
3. I'll want to mix these 2 audio so I have to make sure they have the same format, so I detect what's the best WaveFormat in all these audio devices. In my scenario, it's the speaker. So I decide that the Microphone audio will pass through a MediaFoundationResampler to adapt its format so it has the same than the one from the speaker.
4. Each audio chunks from the Wasapi(Lookback)Capture are sent to a BufferedWaveProvider.
5. Then, I also made a MixingSampleProvider where I pass the ISampleProvider from each recording thread. So I'm passing the MediaFoundationResampler for the Microphone, and BufferedWaveProvider for the Speakers.
6. In loop in a third thread, I read the data from the MixingSampleProvider, which is supposed to asynchronously empty the BufferedWaveProvider(s) while it's getting filled.
7. Because each buffer may not get filled exactly at the same time, I'm looking at what's the minimal common duration there is between these 2 buffers, and I'm reading this amount out of the mixing sample provider.
8. Then I enqueue what I read so my encoder, in a 4th thread, will treat it in parallel too.

Please see the flowchat below that illustrates my description above.

• It works GREAT when recording the microphone and speaker for more than 1h while playing video game that uses the microphone too (for online multiplayer). No crash. The buffers are staying quite empty all the time. It's awesome.
• But for some reason, every time I try my app during a Discord, Skype or Teams audio conversation, I immediately (within 5sec) crash on BufferedWaveProvider.AppSamples because the buffer gets full.

Looking at it in debug mode, I can see that:

• The buffer corresponding to the speaker is almost empty. It has in average 100ms max of audio.
• The buffer corresponding to the microphone (the one I resample) is full (5 seconds).

From what I read on NAudio's author's blog, the documentation and on StackOverflow, I think I'm doing the best practice (but I can be wrong), which is writing in the buffer from a thread, and reading it in parallel from another one. There is of course a risk that it's getting filled faster than I read it, and it's basically what's happening right now. But I'm not understanding why.

I'd like some help to understand what I'm missing here, please. The following points are confusing me:

1. Why does this issue happens only with Discord/Skype/Teams meetings? The video games I'm using are using the microphone too, so I can't imagine it's something like another app is preventing the microphone/speakers to works correctly.

2. I synchronize the startup of both audio recorder. Do to this, I'm using a signal to ask the recorders to starts, and when they all started to generate data (through DataAvailable event), I send a signal to tell them to fill in the buffers with what they will receive in the next event. It's probably not perfect because both audio devices send their DataAvailable at different times, but we're talking about 60ms of difference maximum (on my machine), not 5 seconds. So I don't understand why it's getting filled.

3. To bounce on what I said in #2, my telemetry shows that the buffer is getting filled this way (values are dummy):

I have a dataframe df that contains data in periods of 3 hours:

I am trying to convert a 44.1k 16bit flac file into 48k 32 bit (float) wav file.

This is the command I use:

'ffmpeg -i in.flac -af aresample=resampler=soxr:precision=28:out_sample_fmt=fltp:out_sample_rate=48000 out.wav'

No matter which value I use for out_sample_fmt like s32, flt, fltp the output out.wav is only 16 bit.

What am I doing wrong here? How to get the highest quality (as in resampling) 32 bit floating point wav file with ffmpeg using soxr?

Using Win32 I have access to CLSID_CResamplerMediaObject which means I can reduce my channel count from say 6 to 2.

On UWP this is no longer defined and the only reference to a resampler I can find is CLSID_AudioResamplerMediaObject. When I create an instance of this class however, and pass it my MFMediaType_Float or MFMediaType_PCM type, it says that the provided types aren't supported...

I want to downsample some time series data, giving me the last value of each quarter:

I've noticed that as apparently documented IMFTransform::ProcessOutput() for a resampler can only output one sample per call! I guess its more orientated at large frame size video coding. Given all the code I have been looking at as reference for related audio playback allocates one IMFMediaBuffer per call of ProcessOutput, this seems a little insane and terrible architecture - unless I am missing something?

It is especially bad from the point of view of media buffer usage. For example a SourceReader decoding my test MP3 gives me chunks of about 64KB in one sample with one buffer. Which is sensible. But GetOutputStreamInfo() is requesting a media buffer of just 24 bytes per call for ProcessOutput().

64KB chunks => chopped into many 24B chunks => to further processing, seems very daft overhead (the resampler would be doing a lot of overhead per every 24 bytes, and enforcing that overhead later down the pipeline if its not consolidated).

From https://docs.microsoft.com/en-us/windows/win32/api/mftransform/nf-mftransform-imftransform-processoutput

Its says:

1. The MFT cannot return more than one sample per stream in a single call to ProcessOutput
2. The MFT writes the output data to the start of the buffer, overwriting any data that already exists in the buffer

So it's not even the case it can append to the end of partially full buffer attached to the sample.

I could create my own pooling object that supports the media buffers interface but pointer bumps into a vanilla locked media buffer I guess. The only other option seemingly being to lock/copy those 24 bytes to another larger buffer for processing. But this all seems excessive, and at the wrong granularity.

What is the best way to deal with this?

Here is a simplified sketch of my test so far: