stft | Short time Fourier transform | Video Utils library

Check these pages

http://hyperphysics.phy-astr.gsu.edu/hbase/Music/vowel.html http://my.ilstu.edu/~jsawyer/consonantsvowels/consonantsvowels_print.html

In a very plain answer I would say that vowels have peaks in the spectrum, while the consonants are filtered continuous spectrum noise. This will be clearer for the unvoiced fricative consonants, like /f/, /s/. Not so much for the plosives (due to the relatively short duration).

I have found that the issue happened in the padding step, I mean

There's no need for other variables but Z which is a 2D NumPy matrix and contains all the required information to construct the plot. However, it is needed to take its absolute, as it contains imaginary values.

This does the trick nicely enough. As I suspected, you need to tweak hop_length.

You are already creating a subplot with

fig, ax = plt.subplots(figsize=(10, 5))

So instead of

The time axis for an STFT spectrogram depends on two factors: the sample rate and the hop length.

When you compute the STFT, you specify hop_length=64, win_length=256. Note that this information is not contained in D or S_db—librosa leans more towards a functional approach, not an object-oriented approach.

So when you then go on to show the spectrogram using librosa.display.specshow, you have to specify the hop_length, which you missed. Therefore the default hop_length=512 is used, which leads to a factor 512 / 64 = 8 error. I.e. 0.792 * 8 = 6.336, which matches what you see in your spectrograms.

Also, I believe x_axis='s' should rather be x_axis='time'.

So changing

Use model.predict() on your new audio file. That should return your desired output.

MFCCs are not spectrograms (time-frequency), but "cepstrograms" (time-cepstrum). Comparing MFCC with spectrogram visually is not easy, and I am not sure it is very useful either. If you wish to do so, then invert the MFCC to get back a (mel) spectrogram, by doing an inverse DCT. You can probably use mfcc_to_mel for that. This will allow to estimate how much data has been lost in the MFCC forward transformation. But it may not say much about how much relevant information for your task has been lost, or how much reduction there has been in irrelevant noise. This needs to be evaluated for your task and dataset. The best way is to try different settings, and evaluate performance using the evaluation metrics that you care about.

Note that MFCCs may not be such a great representation for the typical 2D CNNs that are applied to spectrograms. That is because the locality has been reduced: In the MFCC domain, frequencies that are close to eachother are no longer next to eachother in vertical axis. And because 2D CNNs have kernels with limited locality (typ 3x3 or 5x5 early on), this can reduce performance of the model.

The 'FrequencyRange' parameter to stft is new in MATLAB R2020b. (You can see this by checking the reference page from R2020a https://www.mathworks.com/help/releases/R2020a/signal/ref/stft.html ). So, you need to update to R2020b, or not use the 'FrequencyRange' parameter.

In general, I'd recommend trying to use the documentation pages corresponding to the version of MATLAB you're using.