STFT | STFT , ISTFT , mel-filterbank modules | Video Utils library

For the sound processing and feature extraction part, librosa is definitely going to provide you all you need.

For the machine learning part however, speaker identification (also called "voice recognition") is a relatively complex task. You probably will get more success using techniques from deep learning. You can certainly try to use random forests if you like, but you'll probably get a lower accuracy and will have to spend more time doing feature engineering. In fact, it will be a good exercise for you to compare the results you can get with the various techniques.

For an example tutorial on speaker identification using Keras, see e.g. this article.

Answer to your question is in the error message.

With the 2D array voice (most likely Nx1, for mono recording), scipy.fft.fft ends up computing a batch of N 1D FFTs of length 1. Since the FFT of a sequence of 1 value is an identity, what you see in your 2nd plot is the absolute value of the first half of your time domain signal.

Try computing the FFT on a 1D array (a single channel), with e.g. :

TensorFlow can't compute the output shape of your layer. As Conv2D requires a specific shape (4 dimensions), it will fail if the output shape of the previous layer is not known (None).

To fix that, you need to specify which axis you want to squeeze in you call function.

Here, I specify that this is the last axis that need to be squeezed (the channel axis).

Your call to os.path.basename("data/what.wav") returns 'what.wav'

You then split that using "-" as the splitter, which returns ['what.wav'], a list of one element.

But you then try to reference the third element of the list with [2], which throws an exception.

If someone encounter this problem in the future - rendering with pcolorfast is significantly faster than with pcolormesh.

TL;DR Amplitude Spectrum is basically FFT of the signal, and Power Spectrum is a squared value of the Amplitude Spectrum, which is also referred as energy sometimes. Here is one of examples from Meyda website that is calculating Amplitude Spectrum https://github.com/catalli/audiotrainer-server/blob/df41322906c88cd6f899e8f9b9661ebb949f72e1/index.js#L17

Long answer:

Now, lets look into your code sample line by line and figure out what is it doing and how to implement it in javascript.

1. S = numpy.abs(librosa.stft(y=audio_sample, n_fft=1024, hop_length=500)) ** 2

this is calculating square values of 1024 bins fft of audio_sample y, which is basically a Power Spectrum or an Amplitude Spectrum squared. Please note that the abs of complex number is a vector lenth: sqrt(real_part^2 + img_part^2)

1. mel_S = numpy.dot(librosa.filters.mel(sr=44100, n_fft=1024, n_mels=64), S).T

this is an mfcc calculation, which is basically a product of predefined filter banks and fft squared.

1. log_mel_S = librosa.power_to_db(mel_S, ref=1.0, amin=1e-10, top_db=None)

this last one will convert the result to decibel (dB) units (10 * log10(S / ref))

i will extend this answer with js code-sample later, submitting it now because i think it will be helpful already as it is

If you don't mind using pathlib as @Andrew suggests, I think what you're trying to do could be accomplished by using the current working directory and the stem of each .wav file to construct the filename for your .png.

You can do this with a custom collate function, passed to the DataLoader: