spectrogram | Web Audio Spectrogram | Web Framework library

I am currently working on building a CNN for sound classification. The problem is relatively simple: I need my model to detect whether there is human speech on an audio record. I made a train / test set containing records of 3 seconds on which there is human speech (speech) or not (no_speech). From these 3 seconds fragments I get a mel-spectrogram of dimension 128 x 128 that is used to feed the model.

Since it is a simple binary problem I thought the a CNN would easily detect human speech but I may have been too cocky. However, it seems that after 1 or 2 epoch the model doesn’t learn anymore, i.e. the loss doesn’t decrease as if the weights do not update and the number of correct prediction stays roughly the same. I tried to play with the hyperparameters but the problem is still the same. I tried a learning rate of 0.1, 0.01 … until 1e-7. I also tried to use a more complex model but the same occur.

Then I thought it could be due to the script itself but I cannot find anything wrong: the loss is computed, the gradients are then computed with backward() and the weights should be updated. I would be glad you could have a quick look at the script and let me know what could go wrong! If you have other ideas of why this problem may occur I would also be glad to receive some advice on how to best train my CNN.

I based the script on the LunaTrainingApp from “Deep learning in PyTorch” by Stevens as I found the script to be elegant. Of course I modified it to match my problem, I added a way to compute the precision and recall and some other custom metrics such as the % of correct predictions.

Here is the script:

I have a 1024 samples and I chucked it into 32 chunks in order to perform FFT on it, below is the output from FFT:

I am building a CNN project for spectrogram images. The backend code is already finished, and I was told to make a GUI on HTML. I have this code for user to make a selection on epoch, learning rate, and architecture number.

I use my custom dataset class to convert audio files to mel-Spectrogram images. the shape will be padded to (128,1024). I have 10 classes. after a while of training in the first epoch, my network will be crashed inside the hidden layer in GRU shapes due to this error:

I have created a data pipeline using tf.data for speech recognition using the following code snippets:

I am trying to implement my own function that gives the same results as Matlab spectogram function. So far I have accomplished function like this:

I am trying to access the numpy array from a tensor object that is processed with https://www.tensorflow.org/api_docs/python/tf/data/Dataset#map.

I get the error: AttributeError: 'Tensor' object has no attribute 'numpy'

When I try to access the tensor as: np_array = tensor.numpy()

While if I use: dataset.take(n), i am able to access the numpy array.

For more clarity on the situation I am facing, here is a short reproducible example of the error in a google colab:

https://colab.research.google.com/drive/13ectGEMDSygcyuW4ip9zrWaHO3pSxc3p?usp=sharing

Tensorflow version: 2.4.1

Update: Adding code in addition to the colab above:

Currently I'm trying to make some spectogram generation for my uni project. I'm trying to build a static library where all the magic will work and just call it from the main() function.

This is my cmake file:

Is it possible to plot the spectrogram of overnight sleep EEG data in mne? I don't want to create epochs but, have the spectrogram of continuous 8-9 hours. The examples I see in e.g. EEGlab (Matlab) have perfect color distinction which makes the outcome very readable. I would be grateful if you help me produce something similar but in mne.