DTW | Dynamic Time Warping Algorithm can be used to measure | Time Series Database library

I have found the solution. it turns out that at this loop declaration:

A lot of your code can be reduced if you make use if min and max macros:

I think you can enforce this by defining your own window function. For example, take these series:

A C implementation of dynamic time warping is in https://github.com/wannesm/dtaidistance/tree/master/dtaidistance/lib/DTAIDistanceC/DTAIDistanceC

You can always translate python to C using Cython https://people.duke.edu/~ccc14/sta-663/FromPythonToC.html however the generated code sometimes does not work, complete rewriting is better

You need to be careful here. Let's say you have n time series in your 'training' set (let's call it this, even though you are not really training with kNN) of length l. Computing the DTW between a pair of time series has a asymptotic complexity of O(l * m) where m is your maximum warping window. As m <= l also O(l^2) holds. (although there might be more efficient implementations, i don't think they are actually faster in practice in most cases, see here). Classifying a time series using kNN requires you to compute the distance between that time series and all time series in the training set which would mean n comparisons, linear with respect to n.

So your final complexity would be in O(l * m * n) or O(l^2 * n). In words: the complexity is quadratic with respect to time series length and linear with respect to the number of training examples.

Ad 1) Labelling

I am not sure what you mean by "labelling" the dataset. Nowadays, all you need for ASR is an utterance and the corresponding text (search e.g. for CommonVoice to get some data). This depends on the model you're using, but neural networks do not require any segmentation or additional labeling etc for this task.

Ad 2) KFold cross-validation

Doing cross-validation never hurts. If you have the time and resources to test your model, go ahead and use cross-validation. I, in my case, just make the test set large enough to make sure I get a representative word-error-rate (WER). But that's mostly because training a model k-times is quite an effort as ASR-models usually take some time to train. There are datasets such as Librispeech (and others) which already have a train/test/dev split for you available. If you want, you can compare your results with academic results. It can be hard though if they used a lot of computational power (and data) which you cannot match so bear that in mind when comparing results.

Ad 3) MFCC Features

MFCC work fine but from my experience and what I found out by reading through literature etc, using the log-Mel-spectrogram is slightly more performant using neural networks. It's not a lot of work to test them both so you might want to try log-Mel as well.

Ad 4) and 5) DTW for same length

If you use a neural network, e.g. a CTC model or a Transducer, or even a Transformer, you don't need to do that. The audio inputs do not require to have the same lengths. Just one thing to keep in mind: If you train your model, make sure your batches do not contain too much padding. You want to use some bucketing like bucket_by_sequence_length().

Just define a batch-size as "number of spectrogram frames" and then use bucketing in order to really make use of the memory you got available. This can really make a huge difference for the quality of model. I learned that the hard way.

You did not specify your use-case so I'll just mention the following: You need to know what you want to do with your model. If the model is supposed to be able to consume an audio-stream s.t. a user can talk arbitrarily long, you need to know and work towards that from the beginning.

Another approach would be: "I only need to transcribe short audio segments." e.g. 10 to 60 seconds or so. In that case you can simply train any Transformer and you'll get pretty good results thanks to its attention mechanism. I recommend to go that road if that's all you need because this is considerably easier. But keep away from this if you need to be able to stream audio content for a much longer time.

Things get a lot more complicated when it comes to streaming. Any purely encoder-decoder attention based model is going to require a lot of effort in order to make this work. You can use RNNs (e.g. RNN-T) but these models can become incredibly huge and slow and will require additional efforts to make them reliable (e.g. language model, beam-search) because they lack the encoder-decoder attention. There are other flavors that combine Transformers with Transducers but if you want to write all this on your own, alone, you're taking on quite a task.

There's already a lot of code out there where you can learn from:

• TensorFlowASR (Tensorflow)
• ESPnet (PyTorch)

hth

You essentially need to construct a matrix, evaluating the FastDTW algorithm on all possible combinations of the series.

How about this:

Since there are so many bars in one graph, I would use sns.catplot to draw the the different categories into a Facet Grid and then it would be much better for adding labels, which you can do with the custom function add_labels (please note the different parameters -- feel free to remove some/add others. I have adapted from this solution).

You could also make the x-axis more variable if you pass sharex=False when creating the catplots (see end of this solution)

Also, sns.catplot doesn't work well with adding to subplots, so that you can save as one figure. This is why I use plt.close(fig) to get rid of the blank figure we created, and this would also mean adding any formatting (such as adding a title) to that figure would be pointless, since we are getting rid of the figure at the end; however, there are hacks. One is to save as separate figures and use a solution from here: to combine into one .pdf. I think it would be better to have the extra space of one graph per page or image. Another option is to use somewhat of a hack to get into one figure: