anomaly_detection | repository contains a new class | Predictive Analytics library

I am working on a network that uses a LSTM along with MDNs to predict some distributions. The loss function I use for these MDNs involve trying to fit my target data to the predicted distributions. I am trying to compute the log-sum-exp for the log_probs of these target data to compute the loss. When I use standard log-sum-exp, I get reasonable initial loss values (around 50-70) even though later it encounters some NaNs and breaks. Based on what I have read online, a numerically stable version of log-sum-exp is required to avoid this problem. However as soon I use the stable version, my loss values shoot up to the order of 15-20k. They do come down upon training but eventually they also lead to NaNs.

NOTE : I did not use the logsumexp function in PyTorch, since I needed to have a weighted summation based on my mixture components.

Is there a open source function to compute moving z-score like https://turi.com/products/create/docs/generated/graphlab.toolkits.anomaly_detection.moving_zscore.create.html. I have access to pandas rolling_std for computing std, but want to see if it can be extended to compute rolling z scores.

Imagine I have an array like the following:

[0.1,0.12,0.14,0.45,0.88,0.91,0.94,14.3,15,16]

I'd like to identify patterns in this, so I can compare it to another dataset to see if it matches. For instance, if I input 0.89, I'd like to be able to see this belongs to the 0.88-0.94 cluster. However, if I enter 0.5, I'd like to see that this does not belong in the dataset, even though it is close to 0.45 - an anomaly in the data.

(The above array contains sample numbers, but in the actual system I'm comparing properties of HTML code to categorise them. I'm using Tensorflow for text categorisation, but some things (such as CSS length, CSS:HTML ratio) are numbers. While there are patterns in this, it's not obvious or in one place - e.g category A might have a lot of very high values and low values, but almost none in between. I can't give you the real numbers because those are determined by the code inputted and the ML preprocesser, but we can assume the numbers are about 10% anomaly, and almost always try to show one or some combination of middle, lower or upper. When 'training', these numbers are taken from the data and stored in one of the arrays (representing the three categories). I then want to take my input and tell which of the arrays' patterns seems to line up with the input number.)

Now, imagine the array is hundreds or thousands of items long. At least 10% will be anomalies, and I need to account for that. I guess cluster detection isn't the correct term - it's mainly getting rid of anomalies - but the part I got stuck on particularly was having ranges of different sizes. For instance, in the example above I'd still like 14.3-16 to count as one range/cluster, even though there are much further apart than0.1-0.14.

I've done some digging through the Wikipedia article (https://en.m.wikipedia.org/wiki/Anomaly_detection) on the topic, and found that the most likely functional and simple approach would be K-nearest-neighbour-style density analysis. However, I've not been able to find any Python plug-in that can easily do this for me - the issue is, there are so many variations on this specific task that it's basically impossible to find exactly what I'm looking for. I've also tried making my own basic algorithm to compare each item to its neighbour and see which one it is closer to (to cluster), or if it the distance is greater than 2* the mean of the distances between other items in the clusters class it as an anomaly. However, this wasn't very accurate, and still had an element of human bias (why 2*, not 3*?); furthermore, it went completely haywire at the start and end or the array. Therefore, if any of you have a recommendation for a quick algorithm that would work even better, or an implementation of the aforementioned, that would be greatly appreciated.

Thanks in advance.