oddstream | Outlier Detection in Data STREAMs | Machine Learning library
kandi X-RAY | oddstream Summary
kandi X-RAY | oddstream Summary
The goal of oddstream (Outlier Detection in Data Streams) is to propose a framework that provides real time support for early detection of anomalous series within a large collection of streaming time series data. By definition, anomalies are rare in comparison to a system's typical behaviour. We define an anomaly as an observation that is very unlikely given the forecast distribution. The algorithm first forecasts a boundary for the system's typical behaviour using a representative sample of the typical behaviour of the system. An approach based on extreme value theory is used for this boundary prediction process. Then a sliding window is used to test for anomalous series within the newly arrived collection of series. Feature based representation of time series is used as the input to the model. To cope with concept drift, the forecast boundary for the system's typical behaviour is updated periodically. More details regarding the algorithm can be found in our paper Anomaly Detection in Streaming Nonstationary Temporal Data.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of oddstream
oddstream Key Features
oddstream Examples and Code Snippets
Community Discussions
Trending Discussions on oddstream
QUESTION
I configured event-time processing and have a connected stream with a CoFlatMapFunction
. I am writing a test case but I am seeing that the FlatMapFunction
does not invoke its methods flatMap1()
and flatMap2()
with events in event-time order.
Some pseudocode to clarify
...ANSWER
Answered 2018-May-25 at 21:18Flink's co-functions (functions on connected streams) do not give guarantees in which order their methods are called. The methods (e.g., flatMap1()
and flatMap2()
) are called whenever an event is available from either input. In your example, the amount of data that the even source generates is too small such that all data is already processed when the odd numbers arrive.
So, how does event-time processing work for co-functions?
The watermarks of the co-function is always the minimum watermark of both inputs. For a CoFlatMapFunction
this is not really important, because you can neither read the current watermark nor the timestamps of the records. However, with a CoProcessFunction
you have access to both and can register timers that are called when the watermark reaches a certain point in time. If you want to sort the out-going stream on event time, you need to buffer incoming events (in state) and when the watermark progresses, you can emit all records in order up to the time over the watermark.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install oddstream
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page