tods | TODS : An Automated Time-series Outlier Detection System | Machine Learning library

I'm trying to manually create a dataset with a type Set column:

I have a kotlin data class as shown below

I have a Dataset[General] which I would like to split into two Datasets. One Dataset[Mary] and one Dataset[John]. I would like to have as many rows based the amount of structures that exist in the add_marry or add_john arrays. I want to use Datasets and therefore not use the Dataframe API which would allow me to explode. Any advice on this would be greatly appreciated! The input is

I'm currently storing lists in .csv files, however, when I build the .apk it's like those files never existed at all. How do I manage to save those files in the build so that everything works?

This is the snippet of code I currently have for this situation

I am creating a dataframe that gets initialized with some columns set to null. Before writing out, I typed the dataframe as a case class A. Given that we have a Dataset[A], i assumed that the underlying schema of the dataset would have the correct types; however, the schema is left as NullType. The following is an example of how to reproduce:

I want to use copy command to save multiple csv files in parallel to PostgreSQL database. I am able to save a single csv file to PostgreSQL using copy command. I don't want to save the csv files one by one to the PostgreSQL as it would be sequential and I would be wasting the cluster resources as it has lot of computing happening before it reach this state. I want a way by which I can open the csv files on each partition that I have and run multiple copy commands at the same time.

I was able to find one GitHub repo that does something similar so I tried replicating the code but I am getting the error : Task not serializable

The code that I am using is as below :

Import Statements :

For some reason, it seems like I keep getting 10-Byte 802.11 MAC headers from pcap in c/c++ and I don't know why.

Some intro details:

• Yes, I'm in monitor mode

• Yes, I'm using wlan1mon

• I checked that pcap_open_live returned non-null

• I checked that pcap_datalink returned 127 (802.11 w/ radiotap header)

• I have had a really hard time finding a good reference for the 802.11 MAC header. The Ethernet header, IPv4 header, etc have all had really good references which went into all the necessary detail about every field and how you know if it is/isn't present and/or valid... but nobody even says whether addr4 is omitted entirely if unnecessary or if it's just 0-filled/unfilled. Or what the arrangement of the header is given the different types/subtypes (one site suggested that, sometimes, the frame is just the control, duration, and MAC for acknowledgements, but no other site I've found says the same).

A quick reference for code segments below: I made a macro Assert (which I usually give longer names that conform to the spec, but for now it's just that) which has a condition as the first argument and, if it fails, it uses a stringstream to construct a string and throws a runtime_error if it's false. This lets me make very descriptive error messages which include local variable values when necessary.

Ok, here're where I currently am. Note that this is my first program with pcap and I'm writing it entirely on a Raspberry Pi in vim over ssh from git-bash from Windows, so I'm not exactly in ideal circumstances for formatting. Also things get messy when I try to make the stupid thing not

I wrote simple query which should ignore data where created < last event time - 5 seconds. But this query doesn't work. All data is printed out.

Also I tried to use window function window($"created", "10 seconds", "10 seconds"), but that didn't help.

I am trying to use Spark for writing to HBase table. I am using example with HBase Spark Connector from link. I start the following commands with spark-shell call