sliding_window | Python package to run sliding window on numpy array | Machine Learning library

I have been trying to replicate some work similar to this code below but when I try to use this data from this link https://raw.githubusercontent.com/plotly/datasets/master/api_docs/mt_bruno_elevation.csv Its throwing some error. I think its because of shape but don't know exactly how to modify it.

It will be great, if you help me to resolve the issue.

Here is my Code

How can I compute a sliding window in Spark without resorting to spark streaming?

NOTICE: I do not want to use a WINDOW PARTITION BY ORDER BY k ROWS before/after current one, but use the timestamp. The window operator has such a mode:

I have some binary data to parse, and need to find where packets start. All packets start with the same header, but packet size is variable. The header is a 32 bit unsigned integer.

Below is my implementation, but it's slow. Is there some numpy functionality or other options to make this operation faster?

I am using the following code to apply a sliding window on time-series data. I want to set up my first window as fixed and then apply the sliding window as shown below in the desired output.

I have this simple code

I got good use out of pandas' MovingOLS class (source here) within the deprecated stats/ols module. Unfortunately, it was gutted completely with pandas 0.20.

The question of how to run rolling OLS regression in an efficient manner has been asked several times (here, for instance), but phrased a little broadly and left without a great answer, in my view.

Here are my questions:

1. How can I best mimic the basic framework of pandas' MovingOLS? The most attractive feature of this class was the ability to view multiple methods/attributes as separate time series--i.e. coefficients, r-squared, t-statistics, etc without needing to re-run regression. For example, you could create something like model = pd.MovingOLS(y, x) and then call .t_stat, .rmse, .std_err, and the like. In the example below, conversely, I don't see a way around being forced to compute each statistic separately. Is there a method that doesn't involve creating sliding/rolling "blocks" (strides) and running regressions/using linear algebra to get model parameters for each?

2. More broadly, what's going on under the hood in pandas that makes rolling.apply not able to take more complex functions?* When you create a .rolling object, in layman's terms, what's going on internally--is it fundamentally different from looping over each window and creating a higher-dimensional array as I'm doing below?

*Namely, func passed to .apply:

Must produce a single value from an ndarray input *args and **kwargs are passed to the function

Here's where I'm currently at with some sample data, regressing percentage changes in the trade weighted dollar on interest rate spreads and the price of copper. (This doesn't make a ton of sense; just picked these randomly.) I've taken it out of a class-based implementation and tried to strip it down to a simpler script.

I execute python code for an image recognition task and it works very well but when I implement it into an android application using Chaquopy it dosn't work. This is because the first insruction in the python code aims to open the camera using this insruction cap = cv2.videoCapture(0) and this instruction dosn't work on the android device. I have also added camera and storage permissions in the androidManifest.xml This is my Java code :

I have a progression "a", where the first two numbers are given (a1 and a2) and every next number is the smallest sum of subarray which is bigger than the previous number.

For example if i have a1 = 2 and a2 = 3, so the progression will be

2, 3, 5(=2+3), 8(=3+5), 10(=2+3+5), 13(=5+8), 16(=3+5+8), 18(=2+3+5+8=8+10), 23(=5+8+10=10+13), 26(=3+5+8+10), 28(=2+3+5+8+10), 29(=13+16)...

I need to find the Nth number in this progression. ( Time limit is 0.7 seconds)

(a1 is smaller than a2, a2 is smaller than 1000 and N is smaller than 100000)

I tried priority queue, set, map, https://www.geeksforgeeks.org/find-subarray-with-given-sum/ and some other things.

I though that the priority queue would work, but it exceeds the memory limit (256 MB), so i am pretty much hopeless.

Here's what is performing the best at the moment.

I have a custom function that goes through a time series data frame and returns a 30 minutes sliding window from the original time series. From this 30 minutes the function writes to another data frame the starting and finishing timestamps together with the minimum and maximum of this sliding window.

After that sapply statement is used to make this function recursive across the whole data range.

The sapply is too slow but works. I want to be able to parallelize the sapply, but when I do so code returns errors. I attribute this to the requirement of writing the final result of the function to the same data frame in parallel.