poly.poly | Polyglots | Internationalization library

Well it looks like the way to correctly Cross-Validate this is with

I dug around a lot and found out what was going on. It turns out numpy has two sets of polynomial tools, one in the base numpy library, and another in numpy.polynomial and they expect things in opposite order. Both polyfit and polyval are found in both libraries and appear to operate the same on this simple case, but their parameters are different:

From numpy:

I have to say that I find your question formulation very confusing, so I can only help you with a bit general answer. Perhaps you could split your big question in several smaller ones next time.

To start with, I assume that your question is: how does the amount of points in a differentiation stencil matter, when I will do polynomial interpolation on the derivative afterwards?

The number of points in a stencil generally increases the accuracy of the computation of the derivative. You can see that by filling in Taylor expansions for the variables in the numerical derivative. After terms cancels you are left with some higher-order term that gives you a lower bound on the error that you make. The underlying assumption however is, that the function (in your case C) of which you compute the derivative is smooth on the interval that you compute the derivatives on. Meaning that if your function is not nicely behaved on your 15-point stencil, then that derivative is essentially worthless.

The order of the polynomial in polynomial regression is usually a free parameter chosen by the user because the user might know that their series is behaved like a polynomial up to certain degree, but unaware of the polynomial coefficients. If you know something about your data, you can set the degree yourself. For instance if you know your data is linear correlated with step, you could set the degree to 1 and you have linear regression. In this case you don't want to specify any higher degree because your data will likely fit to a polynomial, which you know is not the case. In a similar way, if you know your data behaves like a polynomial of some degree, you certainly don't want to fit any higher. If you have really no clue what degree the polynomial should be, then you should make an educated guess. A good strategy is just to plot the polynomial going through the data points, upping the polynomial one degree at the time. You then want the line to go in between the points, and not diverge towards specific points. In case you have many outliers, there exists method that are better suited than least-squares.

Now onward to your problem specifically.

• There is no way to compute an optimal degree, unless you have more information about your data. Degree is a hyperparameter. If you want an optimum for it, you need to put additional prior information, such as "I want the lowest degree polynomial that fits the data with an error epsilon."
• Overfitting is simply fixed by choosing a lower degree polynomial. If that does not fix it, then least-squares regression is not for you. You need to look into a regression method that chooses a different metric, or you need to preprocess your data, or you need a non-polynomial fit (fit a function of certain shape, then use Levenberg-Marquardt for instance).
• 15-step derivative looks very questionable, you do likely not have this kind of smoothness in your data. If you have a good reason for this, tell us, otherwise just use 2 points for the first derivative, or 3 or 5 for the second.
• The expression with the Landau symbol (big-O) is not switching fourth order to second. If you subtract the two equations and divide by h^2 the O(h^4)/h^2 becomes O(h^2).

You have switched the order of polynomial coefficients and the x-values. As per docs, the first argument is the polynomial coefficients:

numpy.polyval(p, x)

Parameters:

p : array_like or poly1d object 1D array of polynomial coefficients (including coefficients equal to zero)

x : array_like or poly1d object A number, an array of numbers, or an instance of poly1d, at which to evaluate p.

Change the following line

From the numpy.polynomial.polynomial.polyfit docs (not to be confused with numpy.polyfit which is not interchangable) :

x : array_like, shape (M,)

y : array_like, shape (M,) or (M, K)

Your ys needs to be transposed to have ys.shape[0] equal to xs.shape

Since it looks like you are handling each row independently and perform curve fitting not matter what other rows look like, I think you can simply parallelize the code using joblib, so you can do

I use numpy.polyval, docs are at https://docs.scipy.org/doc/numpy/reference/generated/numpy.polyval.html - here is a graphical polynomial fitter as an example that uses polyval.

In general I assume Matplotlib knows the range of your data, so I'd start by looking at the data it is plotting. Looking at the x data, the y-intercept should be 3.08, while x_coefs is array([-5.13333333, 1.66666667] so your fit functions are off somehow (ipython or print(x_coefs) is your friend here).

You need to reverse x and z in x_coefs = poly.polyfit(x, z, 1), similarly for y_coefs.