confidence | Simple configuration usage module for python apps | Configuration Management library

Assuming what I mentioned in my comment on your question, the fix is just to simplify your FieldWithConfidence type significantly. Right now it is trying to add a number of additional levels of structure beyond what you seem to want. Here is a version of that type that works as I think you intend:

We could calculate the interactions by hand; done easily by first creating the terms trms, then evaluating them in an eval(parse()) approach.

At my job I support MySQL instances that have multi-billion row tables. At that scale, care must be taken to optimize queries properly. You don't want to do a table-scan at that scale.

But that's about rows, not columns. You asked first about columns.

The way to choose between adding a column versus adding another table is to follow rules of database normalization. If the new column is for an attribute of the same entity as your current table, add the column to that table. If it's a multi-valued attribute or if it's really an attribute of some other entity, then add it to a different table.

Very, very rarely is it the right choice to make another table solely for the sake of having too many columns. A given MySQL table can have dozens of columns pretty easily, and hundreds if you're careful.

In theory, there is no limit to the number of columns that might be appropriate to put in the same table with respect to normalization. But there are limitations due to the code to store those columns in a given implementation (e.g. InnoDB storage engine in MySQL).

See https://www.percona.com/blog/2013/04/08/understanding-the-maximum-number-of-columns-in-a-mysql-table/

So the maximum number of columns for a table in MySQL is somewhere between 191 and 2829, depending on a number of factors.

In the comments on that blog, I was able to design a table that failed to be created at 59 columns. Read the blog for details.

Answer

You have to add the font.family argument to fviz_pca:

When you print an object, either by using print or by sending it to the R console, some extra processing may happen. Every object (almost always) has its own print and in this case you can see that the print.psych method (called behind the scenes instead of print on any psych package object) is doing the following with your object of (sub)class alpha:

Restating the problem - My understanding given the different comments and your updates is the following:

• someone other than you is in possession of data, which as it happens is 2D data, i.e. an Nx2 matrix;
• using the covariance matrix, they are effectively saying something about the joint distribution of these two dimensions, specifically about the variance;
• if they assume a Gaussian distribution, as is implied by your comment regarding 68%, 95% and 99.7% for 1sigma, 2sigma and 3sigma, they can draw ellipses which represent the 2D-normal distribution: these are in fact some of the contour lines associated with the 3D "bell" surface;
• you have obtained the contour lines in a graph and are trying to obtain the covariance matrix (not the original data...);
• you are concerned about the complexity of having to extract the information from each ellipsis.

Partial answer:

• It is impossible to recover the original data, I hope you are already aware of that, but in case you are not let's just note that the covariance matrix is a summary statistic of the data, much like the average, and although it says something about the data many different datasets could happen to have the same summary statistic (the same way many different sets of numbers can give you an average of 10).
• It is possible to somewhat recover the covariance matrix, i.e. the 3 numbers a, b and c in the matrix [a,b;b,c], though the error in doing so will likely be large because of how imprecise the pixel representation is. Essentially, you will be looking for the dimensions of the two axes, for the variances, as well as the angle of one of the axes, for the covariance.
• Unless I am mistaken, under the Gaussian assumption above, you only need to measure this for one of the three ellipses, and then factor by whatever number of sigmas that contour represents. Here you might want to either use the best-defined ellipse, or attempt to use the largest one, which will provide the maximum precision for your measurements (cf. pixelization).
• Also, the problem of finding the axes and angle for the ellipse need not be as complex as what it seems like in your first trials: instead of trying to find the contour of the ellipses, find the bounding rectangle.
• In order to further simplify this process, if your images are color-coded the way you show, then a filter on blue pixels might be enough in terms of image processing. Then simply take the minimum and maximum (x,y) coordinates in order to obtain the bounding rectangle.
• Once the bounding rectangle is obtained, find the equation to your ellipse (that's a question for a math group, but you could start here for example).

Happy filtering!

Here is a way with aggregate.

blandr creates a ggplot object, so you can modify the colors with the ggplot_build method.

The reason the first query is so much faster is it went parallel. This means the cardinality estimator knew enough about the data it had to handle, and the query was large enough to tip the threshold for parallel execution. Then, the engine passed chunks of data for different processors to handle individually, then report back and repartition the streams.

With the value as a variable, it effectively becomes a scalar function evaluation, and a query cannot go parallel with a scalar function, because the value has to determined before the cardinality estimator can figure out what to do with it. Therefore, it runs in a single thread, and is slower.

Some sort of looping mechanism might help. Create the included indexes to assist the engine in handling this request. You can probably find a better looping mechanism, since you are familiar with the identity ranges you care about, but this should get you in the right direction. Adjust for your needs.

With a loop like this, it commits the changes with each loop, so you aren't locking the table indefinitely.