dble | A High Scalability Middle-ware for MySQL Sharding | Database library

It might help to read the file once, and write it to lots of one-line files, e.g.

Here is one solution: make an array of the matrix size, compute the some_function for all i,j and record how many js give a nonzero result, store that in location i. This is perfectly parallel.

Now you know how much space you need, and you can give each thread its starting point in the storage. Go through the some_function again and actually fill in the elements, with cnt local to each thread.

Ok, so this doubles the amount of scalar work. But you make it completely parallel, so you don't really care, right?

I deleted everything gfortran related and started over.

I downloaded the .tar.xz file here using the browser, which dumped the file into my Downloads folder. I double-clicked it to unpack it.

I moved that directory to where R wanted it:

When you perform a forward and then a back discrete Fourier Transform on some data the normalisation of the result is conventional, usually you either get the data back as it was (to floating point accuracy), or if you are using an unnormalised transform the data will be scaled by the number of points in the data set, or you provide the normalisation as an argument. To find out which you will have read the documentation of whatever software you are using to do the transforms; fftw uses unnormalised transforms . Thus in your code you will need to preform the appropriate scaling. And if you run your code on your datasets you find the scaling is as described - on a 10x10 dataset the final data is 100 times the original data.

I cannot reproduce your claim that the code as given works for the smaller data sets. I get the expected scaling.

Your code explicitly assumes the date will have 5 characters in the form mm/dd, but your first input violates that assumption as it is only 1/1. If the date were formatted in all cases with 2-digit month and 2-digit date (i.e. 01/01), your code would work.

You need to use the date parsing capabilities that are built-in to Java (java.time package).

Math.getExponent() returns the exponent of the binary representation of the number. In your example -13 is the exponent of the decimal representation, and -43 the exponent of the binary representation.

For example,

As stated in the comments one of your lines is too long. With gfortran at least if you turn warnings up to the max (as you should if developing code) you get a more informative message

The problem with vectorizing loops that include the math functions (like log) is that the compiler has to be taught the semantics of the vectorized math functions (and you see if you look at the assembler output that the Fortran version calls the "normal" scalar function (a line like call log) whereas your C++ version calls the vectorized version (call _ZGVdN4v___log_finite)). There has been some work wrt making GFortran understand the glibc vector math library (libmvec), but I'm not sure what the current status is. See the thread starting at https://gcc.gnu.org/legacy-ml/gcc/2018-04/msg00062.html and continuing in June 2018 starting at https://gcc.gnu.org/legacy-ml/gcc/2018-06/msg00167.html for more details.