matrixmultiply | General matrix multiplication of f32 and f64 matrices | Math library

First of all, I suggest that performance questions include complete codes. I generally need to be able to run stuff, and you can save me some typing. Sure, you can leave stuff out. Sure, I can probably figure out what it is. But I'm less likely to help you that way, and I suspect I'm not alone in that view. My advice: Make it easy for others to help you. I've given examples of what would be useful below.

On to the question:

The difference is that C uses a 1D array whereas Fortran uses 2D. But that should not be a problem since underneath the memory will be contiguous.

TL;DR: Your claim ("that should not be a problem") is evidently not supportable. The difference between a 1D allocation and a 2D allocation matters, not only from a storage perspective but also from an index-calculation perspective. If you're sensitive to the length of this answer, skip to note D at the bottom of this post.

Details:

When we have a loop like this:

I think I figured out part of the problem. In my matrixPower method, I was supposed to just take the object I passed in and set it equal to matrixMultiply(fmA) in the for loop, instead of making a new object. Removing fmB from that method is now returning the correct number to my main call. Much to my surprise, since I thought my solution was further off.

Edit: The rest of the problem was that I wasn't using repeated squaring.

While teaching myself c, I thought it would be good practice to write a function which multiplies two 3x3 matrices and then make it more general. The function seems to calculate the correct result for the first and last columns but not the middle one. In addition, each value down the middle column is out by 3 more than the last.

In practice, when coding in C, you should take care of the following issues:

• refer to a good C reference website and read a good C programming book, such as Modern C

• floating point numbers are not mathematical real numbers, see floating-point-gui.de for much more. For example, addition is associative in math, but not on a computer using IEEE-754.

• we all make bugs (e.g. buffer overflows or undefined behavior). So you need to learn how to use a debugger. I recommend GDB. But you need to learn how to use it and spend a few hours reading documentation. Tools like valgrind are also useful (to hunt memory leaks) as soon as you use C dynamic memory allocation.

• recent compilers can be helpful. I recommend GCC. You should invoke it with all warnings and debug info, e.g. gcc -Wall -Wextra -g. Be sure to spend some time in reading the documentation of your compiler. You might later consider using static program analysis tools such as Frama-C or the Clang analyzer or (for precision analysis) Fluctuat or CADNA

• consider having a matrix abstract data type like here. You would then easily generalize your code to "arbitrary" N*M matrixes.

• later, for benchmarking purposes, you will want to use an optimizing compiler. If you use GCC, you could compile your code using gcc -Wall -Wextra -g -O3 but then you could have surprising optimizations, see e.g. this draft report.

• in some cases, you could need arbitrary-precision arithmetic. Consider then using specialized libraries such as GMPlib.

• Most computers today are multi-core. You could want to use Pthreads or MPI to take advantage of that with concurrent programming.

• many open source libraries exist for scientific computations. Look at least for inspiration on github and gitlab and see also this list. You could be interested by GNU GSL and study its source code since it is free software (and later improve it).

If you want to make serious scientific computations, you might consider switching (for expressiveness) to functional languages such as Ocaml. If you care about making a lot of iterative computing (like in finite element methods) you might switch to OpenCL or OpenACC.

Expect to spend a decade in learning it.

I'm open to any criticism on how it's written as well.

I figured it out. The error "Array: Parameter 'values' is required" had to do with sparse matrices, which was a product of filtering, clipping and spesifying regions within to perform PCA. Earth Engine can not work with sparse matrices.

Here is the working code. LandsatCol is my preproccessed image collection.

An af::array instance has a method .get(), from which you can retrieve an af_array instance.

The direct reason for memory leaks you are experiencing is that you are not releasing allocated memory by sub operations, e.g. in that line:

This matrix-multiplication code is very inefficient!

Indeed, The line in2[k*M2Rdim+j] is likely to cause cache thrashing and thus high-instability in the computation timing if lines have often to be reloaded from the MCD-RAM. Although the MCD-RAM have a high bandwidth, it also have a high latency (similar to the one of the DDR-RAM). The latency is probably a huge issue in this case.

Specifically, striding down one column of a matrix is terrible for spatial locality. And even worse when the matrix dimension is a power of 2: you're likely to get conflict misses on cache because all those cache lines will alias to the same set in a set associative cache. This can lead to cache thrashing even with a small working set.

Thus, please use BLAS functions (from MKL, OpenBLAS, ATLAS, etc.)! They are far more optimized than that. If you cannot, please consider improving this code. You can find a quite good explanation of you to do that here. I think that a speed-up of more than 10 is easily achievable.

I also advise you to profile your code using tools like perf or VTune that enable you to analyze hardware events (such as L1/L2 cache operations) and confirm/reject the cash-thrashing hypothesis as well as helping you to improve this code.