ICPC | International Clopidogrel Pharmacogenomics Consortium

1 FP operation per core clock cycle would be pathetic for a modern superscalar CPU. Your Skylake-derived CPU can actually do 2x 4-wide SIMD double-precision FMA operations per core per clock, and each FMA counts as two FLOPs, so theoretical max = 16 double-precision FLOPs per core clock, so 24 * 16 = 384 GFLOP/S. (Using vectors of 4 doubles, i.e. 256-bit wide AVX). See FLOPS per cycle for sandy-bridge and haswell SSE2/AVX/AVX2

There is a a function call inside the timed region, callq 403c0b <_Z12do_timed_runRKmRd+0x1eb> (as well as the __kmpc_end_serialized_parallel stuff).

There's no symbol associated with that call target, so I guess you didn't compile with debug info enabled. (That's separate from optimization level, e.g. gcc -g -O3 -march=native -fopenmp should run the same asm, just have more debug metadata.) Even a function invented by OpenMP should have a symbol name associated at some point.

As far as benchmark validity, a good litmus test is whether it scales reasonably with problem size. Unless you exceed L3 cache size or not with a smaller or larger problem, the time should change in some reasonable way. If not, then you'd worry about it optimizing away, or clock speed warm-up effects (Idiomatic way of performance evaluation? for that and more, like page-faults.)

1. Why are there non-conditional jumps in code (at 403ad3, 403b53, 403d78 and 403d8f)?

Once you're already in an if block, you unconditionally know the else block should not run, so you jmp over it instead of jcc (even if FLAGS were still set so you didn't have to test the condition again). Or you put one or the other block out-of-line (like at the end of the function, or before the entry point) and jcc to it, then it jmps back to after the other side. That allows the fast path to be contiguous with no taken branches.

1. Why are there 3 retq instances in the same function with only one return path (at 403c0a, 403ca4 and 403d26)?

Duplicate ret comes from "tail duplication" optimization, where multiple paths of execution that all return can just get their own ret instead of jumping to a ret. (And copies of any cleanup necessary, like restoring regs and stack pointer.)

Yes, the cols_end array is incorrect as pointed out by CJR. They should be indexed as 2,3,4,5. Please see the documentation regarding the parameter to the function mkl_sparse_d_create_csc

cols_end:

This array contains col indices, such that cols_end[i] - ind - 1 is the last index of col i in the arrays values and row_indx. ind takes 0 for zero-based indexing and 1 for one-based indexing.

https://www.intel.com/content/www/us/en/develop/documentation/onemkl-developer-reference-c/top/blas-and-sparse-blas-routines/inspector-executor-sparse-blas-routines/matrix-manipulation-routines/mkl-sparse-create-csc.html

Change this line

cols_end[i] = static_cast((--X.end_col(i)).pos());

to

cols_end[i] = static_cast((X.end_col(i)).pos());

Now recompile and run the code. I've tested it and it is showing the correct results. Image with results and compilation command

I decided to just first collect all even and then odd ones on the screen as an answer, and in those units of cases in which this did not work, I simply hardcode (there were only two of them: 3x3; 2x3). You can see the code above.

Your kernel code looks good and the cache tiling implementation is correct. Only make sure that the number of bodies is a multiple of local size, or alternatively limit the inner for loop to the global size additionally.

OpenCL allows usage of multiple devices in parallel. You need to make a thread with a queue for each device separately. You also need to take care of device-device communications and synchronization manually. Data transfer happens over PCIe (you also can do remote direct memory access); but you can't use NVLink with OpenCL. This should not be an issue in your case though as you need only little data transfer compared to the amount of arithmetic.

A few more remarks:

• In many cases N-body requires FP64 to sum up the forces and resolve positions at very different length scales. However on the A6000, FP64 performance is very poor, just like on GeForce Ampere. FP32 would be significantly (~64x) faster, but is likely insufficient in terms of accuracy here. For efficient FP64 you would need an A100 or MI100.
• Instead of 1.0/sqrt, use rsqrt. This is hardware supported and almost as fast as a multiplication.
• Make sure to use either FP32 float (1.0f) or FP64 double (1.0) literals consistently. Using double literals with float triggers double arithmetic and casting of the result back to float which is much slower.

EDIT: To help you out with the error message: Most probably the error at clCreateKernel (what value does status have after calling clCreateKernel?) hints that program is invalid. This might be because you give clBuildProgram a vector of 2 devices, but set the number of devices to only 1 and also have context only for 1 device. Try

Try using magma sgetri gpu - inverse matrix in single precision, GPU interface. This function computes in single precision the inverse A^−1 of an m × m matrix A.

The problem comes from collapse(2) clause and is also related to the code auto-vectorization. Indeed, both compilers are not able to auto-vectorize the loop with the collapse, but ICC use a very expensive idiv instruction in the middle of the hot loop (which is very bad) while GCC produce a better code. This comes from the collapse(2) clause which is not well optimized (on many compilers). You can see that on GodBold. Note that optimizing a kernel using a collapse(2) clause is not easy since the compiler do not know the bound of the loops and so the associated cost (as well as the divider for the modulus).

Without the collapse(2), GCC is able to vectorize the loop successfully but surprisingly not ICC. Hopefully, we can help ICC using the simd directive. Once used, the two compilers generate a relatively good code. It is still not optimal because size_t is generally 8 bytes and int is 4 bytes on mainstream x86-64 platforms and the loop comparison of the different types makes the code harder to vectorize efficiently as well as to produce the best scalar instructions. You can use a temporary variable to fix that. You can see the resulting assembly code here.

Note that the assembly generated by ICC is very good once the code is fixed. The code is memory bound and the final code should saturate the RAM with only few threads. Even the L3 cache should be saturated with the ICC produced assembly if the input array would fit into it.

Here is the fixed code:

Ok, so after also posting this question on the Intel forum - seems to be (a very much non critical) bug in the compiler - the workaround would be writing both, virtual and override.

This is discouraged by the item C.128 in the cpp core guidelines (same link as in the comments) but not problematic.

Do you see cxx11 in the linker errors? It looks like your libcall_ts_cpp is compiled in a way that expects the new C++11 ABI for std::string but perhaps the library where those functions are implemented was compiled with the old ABI. Here's a PyTorch forum post about the same problem: https://discuss.pytorch.org/t/issues-linking-with-libtorch-c-11-abi/29510/11

The solution is to download a new copy of the PyTorch libraries built with the new C++11 ABI.

My question is whether it is always a good practice to remove sizes on inlining and inline as much as possible?

No, it is not always a good pratice to remove size limits on inlining nor to inline as much as possible.

Ideally, inlining should be done only when it improves performance.

Are there situations at all where imposing an inlining limit is useful?

If a function is very large, and it is called from many contexts, then inlining such function to all of those contexts will bloat the executable. If the executable itself is let's say several gigabytes because of inlining, then loading the program from the disk may become the bottleneck.

In less pathological cases, the trade-offs are more subtle. The way to find out optimal limits is measurement. Profile guided optimisation can give the optimiser more useful heuristics than simple hard limits.