r8 | r8 – a simple ctf system | Hacking library

Looking at the result of Cachegrind, it doesn't look like the memory is your bottleneck. The NN has to be stored in memory anyway, but if it's too large that your program's having a lot of L1 cache misses, then it's worth thinking to try to minimize L1 misses, but 1.7% of L1 (data) miss rate is not a problem.

So you're trying to make this run fast anyway. Looking at your code, what's happening at the most inner loop is very simple (load-> multiply -> add -> store), and it doesn't have any side effect other than the final store. This kind of code is easily parallelizable, for example, by multithreading or vectorizing. I think you'll know how to make this run in multiple threads seeing that you can write code with some complexity, and you asked in comments how to manually vectorize the code.

I will explain that part, but one thing to bear in mind is that once you choose to manually vectorize the code, it will often be tied to certain CPU architectures. Let's not consider non-AMD64 compatible CPUs like ARM. Still, you have the option of MMX, SSE, AVX, and AVX512 to choose as an extension for vectorized computation, and each extension has multiple versions. If you want maximum portability, SSE2 is a reasonable choice. SSE2 appeared with Pentium 4, and it supports 128-bit vectors. For this post I'll use AVX2, which supports 128-bit and 256-bit vectors. It runs fine on your CPU, and has reasonable portability these days, supported from Haswell (2013) and Excavator (2015).

The pattern you're using in the inner loop is called FMA (fused multiply and add). AVX2 has an instruction for this. Have a look at this function and the compiled output.

Agree with comments: your stack alignment is incorrect.

It is true that the stack must be aligned to 16 bytes. However, the question is when? The normal rule is that the stack pointer must be a multiple of 16 at the site of a call instruction that calls an ABI-compliant function.

Well, you don't use a call instruction, but what that really means is that on entry to an ABI-compliant function, the stack pointer must be 8 less than a multiple of 16, or in other words an odd multiple of 8, since it assumes it was called with a call instruction that pushed an 8-byte return address. That is just the opposite of what your code does, and so the stack is misaligned for the rest of your program, which makes printf crash when it tries to use aligned move instructions.

You could subtract 8 from the sp computed in your C code.

Or, I'm not really sure why you go to the trouble of loading the destination address into a register, then pushing and ret, when an indirect jump or call would do. (Unless you are deliberately trying to fool the indirect branch predictor?) An indirect call will also kill the stack-alignment bird, by pushing the return address (even though it will never be used). So you could leave the rest of your code alone, and replace all the r8/ret stuff in restore_context with just

it's a bug in the library due to a play services update. To fix it, you should explicitly declare that the pendingDynamicLinkData is nullable.

Like this:

Only 32-bit system calls (e.g. via int 0x80) in 64-bit mode step on those registers, along with R11. (What happens if you use the 32-bit int 0x80 Linux ABI in 64-bit code?).

syscall properly saves/restores all regs including R8, R9, and R10, so user-space using it can assume they keep their values, except the RAX return value. (The kernel's syscall entry point even saves RCX and R11, but at that point they've already been overwritten by the syscall instruction itself with the original RIP and before-masking RFLAGS value.)

Those, with R11, are the non-legacy registers that are call-clobbered in the function-calling convention, so compiler-generated code for C functions inside the kernel naturally preserves R12-R15, even if an asm entry point didn't save them.

Currently the 64-bit int 0x80 entry point just pushes 0 for the call-clobbered R8-R11 registers in the process-state struct that it will restore from before returning to user space, instead of the original register values.

Historically, the int 0x80 entry point from 32-bit user-space didn't save/restore those registers at all. So their values were whatever compiler-generated kernel code left sitting around. This was thought to be innocent because 32-bit mode can't read those registers, until it was realized that user-space can far-jump to 64-bit mode, using the same CS value that the kernel uses for normal 64-bit user-space processes, selecting that system-wide GDT entry. So there was an actual info leak of kernel data, which was fixed by zeroing those registers.

IDK whether there used to be or still is a separate entry point from 64-bit user-space vs. 32-bit, or how they differ in struct pt_regs layout. The historical situation where int 0x80 leaked r8..r11 wouldn't have made sense for 64-bit user-space; that leak would have been obvious. So if they're unified now, they must not have been in the past.

After playing around with this for a while, I discovered that the issue comes from the ConfigurationSaveMode.Full option.

In both ProtectSettings() and UnProtectSettings(), instead of

The bottleneck in all of your examples is the predecoder.

I analyzed your examples with my simulator uiCA (https://uica.uops.info/, https://github.com/andreas-abel/uiCA). It predicts the following throughputs, which closely match your measurements:

The trace table that uiCA generates provides some insights into how the code is executed. For g1a, for example, it generates the following trace:

You can see that for the 32 nops, the predecoder requires 8 cycles, and for the remaining instructions, it requires 5 cycles, which together corresponds to the 13 cycles that you measured.

You may notice that in some cycles, only a small number of instructions is predecoded; for example, in the fourth cycle, only one instruction is predecoded. This is because the predecoder works on aligned 16-byte blocks, and it can handle at most five instructions per cycle (note that some sources incorrectly claim that it can handle 6 instructions per cycle). You can find more details on the predecoder, for example how it handles instructions that cross a 16-byte boundary, in this paper.

If you compare this trace with the trace for g1b, you can see that the instructions after the nops now require 6 instead of 5 cycles to be predecoded, which is because several of the instructions in g1b are longer than the corresponding ones in g1a.

This is made harder by the fact that there are some mistakes in these codes.  The third leaq only has one operand so is missing a target register.  M and N are constants or else there would be code involving variables (and maybe multiplication) for the indexing (there isn't any), but the C code says ++M, which would not be allowed on a constant (this should be ++i instead).

Since M & N are constants, so the element at Array2[M-1][N-1] is a constant offset from Array2 (referring to the very last element of the array).  Since that is used in a loop, the code computes that address in what is called loop invariant code motion — an optimization technique that relocates some fixed/constant computation out of the loop, to do it beforehand instead of repeating the same thing each iteration of the loop.

From the Array2[M-1] part, we derive (M-1)*N to get to the offset for that last row.  From the [N-1] part, we add to that N-1, and then multiply the whole thing by 8, since 8 bytes per long in Array2.

The full offset for that constant part of the indexing is computed then by the formula ((M-1)*N+N-1)*8, and, simplified (M*N-1)*8 and M*N*8-8.  Thus 15992 = M*N*8-8 and 16000 = M*N*8 and 2000 = M*N.

The outer loop is stepping forward by 200 bytes per iteration, which corresponds to an incrementing i, which is used in the first index position for Array1.  Since +1 of the first index of Array1 maps to 200 bytes, the size of a row of Array1 (in elements not bytes) is 200/4 or 50, therefore N=50.

Since N=50 we can reason that 2000=M*50 and thus, 2000/50=40=M.

Basically, one approach is to search the code to figure out how it computes Array2[M-1-i][N-1-j].  This is key b/c it is an expression in the assembly code that uses M.

(Array1[i][j] might have involved N, but not M — but here it has been optimized, the author/compiler recognizing the access pattern is sequential, so that i*N+j is not needed, just a running value with an increment of 4 instead).

It is not trivial, as optimization techniques have been applied; these spread that computation out across different parts of the code rather than appearing together in one place as one perhaps might expect.  Variables have also been eliminated (or heavily modified), substituting indexes and loop control variables for pointers.

This line: movq %rcx,8(%rax)// what does this line do does the assignment write to memory of Array2, basically the = operator in Array2[][]=....  Once that is appreciated, we can reason backward to find the entire indexing computation, which has parts spread out and various constants combined.

(Another approach is to figure out how i and j are done, though since these loop control variables have been changed in favor of pointers, analysis is non-trivial, and include some of the above analysis.)

The loop has one read and one write, both in C and in assembly.  Therefore, the memory write must be the assignment to an element of Array2, and the memory read movslq (%rdx),%rcx must be to fetch an element from Array1.

Let's note that further optimization could change things considerably, for example, loop unrolling, and use of vector registers.

The short version is that the intermediate representation of double/float in the CLI is intentionally unspecified. As such the compiler will always emit an explicit cast from double to double (or float to float) in case it would change the meaning of an expression.

It doesn't change the meaning in this case, but the compiler doesn't know that. (The JIT does though and will optimize it away.)

If you want all the gnitty gritty background details...

The ECMA-335 references below specifically come from the version with Microsoft-Specific implementation notes, which can be downloaded from here. (Note that since we're talking about IL I will be speaking from the perspective of the .NET Runtime's virtual machine, not from any particular processor architecture.)

The justification for why Roslyn emits this seemingly unnecessary instruction can be found in CodeGenerator.EmitIdentityConversion:

An explicit identity conversion from double to double or float to float on non-constants must stay as a conversion. An implicit identity conversion can be optimized away. Why? Because (double)d1 + d2 has different semantics than d1 + d2. The former rounds off to 64 bit precision; the latter is permitted to use higher precision math if d1 is enregistered.

(Emphasis and formatting mine.)

The important thing to note here is the "permitted to use higher precision math". To understand why this is we need to understand how the runtime represents different types at a low level. The virtual machine used by the .NET Runtime is stack-based, all intermediate values go onto what is called the evaluation stack. (Not to be confused with the processor's call stack, which may or may not be used for things on the evaluation stack at runtime.)

Partition I §12.3.2.1 The Evaluation Stack (pg 88) describes the evaluation stack, and lists what can be represented on the stack:

While the CLI, in general, supports the full set of types described in §12.1, the CLI treats the evaluation stack in a special way. While some JIT compilers might track the types on the stack in more detail, the CLI only requires that values be one of:

• int64, an 8-byte signed integer
• int32, a 4-byte signed integer
• native int, a signed integer of either 4 or 8 bytes, whichever is more convenient for the target architecture
• F, a floating point value (float32, float64, or other representation supported by the underlying hardware)
• &, a managed pointer
• O, an object reference
• *, a “transient pointer,” which can be used only within the body of a single method, that points to a value known to be in unmanaged memory (see the CIL Instruction Set specification for more details. * types are generated internally within the CLI; they are not created by the user).
• A user-defined value type

Of note is the only floating point type being the F type, which you'll notice is intentionally vague and does not represent a specific precision. (This is done to provide flexibility for runtime implementations since they have to run on many different processors, which may or may not prefer a specific level of precision for floating point operations.)

If we dig around a little further, this is also mentioned in Partition I §12.1.3 Handling of floating-point data types (pg 79):

Storage locations for floating-point numbers (statics, array elements, and fields of classes) are of fixed size. The supported storage sizes are float32 and float64. Everywhere else (on the evaluation stack, as arguments, as return types, and as local variables) floating-point numbers are represented using an internal floating-point type.

For the final piece of the puzzle, we need to understand the exact definition of conv.r8, which is defined in Partiion III §3.27 conv. - data conversion (pg 68):

conv.r8: Convert to float64, pushing F on stack.

and finally, the specifics of converting F to F are defined in Partition III §1.5 Table 8: Conversion Operations (pg 20): (Paraphrased)

If input (from the evaluation stack) is F and convert-to is "All float types": Change precision³

³Converts from the current precision available on the evaluation stack to the precision specified by the instruction. If the stack has more precision than the output size the conversion is performed using the IEC 60559:1989 “round-to-nearest” mode to compute the low order bit of the result.

So in this context you should read conv.r8 as "Convert from unspecified floating-point format to double" rather than "Convert from double to double". (Although in this case, we can be pretty sure that F on the evaluation stack is already double precision since it's from a double argument.)

So in summary:

• The .NET Runtime has a float64 type, but only for storage purposes.
• For evaluation purposes (and passing arguments), a precision-unspecified F type is must be used instead.
• This means that sometimes an "unnecessary" explicit cast to double is actually changing the precision of an expression.
• The C# compiler doesn't know whether or not it will matter so it always emits the conversion from F to float64. (However the JIT does, and in this case will optimize away the cast at runtime.)

TL;DR: The problem mainly comes from the constant loop bound and not directly from Eigen. Indeed, in the first case, Eigen store the size of the vectors in vector attributes while in the second case, you explicitly use the constant N.

Clever compilers can use this information to unroll loops more aggressively because they know that N is quite big. Unrolling a loop with a small N is a bad idea since the code will be bigger and has to read by the processor. If the code is not already loaded in the L1 cache, it must be loaded from the other caches, the RAM or even the storage device in the worst case. The added latency is often bigger than executing a sequential loop with a small unroll factor. This is why compilers do not always unroll loops (at least not with a big unroll factor).

Inlining also plays an important role in this code. Indeed, if the functions are inlined, the compiler can propagate constants and know the size of the vector enabling it to further optimize the code by unrolling the loop more aggressively. However, if the functions are not inlined, then there is no way the compiler can know the loop bounds. Clever compilers can still generate conditional algorithm to optimize both small loops and big ones but this makes the program bigger and introduces a small overhead. Compilers like ICC and Clang do generate the different code alternatives when the code can be vectorized but the loop bounds are unknown or also when aliasing is not known at compile time (the number of generated variants can quickly be huge and so the code size).

Note that inlining functions may not be enough since the constant propagation can be trapped by a complex conditionals dealing with runtime-defined variables or non-inlined function calls. Alternatively, the quality of the constant propagation may not be sufficient for the target example.

Finally, aliasing also play a critical role in the ability of compilers to generate SIMD instructions (and possibly better unroll the loop) in this code. Indeed, aliasing often prevent the use of SIMD instructions and it is not always easy for compilers to check aliasing and generate fast implementations accordingly.

If the vector-based implementation use a loop bound stored in the vector objects, then the code generated by MSVC is not vectorized in the benchmark: the constant is not propagated correctly despite the inlining of the function. The resulting code should be much slower. Here is the generated code: