numba | NumPy aware dynamic Python compiler using LLVM | Compiler library

Have you installed numpy using pip?

Have you tried FLANN?

This code doesn't solve your problem completely. It simply finds the nearest 50 neighbors to each point in your 500000 point dataset:

So, how can I still allow mixed data type intputs but keep the speed, instead of creating functions each for different types?

The problem is that the Numba function is defined only for float64 types and not int64. The specification of the types is required because Numba compile the Python code to a native code with well-defined types. You can add multiple signatures to a Numba function:

This is slow because the memory access pattern is very inefficient. Indeed, random accesses are slow because the processor cannot predict them. As a result, it causes expensive cache misses (if the array does not fit in the L1/L2 cache) that cannot be avoided by prefetching data ahead of time. The thing is the arrays are to big to fit in caches: index_a and a takes each 457 MiB and b takes 156 KiB. As a results, access to b are typically done in the L2 cache with a higher latency and the accesses to the two other array are done in RAM. This is slow because the current DDR RAMs have huge latency of 60-100 ns on a typical PC. Even worse: this latency is likely not gonna be much smaller in a near future: the RAM latency has not changed much since the last two decades. This is called the Memory wall. Note also that modern processors fetch a full cache line of usually 64 bytes from the RAM when a value at a random location is requested (resulting in only 56/64=87.5% of the bandwidth to be wasted). Finally, generating random numbers is a quite expensive process, especially large integers, and np.random.randint can generate either 32-bit or 64-bit integers regarding the target platform.

The first improvement is to prefer indirection on the most contiguous dimension which is generally the last one since a[:,i] is slower than a[i,:]. You can transpose the arrays and swap the indexed values. However, the Numpy transposition function only return a view and does not actually transpose the array in memory. Thus an explicit copy in currently required. The best here is simply to directly generate the array so that accesses are efficient (rather than using expensive transpositions). Note you can use simple precision so array can better fit in caches at the expense of a lower precision.

Here is an example that returns a transposed array:

Try this:

This problem is known as the "AoS VS SoA" where AoS means array of structures and SoA means structure of arrays. You can find some information about this here. SoA is less user-friendly than AoS but it is generally much more efficient. This is especially true when your code can benefit from using SIMD instructions. When you deal with many big array (eg. >=8 big arrays) or when you perform many scalar random memory accesses, then neither AoS nor SoA are efficient. In this case, the best solution is to use arrays of structure of small arrays (AoSoA) so to better use CPU caches while still being able benefit from SIMD. However, AoSoA is tedious as is complexity significantly the code for non trivial algorithms. Note that the number of fields that are accessed also matter in the choice of the best solution (eg. if only one field is frequently read, then SoA is perfect).

OOP is generally rather bad when it comes to performance partially because of this. Another reason is the frequent use of virtual calls and polymorphism while it is not always needed. OOP codes tends to cause a lot of cache misses and optimizing a large code that massively use OOP is often a mess (which sometimes results in rewriting a big part of the target software or the code being left very slow). To address this problem, data oriented design can be used. This approach has been successfully used to drastically speed up large code bases from video games (eg. Unity) to web browser renderers (eg. Chrome) and even relational databases. In high-performance computing (HPC), OOP is often barely used. Object-oriented design is quite related to the use of SoA rather than AoS so to better use cache and benefit from SIMD. For more information, please read this related post.

To conclude, I advise you to use the first code (SoA) in your case (since you only have two arrays and they are not so huge).

At a guess (and see @IanBush's comments if you want to enable us to do better than guessing), it's the line

Numba currently uses LLVM-Lite to compile the code efficiently to a binary (after the Python code has been translated to an LLVM intermediate representation). The code is optimized like en C++ code would be using Clang with the flags -O3 and -march=native. This last parameter is very important as is enable LLVM to use wider SIMD instructions on relatively-recent x86-64 processors: AVX and AVX2 (possible AVX512 for very recent Intel processors). Otherwise, by default Clang and GCC use only the SSE/SSE2 instructions (because of backward compatibility).

Another difference come from the comparison between GCC and the LLVM code from Numba. Clang/LLVM tends to aggressively unroll the loops while GCC often don't. This has a significant performance impact on the resulting program. In fact, you can see that the generated assembly code from Clang:

With Clang (128 items per loops):

The slower execution time of the Numba implementation is due to the compilation time since Numba compile the function at the time it is used (only the first time unless the type of the argument change). It does that because it cannot know the type of the arguments before the function is called. Hopefully, you can specify the argument type to Numba so it can compile the function directly (when the decorator function is executed). Here is the resulting code: