ijk | A real editor for real programmers | Editor library

You theoreticaly can compute the 3d-gaussian convolution using three 2d-convolutions, but that would mean you have to reduce the size of the 2d-kernel, as you're effectively convolving in each direction twice.

But computationally more efficient (and what you usually want) is a separation into 1d-kernels. I changed the second part of your function to implement this. (And I must say I really liked your permutation-based appraoch!) Since you're using a 3d volume you can't really use the conv2d or conv1d functions well, so the best thing is really just using conv3d even if you're just computing 1d-convolutions.

Note that allclose uses a threshold of 1e-8 which we do not reach with this method, probably due to cancellation errors.

When I saw your post I though the solution should be a simple optional match for separator like:

The code that gave you the ValueError: matrices are not aligned just needs .values added for the matrix multiplication to work and you can use @ for both multiplication steps so you get a DataFrame back.

First of all, Numba can perform parallel computations resulting in a faster code if you manually request it using mainly parallel=True and prange. This is useful for big arrays (but not for small ones).

Moreover, your computation is mainly memory bound. Thus, you should avoid creating big arrays when they are not reused multiple times, or more generally when they cannot be recomputed on the fly (in a relatively cheap way). This is the case for r_0 for example.

In addition, memory access pattern matters: vectorization is more efficient when accesses are contiguous in memory and the cache/RAM is use more efficiently. Consequently, arr[0, :, :] = 0 should be faster then arr[:, :, 0] = 0. Similarly, arr[:, :, 0] = arr[:, :, 1] = 0 should be mush slower than arr[:, :, 0:2] = 0 since the former performs to noncontinuous memory passes while the latter performs only one more contiguous memory pass. Sometimes, it can be beneficial to transpose your data so that the following calculations are much faster.

Moreover, Numpy tends to create many temporary arrays that are costly to allocate. This is a huge problem when the input arrays are small. The Numba jit can avoid that in most cases.

Finally, regarding your computation, it may be a good idea to use GPUs for big arrays (definitively not for small ones). You can give a look to cupy or clpy to do that quite easily.

Here is an optimized implementation working on the CPU:

Numpy arrays can be indexed using other arrays as indices. See also: NumPy selecting specific column index per row by using a list of indexes.

With that in mind, we can vectorize your loop to simply use b for indexing:

I am trying to get a Matrix with shape (N,M,d) in such manner that I do element wise product between B and each element of A (which are M elements).

The operation you are trying to perform is a broadcasted element wise product over axis = 1 of A (M size) -

The use of eval(parse(.)) should be avoided in all but the most extreme situations.

A first cut uses a lesser-known way to index on an array with a matrix. For demonstration, I'll interrupt the function call on the first for loop and show the indexing on a modified array:

You can use

You can use the .replace() function.