Kernels | simple programs that can be used to explore the features | GPU library

There isn't really a good reason other than we don't bother testing the control plane on Windows. In theory it's all just Go daemons that should compile fine on Windows but you would be on your own if any problems arise.

In keras, loss function should return the loss value without regularization losses. The regularization losses will be added automatically by setting kernel_regularizer or bias_regularizer in each of the keras layers.

In other words, when you write your custom loss function, you don't have to care about regularization losses.

Edit: the reason why you got the warning messages that gradients don't exist is because of the usage of numpy() in your loss function. numpy() will stop any gradient propagation.

The warning messages disappeared after you added regularizers to the layers do not imply that the gradients were then computed correctly. It would only include the gradients from the regularizers but not from the data. numpy() should be removed in the loss function in order to get the correct gradients.

One of the solutions is to keep everything in tensors and use tf.math library. e.g. use tf.pow to replace np.float_power and tf.reduce_sum to replace np.sum

Your GTX770 GPU is a "Kepler" architecture compute capability 3.0 device. These devices were deprecated during the CUDA 10 release cycle and support for them dropped from CUDA 11.0 onwards

The CUDA 10.2 release is the last toolkit with support for compute 3.0 devices. You will not be able to make CUDA 11.0 or newer work with your GPU. The query and bandwidth tests use APIs which don't attempt to run code on your GPU, that is why they work where any other example will not work.

fast_compiled is faster than fast_compiled_strides because it works on contiguous data known at compile time enabling compilers to use SIMD instructions (eg. typically SSE on x86-like platforms or NEON on ARM ones). It should also be faster because of less data cache to retrieve from the L1 cache (more fetches are needed due to the indirection).

Indeed, dans[j] += weights[k] can be vectorized by loading m items of dans and m items of weights adding the m items using one instruction and storing the m items back in dans. This solution is efficient and cache friendly.

dans[strides[i]] += weights[i] cannot be efficiently vectorized on most mainstream hardware. The processor need to perform a costly gather from the memory hierarchy due to the indirection, then do the sum and then perform a scatter store which is also expensive. Even if strides would contain contiguous indices, the instructions are generally much more expensive than loading a contiguous block of data from memory. Moreover, compiler often fail to vectorize the code or just find that this is not worth using SIMD instruction in that case. As a result the generated code is likely a less efficient scalar code.

Actually, the performance difference between the two codes should be bigger on modern processors with good compilation flags. I suspect you only use SSE on a x86 processor here and so the speed up is close to 2 theoretically since 2 double-precision floating-point numbers can be computed in a row. However, using AVX/AVX-2 would lead to a speed up close to 4 theoretically (as 4 numbers can be computed in a row). Very recent Intel processors can even compute 8 double-precision floating-point numbers in a row. Note that computing simple-precision floating-point numbers can also results in a theoretical 2x speed up. The same apply for other architecture like ARM with NEON and SVE instruction sets or POWER. Since future processors will likely use wider SIMD registers (because of their efficiency), it is very important to write SIMD-friendly codes.

#define directive can be used for that:

cd to your cert folder, and type this command:

You need to check whether you have installed ipython and ipykernel with the command pip list.

Then try to reinstall or upgrade it with command:

(Spyder maintainer here) That error message is caused by a bug in Spyder and it was fixed in our 5.0.1 version, released on April 16th 2020.

You can safely ignore it for now because it incorrectly reports that the right version of spyder-kernels is missing, when it's actually installed.

(Spyder maintainer here) You said

From random blogs and questions on StackOverflow I know that Spyder >= 4 requires Python >= 3

This is incorrect. Spyder 4.1.5 is compatible with Python 2.7. We drop support for Python 2.7 in our 4.2.0 version, released in November 2020.

and spyder-kernels at least 1.9 up (maybe lower, haven't tried all...)

Here you can find the list of spyder-kernels versions that are necessary for different Spyder ones. That needs to be updated for Spyder 5, but we will do that soon.

For Python 2.7 I can only go as far as Spyder 3, but I can't find the proper Spyer-kernels to install.

There's a misunderstanding here. You can still use Spyder 5 (which only supports Python 3) and run your Python 2 code in a different environment with the latest spyder-kernels, which still supports Python 2.7.

For that, first you need to run the following commands