pytree | Tree Notation Python Library | Dataset library

I made a simple script to try to do gradient accumulation with JAX. The idea is to have large batch size (e.g. 64) that are split in small chunks (e.g. 4) that fit in the GPU's memory. For each chunck, the resulting gradient, stored in a pytree, is added to the current batch gradient. The update is done only when all chunks of the large batch are computed. In this particular example, we simply try to fit random 512-dimensional vectors to random booleans with a linear layer. Here is the script:

In order to understand JAX's reverse mode auto-diff I tried to write a custom_vjp for softmax like this:

I am trying to use PyDev to attach to a process on MS-Windows 10. Actually, to be more precise I was doing this. It worked wonderfully and I value it immensely, but now doesn't work and I wonder why.

I always do this to the same process, it is one written in C++ that loads a python interpreter internally to run Python plugin code. I have in past been happily breaking inside the plugin code and debugging with PyDev.

Come Dec 2020 and I try again and I get this error when trying to attach to the same process:

I am working on a project in pycharm which is in a virtual environment on Mac OS. I have been trying to install the SHAP package unfortunately it is not working out. Below is the full error message. There seems to be a problem with an SDK that does not seem to exist, but when I install the package outside of the virtual environment I do not encounter any problem.