cnmem | simple memory manager for CUDA | GPU library

In windows pathe is given by back slash \ instead of forward slash / which is used in linux/unix.

Try it like blow if file is 1 folder back:

My initial candidates would be a mix of:

• highly efficient use of available CPU-cores' L1-/ L2- sizes within the fastest [ns]-distances
• smart numpy vectorised execution being friendly to CPU cache-lines
• dataset so small, it can completely remain non-evicted from cache ( test to scale the dataset-under-review way above the L2-/L3-cache sizes to see the DDRx-memory-cost effects on the observed performance ( details are in the URL below ) )
• might enjoy even better timing on numpy, if avoiding .astype() conversions ( test it )

• auto-generated GPU-kernels do not have much chance to get ultimate levels of global memory latency-masking, compared to manually tweaked kernel-designs, tailor fit to respective GPU-silicon-architecture / latencies observed in-vivo
• data-structures larger than just a few KB remain paying GPU-SM/GDDR-MEM distances of ~ large hundreds of [ns], nearly [us] -v/s- compared to small units ~ small tens of [ns] at CPU/L1/L2/L3/DDRx ) ref. timing details in >>> https://stackoverflow.com/a/33065382
• not being able to enjoy much of the GPU/SMX power, due to this task's obvious low-reuse of data points and dataset size beyond the GPU/SM-silicon limits, that causes and must cause GPU/SM-register capacity spillovers in any kind of GPU-kernel design attempts and tweaking
• the global task is not having a minimum reasonable amount of asynchronous, isolated ( non-communicating islands ) mathematically-dense, yet SMX-local, GPU-kernel processing steps ( there is not much to compute so as to adjust for the add-on overheads and expensive SMX/GDDR memory costs )

GPU-s can lovely exhibit it's best-performance, if sufficiently enough densely-convoluted re-processing operations take place -- like in large-scale/high-resolution image-processing -- on [m,n,o]-convolution-kernel matrices so small, so as that all these m*n*o constant values can reside local to SM, inside an available set of SMX-SM_registers and if the GPU-kernel-launchers are optimally tweaked by the 3D-tblock/grid processing-layout geometries, so that the global memory access latencies are at its best-masked performance, having all the GPU-threads enforced within the hardware WARP-aligned SMx:WarpScheduler RoundRobin thread-scheduling capabilites ( the first swap from Round-Robin into Greedy-WarpSchedule mode loses the whole battle in case of divergent execution-paths in GPU-kernel-code ).

Try to reinstall Theano and Lasagne like this:

According to the github libgpuarray issues :

The last two tests require nccl and will fail if it's not present. If you're not trying to use nccl, you can ignore those failures.

—

https://github.com/Theano/libgpuarray/issues/383#issuecomment-287491789

Installing both tensorflow and tensorflow-gpu on the same machine might cause issues at the moment.

Install either tensorflow (for cpu only) or tensorflow-gpu (for gpu only) for version 1.0

No. Tensorflow has its own GPU memory management. Indeed it takes upfront the whole GPU memory regardless of the size of your problem.

Comment the line from keras.backend.theano_backend import _on_gpu and define _on_gpu as:

1. This sounds like a problem of convergence or model construction, not related to njobs or parallelism. Without the model or traces there is not a lot that can be said here.

GPU is still experimental and we've seen speed-ups for some models and slow-downs for others. ADVI seems to be easier to run on the GPU, though. You can also check that all your model types and input data are float32.