cuda-samples | cuda programs for senior project | GPU library

Here is a drawing to understand the values set to the first variables of the CUDA kernel and the overall computation performed:

Matrices are stored using a row-major ordering. The CUDA code assume the matrix sizes can be divided by BLOCK_SIZE.

The matrices A, B and C are virtually split in blocks according to the kernel CUDA grid. All blocks of C can be computed in parallel. For a given dark-grey block of C, the main loop walk through the several light-grey blocks of A and B (in lockstep). Each block is computed in parallel using BLOCK_SIZE * BLOCK_SIZE threads.

bx and by are the block-based position of the current block within the CUDA grid. tx and ty are the cell-based position of the cell computed by the current thread within the current computed block of the CUDA grid.

Here is a detailed analysis for the aBegin variable: aBegin refers to the memory location of the first cell of the first computed block of the matrix A. It is set to wA * BLOCK_SIZE * by because each block contains BLOCK_SIZE * BLOCK_SIZE cells and there is wA / BLOCK_SIZE blocks horizontally and by blocks above the current computed block of A. Thus, (BLOCK_SIZE * BLOCK_SIZE) * (wA / BLOCK_SIZE) * by = BLOCK_SIZE * wA * by.

The same logic apply for bBegin: it is set to BLOCK_SIZE * bx because there is bx block of size BLOCK_SIZE in memory before the first cell of the first computed block of the matrix B.

a is incremented by aStep = BLOCK_SIZE in the loop so that the next computed block is the following on the right (on the drawing) of the current computed block of A. b is incremented by bStep = BLOCK_SIZE * wB in the same loop so that the next computed block is the following of the bottom (on the drawing) of the current computed block of B.

This might be more of an extended comment, bet hear me out ...

As pointed out in the comments CUDA Samples are not meant as performance measuring tools. The second benchmark you provided does not actually use tensor cores, but just a normal instruction executed on FP32 or FP64 cores.

The CUDA compiler must compile for a GPU target (i.e. a device architecture). If you don't specify a target architecture on the compile command line, historically, CUDA has chosen a very flexible default architecture specification that can run on all GPUs that the CUDA version supports.

That isn't always the case, however, and its not the case with CUDA 11. CUDA 11 compiles for a default architecture of sm_52 (i.e. as if you had specified -arch=sm_52 on the command line). But CUDA 11 supports architectures down to sm_35.

Therefore if you don't specify the target architecture on the compile command line with CUDA 11, and attempt to run on a GPU with an architecture that predates sm_52, any CUDA code (kernels) that you have written definitely won't work.

It's good practice, any time you are having trouble with a CUDA code, to use proper CUDA error checking, and if you had done that here you would have gotten a runtime error indication that would have immediately identified the issue (at least for someone who is familiar with CUDA errors).

The solution in these cases is to specify a compilation command that includes the GPU you intend to run on (this is usually good practice anyway). If you do that, and the architecture you specify is "deprecated", then the nvcc compiler will issue a warning letting you know that a future CUDA version may not support the GPU you are trying to run on. The warning does not mean anything you are doing is wrong or illegal or needs to be changed, but it means that in the future, a future CUDA version may not support that GPU.

If you want to suppress that warning, you can pass the -Wno-deprecated-gpu-targets switch on the compile command line.

The gcc recipes is located in

From Could not dlopen library 'libcudart.so.10.0'; we can get that you tensorflow package is built against CUDA 10.0. You should install CUDA 10.0 or build it from source (against CUDA 10.1 or 10.2) by yourself.

The .csv that are included in the rootfs from the NVIDIA SDK-manager contains specific lib/dir/sym that are needed for the passing of GPU access to the container. The files that are listed in the .csv files are merged into the container and allows access to these files. What specific files are needed, depends on what is needed in the container.

It is of course very important that the actual paths to the files listed in the csv files are the same on the host, otherwise the merge will fail. These paths are not the correct paths on the default Yocto setup as they are made for the default NVIDIA SDK-manager image rootfs setup and thus needs to be corrected.

Once corrected, the access to GPU acceleration in the container should be possible and can be confirmed by doing a deviceQuery test.

I debugged it. The problem is version mismatch between nvidia-driver(410.93) and cuda(with driver 410.48 came with cuda run file). Gave autoremove all the drivers and reinstalled from the beginning. Deleted all the link files in /var/lib/dkms/nvidia/*. Now it works fine. And nvidia-uvm also loaded.

lsmod | grep nvidia

After trying out different versions of the nvidia driver, I came to the conclusion, that there is a problem with version 384.

I deleted all nvidia driver related stuff via sudo apt-get purge nvidia. and installed version 390:

I just got this to work by symlinking /usr/bin/gcc-5 to /usr/local/cuda-8.0/bin/gcc and /usr/bin/g++-5 to /usr/local/cuda-8.0/bin/g++: