grcuda | Polyglot CUDA integration for the GraalVM | GPU library
kandi X-RAY | grcuda Summary
kandi X-RAY | grcuda Summary
This Truffle language exposes GPUs to the polyglot GraalVM. The goal is to.
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of grcuda
grcuda Key Features
grcuda Examples and Code Snippets
Community Discussions
Trending Discussions on GPU
QUESTION
In my understanding, VkPhysicalDevice represents an implementation of Vulkan, which could be represented as a GPU and its drivers. We are supposed to record commands with VkCommandBuffers and send them through queues to, potentially, multithread the work we send to the gpu. That is why I understand the fact there can be multiple queues. I understand as well that QueueFamilies groups queues depending on the features they can do (the extensions available for them e.g. presentation, as well as graphics computations, transfer, etc).
However, if a GPU is able to do Graphics work, why are there queues unable to do so? I heard that using queues with less features could be faster, but why? What is a queue concretely? Is it only tied to vulkan implementation? Or is it related to hardware specific things?
I just don't understand why queues with different features exist, and even after searching through the Vulkan doc, StackOverflow, vulkan-tutorial and vkguide, the only thing I found was "Queues in Vulkan are an “execution port” for GPUs.", which I don't really understand and on which I can't find anything on google.
Thank you in advance for your help!
...ANSWER
Answered 2022-Apr-03 at 21:56A queue is a thing that consumes and executes commands, such that each queue (theoretically) executes separately from every other queue. You can think of a queue as a mouth, with commands as food.
Queues within a queue family typically execute commands using the same underlying hardware to process them. This would be like a creature with multiple mouths but all of them connect to the same digestive tract. How much food they can eat is separate from how much food they can digest. Food eaten by one mouth may have to wait for food previously eaten by another to pass through the digestive tract.
Queues from different families may (or may not) have distinct underlying execution hardware. This would be like a creature with multiple mouths and multiple digestive tracts. If a mouth eats, that food need not wait for food from a different mouth to digest.
Of course, distinct underlying execution hardware is typically distinct for a reason. Several GPUs have specialized DMA hardware for doing copies to/from device-local memory. Such hardware will typically expose a queue family that only allows transfer operations, and those transfer operations may be restricted in their byte alignment compared to transfers done on graphics-capable queues.
Note that these are general rules. Sometimes queues within a family do execute on different hardware, and sometimes queues between families use much of the same hardware. The API and implementations don't always make this clear, so you may have to benchmark different circumstances.
QUESTION
- Does OpenCL local memory really exist on Mali/Adreno GPU or they only exist in some special mobile phones?
- If they exist, in which case should we use local memory, such as GEMM/Conv or other cl kernel?
ANSWER
Answered 2022-Mar-24 at 15:27Interesting question. OpenCL defines a number of conceptual memories including local memory, constant memory, global memory, and private memory. And physically as you know, the hardware implementation of these memories is hardware dependent. For instance, some may emulate local memory using cache or system memory instead of having physical memory.
AFAIK, ARM Mali GPU does not have local memory, whereas Qualcomm Adreno GPU does have local memory.
For instance below table shows the definition of each memory in OpenCL and their relative latency and physical locations in Adreno GPU cited from OpenCL Optimization and Best Practices for Qualcomm Adreno GPUs∗
Answer updated:
as commented by SK-logic below, Mali6xx have a local memory (shared with cache).
QUESTION
I use JavaFX with Java 8 and i set this properties before launching my app
System.setProperty("prism.forceGPU","true");
System.setProperty("prism.order","d3d,sw");
The verbose mode for prism gives me this :
ANSWER
Answered 2022-Mar-09 at 05:23For those who are trying to solve a similar issue, it might be coming from the java.exe executable not using the gpu you want as a default device, you can change that in Windows' settings.
QUESTION
I'm trying to setup a Google Kubernetes Engine cluster with GPU's in the nodes loosely following these instructions, because I'm programmatically deploying using the Python client.
For some reason I can create a cluster with a NodePool that contains GPU's
...But, the nodes in the NodePool don't have access to those GPUs.
I've already installed the NVIDIA DaemonSet with this yaml file: https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/master/nvidia-driver-installer/cos/daemonset-preloaded.yaml
You can see that it's there in this image:
For some reason those 2 lines always seem to be in status "ContainerCreating" and "PodInitializing". They never flip green to status = "Running". How can I get the GPU's in the NodePool to become available in the node(s)?
Update:Based on comments I ran the following commands on the 2 NVIDIA pods; kubectl describe pod POD_NAME --namespace kube-system.
To do this I opened the UI KUBECTL command terminal on the node. Then I ran the following commands:
gcloud container clusters get-credentials CLUSTER-NAME --zone ZONE --project PROJECT-NAME
Then, I called kubectl describe pod nvidia-gpu-device-plugin-UID --namespace kube-system and got this output:
ANSWER
Answered 2022-Mar-03 at 08:30According the docker image that the container is trying to pull (gke-nvidia-installer:fixed), it looks like you're trying use Ubuntu daemonset instead of cos.
You should run kubectl apply -f https://raw.githubusercontent.com/GoogleCloudPlatform/container-engine-accelerators/master/nvidia-driver-installer/cos/daemonset-preloaded.yaml
This will apply the right daemonset for your cos node pool, as stated here.
In addition, please verify your node pool has the https://www.googleapis.com/auth/devstorage.read_only scope which is needed to pull the image. You can should see it in your node pool page in GCP Console, under Security -> Access scopes (The relevant service is Storage).
QUESTION
my computer has only 1 GPU.
Below is what I get the result by entering someone's code
...ANSWER
Answered 2021-Oct-12 at 08:52For the benefit of community providing solution here
This problem is because when keras run with gpu, it uses almost all
vram. So we needed to givememory_limitfor each notebook as shown below
QUESTION
I've run into an issue while attempting to use SSBOs as follows:
...ANSWER
Answered 2022-Feb-10 at 13:25GLSL structs and C++ structs have different rules on alignment. For structs, the spec states:
If the member is a structure, the base alignment of the structure is N, where N is the largest base alignment value of any of its members, and rounded up to the base alignment of a vec4. The individual members of this substructure are then assigned offsets by applying this set of rules recursively, where the base offset of the first member of the sub-structure is equal to the aligned offset of the structure. The structure may have padding at the end; the base offset of the member following the sub-structure is rounded up to the next multiple of the base alignment of the structure.
Let's analyze the struct:
QUESTION
Consider the following kernel, which reduces along the rows of a 2-D matrix
...ANSWER
Answered 2022-Jan-21 at 18:57Here is the code:
QUESTION
I'm working on Convolution Tasnet, model size I made is about 5.05 million variables.
I want to train this using custom training loops, and the problem is,
...ANSWER
Answered 2022-Jan-07 at 11:08Gradient tape triggers automatic differentiation which requires tracking gradients on all your weights and activations. Autodiff requires multiple more memory. This is normal. You'll have to manually tune your batch size until you find one that works, then tune your LR. Usually, the tune just means guess & check or grid search. (I am working on a product to do all of that for you but I'm not here to plug it).
QUESTION
I've installed Windows 10 21H2 on both my desktop (AMD 5950X system with RTX3080) and my laptop (Dell XPS 9560 with i7-7700HQ and GTX1050) following the instructions on https://docs.nvidia.com/cuda/wsl-user-guide/index.html:
- Install CUDA-capable driver in Windows
- Update WSL2 kernel in PowerShell:
wsl --update - Install CUDA toolkit in Ubuntu 20.04 in WSL2 (Note that you don't install a CUDA driver in WSL2, the instructions explicitly tell that the CUDA driver should not be installed.):
ANSWER
Answered 2021-Nov-18 at 19:20Turns out that Windows 10 Update Assistant incorrectly reported it upgraded my OS to 21H2 on my laptop.
Checking Windows version by running winver reports that my OS is still 21H1.
Of course CUDA in WSL2 will not work in Windows 10 without 21H2.
After successfully installing 21H2 I can confirm CUDA works with WSL2 even for laptops with Optimus NVIDIA cards.
QUESTION
I tried to train a model using PyTorch on my Macbook pro. It uses the new generation apple M1 CPU. However, PyTorch couldn't recognize my GPUs.
...ANSWER
Answered 2021-Nov-18 at 03:08It looks like PyTorch support for the M1 GPU is in the works, but is not yet complete.
From @soumith on GitHub:
So, here's an update. We plan to get the M1 GPU supported. @albanD, @ezyang and a few core-devs have been looking into it. I can't confirm/deny the involvement of any other folks right now.
So, what we have so far is that we had a prototype that was just about okay. We took the wrong approach (more graph-matching-ish), and the user-experience wasn't great -- some operations were really fast, some were really slow, there wasn't a smooth experience overall. One had to guess-work which of their workflows would be fast.
So, we're completely re-writing it using a new approach, which I think is a lot closer to your good ole PyTorch, but it is going to take some time. I don't think we're going to hit a public alpha in the next ~4 months.
We will open up development of this backend as soon as we can.
That post: https://github.com/pytorch/pytorch/issues/47702#issuecomment-965625139
TL;DR: a public beta is at least 4 months out.
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install grcuda
The original version of GrCUDA can be downloaded as a binary JAR from grcuda/releases and manually copied into a GraalVM installation.
Download GraalVM CE 21.2.0 for Linux graalvm-ce-java11-linux-amd64-21.2.0.tar.gz from GitHub and untar it in your installation directory.
Download the GrCUDA JAR from grcuda/releases. If using the official release, the latest features (e.g. the asynchronous scheduler) are not available. Instead, follow the guide below to install GrCUDA from the source code.
Test GrCUDA in Node.JS from GraalVM.
Download other GraalVM languages.
If you want to build GrCUDA yourself, instead of using an existing release, you will need a couple of extra steps. This section contains all the steps required to setup GrCUDA if your goal is to contribute to its development, or simply hack with it. For simplicity, let's assume that your installation is done in your home directory, ~. If you are installing GrCUDA on a new machine, you can simply follow or execute oci_setup/setup_from_scratch.sh. Here we repeat the same steps, with additional comments. The installation process has been validated with CUDA 11.4 and Ubuntu 20.04. The same oci_setup has a number of useful scripts to configure machines on OCI and easily use GrCUDA.
First, download GraalVM as above.
To build GrCUDA, you also need a custom JDK that is used to build GraalVM.
GrCUDA requires the mx build tool. Clone the mx repository and add the directory into $PATH, such that the mx can be invoked from the command line. We checkout the commit corresponding to the current GraalVM release.
You might also want the source files for GraalVM CE, at the commit corresponding to the current release of GraalVM. This is not required for building, but if you want to modify GrCUDA's source code, it is useful to also have access to GraalVM's code.
Last but not least, build GrCUDA
./install.sh does this sequence of command for you, if you need to rebuild GrCUDA.
To develop GrCUDA, you will greatly benefit from having an IDE that allows jumping between symbols and debugging individual tests. Here, we explain how to setup IntelliJ Idea.
mx ideinit from $GRCUDA_HOME, to setup the IDE
Open Idea and select "open project", then open GrCUDA
See this guide to configure the syntax checker File -> Settings -> Plugins -> Marketplace -> Search "Eclipse Code Formatter" and install it
In IntelliJ Idea, install the Python plugin with Settings -> Plugin -> Search "Python", then do Project Structure -> SDKs -> Create a new Python 3.8 Virtual Environment, it is used by mx
Select the right JVM. It should select automatically your $JAVA_HOME. Othewise, Project Structures -> Modules -> Set the Module SDK (under Dependencies) of mx and submodules to your Java SDK (e.g. 11). You can pick either the labsjdk or graalvm. This is also given by the configure option if you try to build the project in IntelliJ Idea before setting these options. Set your project Java SDK (e.g. 11) for those missing modules When building for the first time in Intellij Idea, you might get errors like cannot use --export for target 1.8, which means that some package is being build with Java 8. For these packages, there are two possible solutions. Try either of them, and stick to the one that works for you a. For those packages (look at the log to find them), manually specify a more recent SDK (e.g. 11) as you did in step above. If you get errors of missing symbols, follow IntelliJ's hints and export the requested packages b. Remove the exports. File -> Settings -> Build ... -> Compiler -> Java Compiler, then remove all the --export flags.
To run tests: a. Go to Run (top bar) -> Edit Configurations -> Edit configuration templates -> Junit b. (Not always necessary) By default, Idea should use your env. If not, make sure to have the same. Update the PATH variable so that it can find nvcc, and export $GRAAL_HOME. See setup_machine_from_scratch.sh to find all the environment variables. c. Modify the template Junit test configuration adding -Djava.library.path="$GRAAL_HOME/lib" (in Java 11) to the VM options to find trufflenfi d. In IntelliJ Idea, Run -> Edit Configurations. Create a new JUnit configuration set to All in package with com.nvidia.grcuda as module and com.nvidia.grcuda.test selected below. Add -Djava.library.path="$GRAAL_HOME/lib" (or your version of GraalVM) if it's not already in VM options. Specify the SDK by setting the GraalVM JRE in e.g. $GRAAL_HOME, if not specified already. e. If you change something in GrCUDA, rebuild it with ./install.sh before running tests. That's because tests that use the GPU load the .jar in $GRAAL_HOME, which is updated by ./install.sh
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
