CUDA Pro Tip: Control GPU Visibility with CUDA_VISIBLE_DEVICES

post on the Acceleware blogAs a CUDA developer, you will often need to control which devices your application uses. In a short-but-sweet post on the Acceleware blog, Chris Mason writes:

Acceleware LogoDoes your CUDA application need to target a specific GPU? If you are writing GPU enabled code, you would typically use a device query to select the desired GPUs. However, a quick and easy solution for testing is to use the environment variable CUDA_VISIBLE_DEVICES to restrict the devices that your CUDA application sees. This can be useful if you are attempting to share resources on a node or you want your GPU enabled executable to target a specific GPU

As Chris points out, robust applications should use the CUDA API to enumerate and select devices with appropriate capabilities at run time. To learn how, read the section on Device Enumeration in the CUDA Programming Guide. But the CUDA_VISIBLE_DEVICES environment variable is handy for restricting execution to a specific device or set of devices for debugging and testing.  You can also use it to control execution of applications for which you don’t have source code, or to launch multiple instances of a program on a single machine, each with its own environment and set of visible devices.

To use it, set CUDA_VISIBLE_DEVICES to a comma-separated list of device IDs to make only those devices visible to the application. Chris Mason’s post gives full details. Note that you can use this technique both to mask out devices or to change the visibility order of devices so that the CUDA runtime enumerates them in a specific order.

There is a specific case where CUDA_VISIBLE_DEVICES is useful in our upcoming CUDA 6 release with Unified Memory (see my post on Unified Memory). Unified Memory enables multiple GPUs and CPUs to share a single, managed memory space. Unified Memory between GPUs requires that the GPUs all support peer-to-peer memory access, but this is not the case in some systems where the GPUs are different models, or they connect to different I/O controller hubs on the PCI-express bus. If the GPUs are not all P2P compatible, then allocations with cudaMallocManaged() fall back to device-mapped host memory (also known as “zero copy” memory). Access to this memory is via PCI-express and has much lower bandwidth and higher latency. To avoid this fallback, you can use CUDA_VISIBLE_DEVICES to limit your application to run on a single device or on a set of devices that are P2P compatible.

For more great tips and CUDA training, check out the Acceleware blog.


About Mark Harris

Mark is Chief Technologist for GPU Computing Software at NVIDIA. Mark has fifteen years of experience developing software for GPUs, ranging from graphics and games, to physically-based simulation, to parallel algorithms and high-performance computing. Mark has been using GPUs for general-purpose computing since before they even supported floating point arithmetic. While a Ph.D. student at UNC he recognized this nascent trend and coined a name for it: GPGPU (General-Purpose computing on Graphics Processing Units), and started to provide a forum for those working in the field to share and discuss their work. Follow @harrism on Twitter
  • alex

    Hi, does the CUDA_VISIBLE_DEVICE permits to be sure to activate the desired GPU for a specific calculation if OPTIMUS technology is present?