Change default GPU in TensorFlow

Question

Based on the documentation, the default GPU is the one with the lowest id:

If you have more than one GPU in your system, the GPU with the lowest ID will be selected by default.

Is it possible to change this default from command line or one line of code?

Answer 1

Suever's answer correctly shows how to pin your operations to a particular GPU. However, if you are running multiple TensorFlow programs on the same machine, it is recommended that you set the CUDA_VISIBLE_DEVICES environment variable to expose different GPUs before starting the processes. Otherwise, TensorFlow will attempt to allocate almost the entire memory on all of the available GPUs, which prevents other processes from using those GPUs (even if the current process isn't using them).

Note that if you use CUDA_VISIBLE_DEVICES, the device names "/gpu:0", "/gpu:1", etc. refer to the 0th and 1st visible devices in the current process.

Answer 2

As is stated in the documentation, you can use tf.device('/gpu:id') to specify a device other than the default.

# This will use the second GPU on your system
with tf.device('/gpu:1'):
    a = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[2, 3], name='a')
    b = tf.constant([1.0, 2.0, 3.0, 4.0, 5.0, 6.0], shape=[3, 2], name='b')

c = tf.matmul(a, b)
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
print sess.run(c)

Answer 3

Just to be clear regarding the use of the environment variable CUDA_VISIBLE_DEVICES:

To run a script my_script.py on GPU 1 only, in the Linux terminal you can use the following command:

username@server:/scratch/coding/src$ CUDA_VISIBLE_DEVICES=1 python my_script.py 

More examples illustrating the syntax:

Environment Variable Syntax      Results
CUDA_VISIBLE_DEVICES=1           Only device 1 will be seen
CUDA_VISIBLE_DEVICES=0,1         Devices 0 and 1 will be visible
CUDA_VISIBLE_DEVICES="0,1"       Same as above, quotation marks are optional
CUDA_VISIBLE_DEVICES=0,2,3       Devices 0, 2, 3 will be visible; device 1 is masked
CUDA_VISIBLE_DEVICES=""          No GPU will be visible

FYI:

• CUDA Environment Variables
• Forcing TensorFlow-GPU to use the CPU from the command line

Answer 4

If you want to run your code on the second GPU,it assumes that your machine has two GPUs, You can do the following trick.

1. open Terminal

2. open tmux by typing tmux (you can install it by sudo apt-get install tmux)

3. run this line of code in tmux: CUDA_VISIBLE_DEVICES=1 python YourScript.py

Note: By default, tensorflow uses the first GPU, so with above trick, you can run your another code on the second GPU, separately.

Hope it would be helpful!!