Ray: How to init Ray with a specified GPU id to run all trials of Tune?

[guoxuxu]

What is your question?

Say, I have 4 GPUs with ids=[0, 1, 2, 3] and I only want to run all trials for Tune on id=2 and id=3 only. That means I can only maximize the use of the third and fourth GPU without touching the first two GPUs. How can I achieve this?

ray.init(num_cpus=num_cpus, num_gpus=num_gpus, temp_dir=ray_log)

The attribute num_gpus is the number of GPUs ray can use. When setting num_gpus=1, all the trials run on the first device (GPU id=0). When increasing num_gpus, all the trials will ordinally use GPUs from id=0 to id=3... I want to know how to specify the exact GPU ids, e.g., all trials run on id=2 and id=3.

I've tried specifying GPU id in the training functions, but raised RuntimeError: CUDA error: invalid device ordinal.

I'm still new to this great project. Appreciate your warm help!

Ray version and other system information (Python version, TensorFlow version, OS):
OS: Linux
Python: 3.7.4
Ray: 0.8.6

[richardliaw]

Try setting the CUDA_VISIBLE_DEVICES before running the ray script?

[guoxuxu]

Soga. It works now. Thanks very much!