Mask_rcnn: How to setting the GPU No. for training?

[ChienLiu]

I was trying to run the coco.py.
In the coco.py, I saw ''cococonfig'' to set GPU_count.
Now, I wanna ask for help about
""How to set some GPUs for training""

In the server of our research group, anyone of us has several fixed GPUs.
So, I need to set the GPU number for training to avoid using other's GPU.
What should I do?

Thanks for answering.

[Tay2510]

@ChienLiu

TensorFlow takes control of all the available GPUs. A workaround is to try the following code before you import Keras/TF. (For example, for the very beginning of coco.py). It re-writes the environment variables and makes only certain NVIDIA GPU(s) visible for that process.

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0,3"  # specify which GPU(s) to be used

It only uses GPU no. 0 and GPU no. 3 in this example

[ChienLiu]

@Tay2510
Thanks a lot.
Your code helps me sucessfully.
I only used caffe before, but now for Maskrcnn I tried to use tensorflow.
So, there are many things I have to learn. Thanks again.

[dongdongrj]

@ChienLiu
Hi, this project run ok with CPU computer?

[ChienLiu]

Yes.
If you want to train on cpu, you only need to install cpu-mode tensorflow (pip install tensorflow) instead of gpu-mode tensorflow (pip install tensorflow-gpu)

[wenchengxucool]

@ChienLiu

TensorFlow takes control of all the available GPUs. A workaround is to try the following code before you import Keras/TF. (For example, for the very beginning of coco.py). It re-writes the environment variables and makes only certain NVIDIA GPU(s) visible for that process.

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0,3"  # specify which GPU(s) to be used

It only uses GPU no. 0 and GPU no. 3 in this example

hello! if i use 'import tensorflow as tf' in 4 py files, do i really put the two lines of codes os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID" and os.environ["CUDA_VISIBLE_DEVICES"]="0,3" in each py file? Thanks~ 😄

[Tay2510]

@weichencoder
You don't. You just need to put it at the beginning of your program entry point (something like main.py for example). Namely before the first time you import tensorflow. Since it just overwrite the process environment variable, you can also use Linux export before running your python code:

export CUDA_DEVICE_ORDER="PCI_BUS_ID"
export CUDA_VISIBLE_DEVICES="0,3" 

[neonb88]

Yes.
If you want to train on cpu, you only need to install cpu-mode tensorflow (pip install tensorflow) instead of gpu-mode tensorflow (pip install tensorflow-gpu)

Should we delete these lines from our code (os.environ["CUDA_VISIBLE_DEVICES"]="0,3") if we're using CPU only?

[meechos]

TensorFlow takes control of all the available GPUs. A workaround is to try the following code before you import Keras/TF. (For example, for the very beginning of coco.py). It re-writes the environment variables and makes only certain NVIDIA GPU(s) visible for that process.
import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0,3" # specify which GPU(s) to be used

Just a heads up that, while the above solution works for tensorflow==2.0.0, it does not for more recent versions:

AttributeError: module 'tensorflow_core._api.v2.config' has no attribute 'experimental_list_devices')

[freedom9393]

@ChienLiu

TensorFlow takes control of all the available GPUs. A workaround is to try the following code before you import Keras/TF. (For example, for the very beginning of coco.py). It re-writes the environment variables and makes only certain NVIDIA GPU(s) visible for that process.

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0,3"  # specify which GPU(s) to be used

It only uses GPU no. 0 and GPU no. 3 in this example

Didn't work for me. Or doesn't it work for recent Tensorflow version?

[tienhoang1094]

@ChienLiu
TensorFlow takes control of all the available GPUs. A workaround is to try the following code before you import Keras/TF. (For example, for the very beginning of coco.py). It re-writes the environment variables and makes only certain NVIDIA GPU(s) visible for that process.

import os
os.environ["CUDA_DEVICE_ORDER"]="PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"]="0,3"  # specify which GPU(s) to be used

It only uses GPU no. 0 and GPU no. 3 in this example

Didn't work for me. Or doesn't it work for recent Tensorflow version?

try set ur environment var in bashrc
export CUDA_VISIBLE_DEVICES="0"