Flair: How to specify CPU or GPU?

Created on 6 Feb 2019 · 5Comments · Source: flairNLP/flair

I'd like to know how to move the model between CPU and GPU. Alternatively, I'd like to know if it's possible to specify the device before instantiating a model.

question

Source

romanovzky

Most helpful comment

Hello @romanovzky the default behavior is that the model gets put on GPU if available and runs on CPU if there is no GPU.

This is done in the __init__.py, here:

device = None
if torch.cuda.is_available():
    device = torch.device('cuda:0')
else:
    device = torch.device('cpu')

The flair.device parameter gets called all over the code to move models and tensor to the device on which flair is run.

So if you would like to explicitly change this behavior, for instance to direct it to run on CPU even if you have a GPU available, you need to run this code before instantiating your model:

import flair, torch
flair.device = torch.device('cpu')

This overwrites the flair.device with your desired destination.

Hope this clarifies!

alanakbik on 6 Feb 2019

👍7

All 5 comments

Hello @romanovzky the default behavior is that the model gets put on GPU if available and runs on CPU if there is no GPU.

This is done in the __init__.py, here:

device = None
if torch.cuda.is_available():
    device = torch.device('cuda:0')
else:
    device = torch.device('cpu')

The flair.device parameter gets called all over the code to move models and tensor to the device on which flair is run.

So if you would like to explicitly change this behavior, for instance to direct it to run on CPU even if you have a GPU available, you need to run this code before instantiating your model:

import flair, torch
flair.device = torch.device('cpu')

This overwrites the flair.device with your desired destination.

Hope this clarifies!

alanakbik on 6 Feb 2019

👍7

Closing since question is answered (hopefully) - feel free to reopen if you have other questions.

alanakbik on 13 Feb 2019

Hi Alan,

I'm running the language model trainer on a server with 2 GPUs. Both GPUs are available. It works on GPU 0, however, when I try to change the device to GPU 1, I face the following error:

Traceback (most recent call last):
File "train_LM_flair.py", line 42, in
trainer.train('/home/tobdau/data/language_model1', learning_rate= 20, mini_batch_size=100, sequence_length=250, patience=20, max_epochs=20, anneal_factor=0.999)
File "/home/tobdau/.local/lib/python3.6/site-packages/flair/trainers/language_model_trainer.py", line 336, in train
log.info(self.model.generate_text())
File "/home/tobdau/.local/lib/python3.6/site-packages/flair/models/language_model.py", line 259, in generate_text
prediction, _, hidden = self.forward(input, hidden)
File "/home/tobdau/.local/lib/python3.6/site-packages/flair/models/language_model.py", line 71, in forward
encoded = self.encoder(input)
File "/home/tobdau/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in __call__
result = self.forward(input, *kwargs)
File "/home/tobdau/.local/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 118, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "/home/tobdau/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 1454, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: arguments are located on different GPUs at /pytorch/aten/src/THC/generic/THCTensorIndex.cu:519