Infersent: embedding on GPU

[dami23]

The demo.ipynb and extract_feature.py in dir encoder/ are run on cpu. Could you give me a cue to run the two codes on GPU? Thank you for your help!

[aconneau]

Hi,
could you try
model = model.cuda()

in cell 2?
Thanks,
Alexis

[dami23]

@aconneau thank you for your reply, I tried model = model.cuda(), the error: Expected object of type torch.cuda.FloatTensor but found type torch.FloatTensor for argument #4 'mat1' from line
"embeddings = model.encode(sentences, bsize=128, tokenize=False, verbose=True)".
the problem is the sentences also need to be transformed to a tensor, but from_numpy() can't convert type numpy.str_ to a tensor. So, could you give me a solution about this problem? Sincerely thank you for your kindness help!

[aconneau]

I tried on my side and it worked so I wasn't able to reproduce the issue. Could you output the full error message?

Which pytorch version are you using?

[dami23]

File "/informatik2/tams/home/mi/lieber_mi/projects/object_affordance/code/InferSent/demo.py", line 36, in
embeddings = model.encode(sentences, bsize=128, tokenize=False, verbose=True)
File "/informatik2/tams/home/mi/lieber_mi/projects/object_affordance/code/InferSent/models.py", line 226, in encode
(batch, lengths[stidx:stidx + bsize])).data.numpy()
File "/informatik2/tams/home/mi/lieber_mi/projects/object_affordance/code/InferSent/models.py", line 68, in forward
sent_output = self.enc_lstm(sent_packed)[0] # seqlen x batch x 2nhid
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 491, in __call__
result = self.forward(input, *kwargs)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/rnn.py", line 192, in forward
output, hidden = func(input, self.all_weights, hx, batch_sizes)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 323, in forward
return func(input, *fargs, *fkwargs)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 243, in forward
nexth, output = func(input, hidden, weight, batch_sizes)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 86, in forward
hy, output = inner(input, hidden[l], weight[l], batch_sizes)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 158, in forward
hidden = inner(step_input, hidden, *weight)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/_functions/rnn.py", line 33, in LSTMCell
gates = F.linear(input, w_ih, b_ih) + F.linear(hx, w_hh, b_hh)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/functional.py", line 992, in linear
return torch.addmm(bias, input, weight.t())
RuntimeError: Expected object of type torch.cuda.FloatTensor but found type torch.FloatTensor for argument #4 'mat1'.

This is the full error. I use python3.5 and pytorch 0.4.0.

[aconneau]

Hi,
I think I understand where the error comes from.
In https://github.com/facebookresearch/InferSent/blob/master/models.py#L51 I check whether the model has been put on cuda or not. This might not work anymore since the new version of pytorch. Could you modify this line and print
print(type(self.enc_lstm.bias_hh_l0.data))
right before line 51?

Thanks

[dami23]

the output is . It is true, the model has not been put on cuda. I tried on pytorch 0.5.0, the error is same. Thank you for your help!

[dami23]

Maybe I find an "answer". I modified the line 51 in model.py to return 'cuda' (the latter part "in str(type(self.enc_lstm.bias_hh_l0.data))" is commented). And now, the embedding can run on GPU.

[aconneau]

So in the new pytorch version (>=0.4), the type of a Tensor which is on cuda is not "torch.cudaTensor" anymore but still "torch.Tensor", hence the error.

To fix the issue, could you modify the line : https://github.com/facebookresearch/InferSent/blob/master/models.py#L51
by
return self.enc_lstm.bias_hh_l0.data.is_cuda
and see if it works?
Thanks

[dami23]

@aconneau your solution is the correct answer, it worked. Thank you for reply and help, best regards!!