Transformers: IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

[guhur]

A simple call to BertModel does not work well here.
Here is a minimal code example:

from pytorch_pretrained_bert.modeling import BertModel
from pytorch_pretrained_bert.tokenization import BertTokenizer
import torch

embed = BertModel.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

sentence = "the red cube is at your left"
tokens = ["[CLS]"] + tokenizer.tokenize(sentence) + ["[SEP]"] 
input_ids = torch.tensor(tokenizer.convert_tokens_to_ids(tokens))

print(input_ids)
embed(input_ids)

I obtained the following error:

tensor([ 101, 1996, 2417, 14291, 2003, 2012, 2115, 2187, 102])

IndexError Traceback (most recent call last)
in
11
12 print(input_ids)
---> 13 embed(input_ids)

~/.pyenv/versions/3.6.7/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, input, *kwargs)
491 result = self._slow_forward(input, *kwargs)
492 else:
--> 493 result = self.forward(input, *kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)

~/src/pytorch-pretrained-BERT/pytorch_pretrained_bert/modeling.py in forward(self, input_ids, token_type_ids, attention_mask, output_all_encoded_layers)
731 extended_attention_mask = (1.0 - extended_attention_mask) * -10000.0
732
--> 733 embedding_output = self.embeddings(input_ids, token_type_ids)
734 encoded_layers = self.encoder(embedding_output,
735 extended_attention_mask,

~/.pyenv/versions/3.6.7/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, input, *kwargs)
491 result = self._slow_forward(input, *kwargs)
492 else:
--> 493 result = self.forward(input, *kwargs)
494 for hook in self._forward_hooks.values():
495 hook_result = hook(self, input, result)

~/src/pytorch-pretrained-BERT/pytorch_pretrained_bert/modeling.py in forward(self, input_ids, token_type_ids)
262
263 def forward(self, input_ids, token_type_ids=None):
--> 264 seq_length = input_ids.size(1)
265 position_ids = torch.arange(seq_length, dtype=torch.long, device=input_ids.device)
266 position_ids = position_ids.unsqueeze(0).expand_as(input_ids)

IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)

I am using Python 3.6.7 and the code on the master branch.

[guhur]

Oops... I forgot to "batch" the input...

Here is a working sample:

from pytorch_pretrained_bert.modeling import BertModel
from pytorch_pretrained_bert.tokenization import BertTokenizer
import torch

embed = BertModel.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

sentence = "the red cube is at your left"
tokens = ["[CLS]"] + tokenizer.tokenize(sentence) + ["[SEP]"] 
input_ids = torch.tensor(tokenizer.convert_tokens_to_ids(tokens))

embed(input_ids.unsqueeze(0))

[CaryDai]

Oops... I forgot to "batch" the input...

Here is a working sample:

from pytorch_pretrained_bert.modeling import BertModel
from pytorch_pretrained_bert.tokenization import BertTokenizer
import torch

embed = BertModel.from_pretrained('bert-base-uncased')
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')

sentence = "the red cube is at your left"
tokens = ["[CLS]"] + tokenizer.tokenize(sentence) + ["[SEP]"] 
input_ids = torch.tensor(tokenizer.convert_tokens_to_ids(tokens))

embed(input_ids.unsqueeze(0))

Thank u so much, u helped me a lot.