Sentence-transformers: getting Wrong shape for input_ids, while trying to replicate example
error
ValueError Traceback (most recent call last)
<ipython-input-8-1e9a71fa956b> in <module>
----> 1 all_common_phase_vec = model_sent.encode(all_common_phase)
/flucast/anaconda3/envs/e2/lib/python3.8/site-packages/sentence_transformers/SentenceTransformer.py in encode(self, sentences, batch_size, show_progress_bar, output_value, convert_to_numpy, convert_to_tensor, is_pretokenized)
185
186 with torch.no_grad():
--> 187 out_features = self.forward(features)
188 embeddings = out_features[output_value]
189
/flucast/anaconda3/envs/e2/lib/python3.8/site-packages/torch/nn/modules/container.py in forward(self, input)
115 def forward(self, input):
116 for module in self:
--> 117 input = module(input)
118 return input
119
/flucast/anaconda3/envs/e2/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),
/flucast/anaconda3/envs/e2/lib/python3.8/site-packages/sentence_transformers/models/RoBERTa.py in forward(self, features)
32 def forward(self, features):
33 """Returns token_embeddings, cls_token"""
---> 34 output_states = self.roberta(**features)
35 output_tokens = output_states[0]
36 cls_tokens = output_tokens[:, 0, :] # CLS token is first token
/flucast/anaconda3/envs/e2/lib/python3.8/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
720 result = self._slow_forward(*input, **kwargs)
721 else:
--> 722 result = self.forward(*input, **kwargs)
723 for hook in itertools.chain(
724 _global_forward_hooks.values(),
/flucast/anaconda3/envs/e2/lib/python3.8/site-packages/transformers/modeling_bert.py in forward(self, input_ids, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, output_attentions, output_hidden_states, return_dict)
802 # We can provide a self-attention mask of dimensions [batch_size, from_seq_length, to_seq_length]
803 # ourselves in which case we just need to make it broadcastable to all heads.
--> 804 extended_attention_mask: torch.Tensor = self.get_extended_attention_mask(attention_mask, input_shape, device)
805
806 # If a 2D ou 3D attention mask is provided for the cross-attention
/flucast/anaconda3/envs/e2/lib/python3.8/site-packages/transformers/modeling_utils.py in get_extended_attention_mask(self, attention_mask, input_shape, device)
258 extended_attention_mask = attention_mask[:, None, None, :]
259 else:
--> 260 raise ValueError(
261 "Wrong shape for input_ids (shape {}) or attention_mask (shape {})".format(
262 input_shape, attention_mask.shape
ValueError: Wrong shape for input_ids (shape torch.Size([40])) or attention_mask (shape torch.Size([40]))
- error is generated when using pertained model(model_name : roberta-large-nli-stsb-mean-tokens)
system running on
Python 3.8.5 (default, Aug 5 2020, 08:36:46)
[GCC 7.3.0] :: Anaconda, Inc. on linux
package installed in env
Package Version
--------------------- -------------------
argon2-cffi 20.1.0
attrs 20.1.0
backcall 0.2.0
bleach 3.1.5
blis 0.4.1
catalogue 1.0.0
certifi 2020.6.20
cffi 1.14.2
chardet 3.0.4
click 7.1.2
cymem 2.0.3
decorator 4.4.2
defusedxml 0.6.0
entrypoints 0.3
filelock 3.0.12
future 0.18.2
idna 2.10
ipykernel 5.3.4
ipython 7.17.0
ipython-genutils 0.2.0
jedi 0.17.2
Jinja2 2.11.2
joblib 0.16.0
json5 0.9.5
jsonschema 3.2.0
jupyter-client 6.1.7
jupyter-core 4.6.3
jupyterlab 2.2.6
jupyterlab-server 1.2.0
MarkupSafe 1.1.1
mistune 0.8.4
mkl-fft 1.1.0
mkl-random 1.1.1
mkl-service 2.3.0
murmurhash 1.0.2
nbconvert 5.6.1
nbformat 5.0.7
nltk 3.5
notebook 6.1.3
numpy 1.19.1
olefile 0.46
packaging 20.4
pandas 1.1.1
pandocfilters 1.4.2
parso 0.7.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 7.2.0
pip 20.2.2
plac 1.1.3
preshed 3.0.2
prometheus-client 0.8.0
prompt-toolkit 3.0.6
ptyprocess 0.6.0
pycparser 2.20
Pygments 2.6.1
pyparsing 2.4.7
pyrsistent 0.16.0
python-dateutil 2.8.1
pytz 2020.1
pyzmq 19.0.2
regex 2020.7.14
requests 2.24.0
sacremoses 0.0.43
scikit-learn 0.23.2
scipy 1.5.2
Send2Trash 1.5.0
sentence-transformers 0.3.3
sentencepiece 0.1.91
setuptools 49.6.0.post20200814
six 1.15.0
spacy 2.3.2
srsly 1.0.2
terminado 0.8.3
testpath 0.4.4
thinc 7.4.1
threadpoolctl 2.1.0
tokenizers 0.8.1rc2
torch 1.6.0
torchvision 0.7.0
tornado 6.0.4
tqdm 4.48.2
traitlets 4.3.3
transformers 3.0.2
urllib3 1.25.10
wasabi 0.8.0
wcwidth 0.2.5
webencodings 0.5.1
wheel 0.35.1
xlrd 1.2.0
amiyamandal-dev
All 13 comments
How does your code look like that produces this error?
nreimers
on 28 Aug 2020
code
model_sent = SentenceTransformer('roberta-large-nli-stsb-mean-tokens')
with open("result/_t_phase.txt", "r") as f:
all_common_phase = f.read().split("\n")
all_common_phase_vec = model_sent.encode(all_common_phase)
include all the phase which i want to encode are all in _t_phase.txt
amiyamandal-dev
on 28 Aug 2020
Does this example work for you?
https://github.com/UKPLab/sentence-transformers/blob/master/examples/applications/computing_embeddings.py
nreimers
on 28 Aug 2020
still facing error but its different, now IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
https://gist.github.com/amiyamandal-dev/6d909f3bc5be5dbc103e88532aff30c1
amiyamandal-dev
on 28 Aug 2020
Hi @amiyamandal-dev
Your error message indicates that you are using transformers from the master branch. However, the master branch is under active development and is not necessarily a working version of transformers. And as it appears, the current master branch of transformers is not compatible with sentence-transformers.
I can recommend to always use stable releases of packages. Hence, if you use the latest release of transformers (v3.0.2), everything should work fine.
nreimers
on 28 Aug 2020
still facing error but its different, now IndexError: Dimension out of range (expected to be in range of [-1, 0], but got 1)
https://gist.github.com/amiyamandal-dev/6d909f3bc5be5dbc103e88532aff30c1
Did you install transformer from source? I concurred the same error a few days ago. #377
WangHexie
on 28 Aug 2020
yes i have installed transformer from source.... any work around for that
amiyamandal-dev
on 28 Aug 2020
Install it with pip ;)
pip install transformers
I am also getting the same error on running the code:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer('distilbert-base-nli-mean-tokens')
sentences = ['This framework generates embeddings for each input sentence',
'Sentences are passed as a list of string.',
'The quick brown fox jumps over the lazy dog.']
sentence_embeddings = model.encode(sentences)
I have installed transformers with pip only. Earlier it used to work fine, finding this issue since yesterday.
subhamiitk
on 31 Aug 2020
Hi @subham1
Can you try to re-install ensuring that transformers v 3.0.2 is installed via pip?
nreimers
on 31 Aug 2020
Yes, it works when you downgrade the transformers to version 3.0.2 from 3.1.0.
nandinib1999
on 1 Sep 2020
I had the same problem and downgrade the version worked for me. Thank you a lot <3
natalypatti
on 11 Sep 2020
Yes, it works when you downgrade the transformers to version 3.0.2 from 3.1.0.
I ran into a similar issue and downgrading transformers solved it.
VictorZuanazzi
on 5 Nov 2020
Related issues
AMChierici
路
5Comments
PhilipMay
路
5Comments
naserahmadi
路
3Comments
anatoly-khomenko
路
5Comments
RuskinManku
路
5Comments
Most helpful comment
Yes, it works when you downgrade the transformers to version 3.0.2 from 3.1.0.