Sentence-transformers: Model 'distiluse-base-multilingual-cased' couldn't be loaded if CUDA is not enabled

[serge-sotnyk]

I'm trying to work with the multilingual model - 'distiluse-base-multilingual-cased'. I need to load it on CPU-only devices. But I can't load it there because I've got an error:

RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False...

(the full message can be found in GIST with simple code for reproducing)

I am working in the Windows environment + Anaconda, Python 3.7. But the same behavior can be repeated in Colab with using CPU-only runtime.

Please, find minimal code for replication of this problem in gist https://gist.github.com/serge-sotnyk/bb031b622f2e9b27cb466897f7b61315. Here you could find the full error message with the stack trace.

If you uncomment the following line:

model = SentenceTransformer('bert-base-nli-mean-tokens')

and comment

model = SentenceTransformer('distiluse-base-multilingual-cased')

all works, but I need the multilingual model...

P.S.: It seems to me, this issue can be connected with https://github.com/explosion/spacy-transformers/issues/33.

[nreimers]

Hi @serge-sotnyk
It was fixed in the last commit:
https://github.com/UKPLab/sentence-transformers/commit/a96ccd3f2a6c031a33a929bbc5f110831f3a9f15

Thanks to pointing this out in issue #113

If you install it from sources (pip install -e .), you should then be able to use the model on a CPU only machine.

Best
Nils Reimers

[serge-sotnyk]

I've reinstalled with the following command:
(I leave it here for myself :) )

pip install -e git+git://github.com/UKPLab/sentence-transformers@a96ccd3#egg=sentence-transformers

CPU-only machine passed initialization and returned me embeddings.

Nils, many thanks for the quick fix!

[VinuAbraham]

I;m having the above issue. Tried both pip3 install and cloning from repo