Spacy: "cudaErrorInvalidDevice: invalid device ordinal" when attempting to pretrain

[fcggamou]

How to reproduce the behaviour

When trying to pretrain a model using the CLI pretrain command, I'm getting the following error:
cupy.cuda.runtime.CUDARuntimeError: cudaErrorInvalidDevice: invalid device ordinal

The command I'm running is:
!python -m spacy pretrain "train.jsonl" 'es_core_news_md' 'pre_trained' -i=1

I think it's totally unrelated to my data.

Your Environment

I'm working on Google Colab.

• spaCy version: 2.3.2
• Platform: Linux-4.19.104+-x86_64-with-Ubuntu-18.04-bionic
• Python version: 3.6.9

[adrianeboyd]

Thanks for the report! I think this accidentally got hard-coded in some of the experimental updates for v2.3.2. We'll fix it for the next patch release of spacy v2.3.

In the meanwhile, you can find this line in spacy/cli/pretrain.py and edit it to take out gpu_id=1 or provide the right ID:

https://github.com/explosion/spaCy/blob/6d4d5c074c4add57e7643c531109d8c34f7c8b5f/spacy/cli/pretrain.py#L128

[1danjordan]

For others stuck on this, until the v2.3.3 patch is released, this solved my problem:

pip install -U spacy==2.3.1