Spacy: Model installation problem and wrong error message

[randomgambit]

Hello there,

Thanks for this wonderful package. I am using Anaconda on a windows 64 machine, and I have a hard time installing it and using it. I ran

conda config --add channels conda-forge
conda install spacy

and I also ran python -m spacy.en.download all

Now, if I use the two-liner available on your website, I get an error

import spacy                         # See "Installing spaCy"
nlp = spacy.load('en')               # You are here.
doc = nlp(u'Hello, spacy!')          # See "Using the pipeline"
print((w.text, w.pos_) for w in doc) 

<generator object <genexpr> at 0x000000000F162678>

and

sentences = [s for s in doc.sents]
print(len(sentences))

gives

ValueError: sentence boundary detection requires the dependency parse, which requires data to be installed. If you haven't done so, run: 
python -m spacy.en.download all
to install the data

Any ideas?
Thanks!!

[ines]

Hey – sorry, I think you've accidentally installed the old version v1.6 that doesn't yet use the new download commands. The latest version 1.7+ is not on conda yet (see #923), but we're working on that.

In the meantime, you can always use pip to install the latest version:

pip install -U spacy

[ines]

I just checked again and I misread your issue, sorry. Turns out I got confused by the conda badge that still lists version 1.6. The new version should be up and you've probably downloaded it correctly. You can make sure it downloads the latest version using this command:

conda install -c conda-forge spacy=1.7.3

Your problem seems to be that you've used the old download commands to download the models. As of 1.7+, models can be downloaded like this:

python -m spacy download en
python -m spacy download de

For more info on this, see the models documentation.

Edit: I also just noticed that the error message still lists the wrong download command. This is bad, sorry – will fix this!

[randomgambit]

Hello thanks for the answer.
I just did both

conda install -c conda-forge spacy=1.7.3

and

python -m spacy download en
python -m spacy download de

it says linking successful but after restarting Spyder I still get the above errors. Is there anything else to do to make it work?

Thanks

[rmax]

Could you show us the output of python -m spacy info in the console? And inside spyder, the output of the following code:

from spacy.cli import info
info()

[randomgambit]

yup!

D:\anaconda>conda install -c conda-forge spacy=1.7.3
Fetching package metadata ...........
Solving package specifications: ..........

# All requested packages already installed.
# packages in environment at D:\anaconda:
#
spacy                     1.7.3           np111py27_vc9_0  [vc9]  conda-forge

D:\anaconda>python -m spacy download en

    Downloading en_core_web_sm-1.2.0/en_core_web_sm-1.2.0.tar.gz

Collecting https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-1.2.0/en_core_web_sm-1.2.0.tar.gz
  Downloading https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-1.2.0/en_core_web_sm-1.2.0.tar.gz (52.2MB)
    100% |################################| 52.2MB 6.4MB/s
  Requirement already satisfied (use --upgrade to upgrade): en-core-web-sm==1.2.0 from https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-1.2.0/en_core_web_sm-1.2.0.tar.gz in d:\anaconda\lib\site-packages
Requirement already satisfied: spacy<2.0.0,>=1.7.0 in d:\anaconda\lib\site-packages (from en-core-web-sm==1.2.0)
Requirement already satisfied: numpy>=1.7 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: murmurhash<0.27,>=0.26 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: cymem<1.32,>=1.30 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: preshed<2.0.0,>=1.0.0 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: thinc<6.6.0,>=6.5.0 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: plac<1.0.0,>=0.9.6 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: six in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: pathlib in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: ujson>=1.35 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: dill<0.3,>=0.2 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: requests<3.0.0,>=2.13.0 in d:\anaconda\lib\site-packages (from spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: wrapt in d:\anaconda\lib\site-packages (from thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: tqdm<5.0.0,>=4.10.0 in d:\anaconda\lib\site-packages (from thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: cytoolz<0.9,>=0.8 in d:\anaconda\lib\site-packages (from thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Requirement already satisfied: termcolor in d:\anaconda\lib\site-packages (from thinc<6.6.0,>=6.5.0->spacy<2.0.0,>=1.7.0->en-core-web-sm==1.2.0)
Invalid switch - "anaconda".

    Linking successful

    D:/anaconda/lib/site-packages/en_core_web_sm/en_core_web_sm-1.2.0 -->
    D:/anaconda/lib/site-packages/spacy/data/en

    You can now load the model via spacy.load('en').

    Linking successful

    D:/anaconda/lib/site-packages/de_core_news_md/de_core_news_md-1.0.0 -->
    D:/anaconda/lib/site-packages/spacy/data/de

    You can now load the model via spacy.load('de').

    Info about spaCy

    Python version     2.7.13
    Platform           Windows-10-10.0.14393
    spaCy version      1.7.3
    Installed models
    Location           D:\anaconda\lib\site-packages\spacy

from spacy.cli import info
info()


    Info about spaCy

    Python version     2.7.13         
    Platform           Windows-10-10.0.14393
    spaCy version      1.7.3          
    Installed models                   
    Location           D:\anaconda\lib\site-packages\spacy

[abbottLane]

I'm seeing the exact same issue as @randomgambit . A workaround that I've found is to import the model like this:

import spacy
import en_core_web_sm

nlp = en_core_web_sm.load()
r = nlp(u'This is a sentence. This is another one. And this is a third, for good measure.' )
for s in r.sents:
    print(s)

-----------------------------------output----------------------
>>>This is a sentence.
>>>This is another one.
>>>And this is a third, for good measure.

[randomgambit]

workaround works for me!

[ines]

This sounds a lot like the problem described in #934. Especially since you're on Windows + Python 2, and also got the Invalid switch message.

For Windows and Python 2, spaCy uses mlink to create the symlinks, and the paths were being passed in as strings, not Windows paths, which for some reason fails silently and still displays the success message.

The fix is already on master, so if you build from source, it'll likely solve this issue. Otherwise, importing models as modules as described above should always work. We tried to make sure spaCy never fully relies on the download and link commands that interact with the user's file system.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.