Rasa: Spacy installation issue

[stevenev]

Rasa 1.6.1
Windows 10
Python 3.7.4

$ git clone https://github.com/RasaHQ/rasa.git
$ cd rasa
$ pip install rasa[spacy]

ERROR: en-core-web-md 2.2.5 has requirement spacy>=2.2.2, but you'll have spacy 2.1.9 which is incompatible.

[sara-tagger]

Thanks for raising this issue, @erohmensing will get back to you about it soon✨

Please also check out the docs and the forum in case your issue was raised there too 🤗

[erohmensing]

Hi @stevenev, if you want to install the source code, you should install from source, not pip. Can I ask why you want to install from source?

[stevenev]

Hi Ella, I tried both ways. The following does not work either:
pip install rasa[spacy]
above is from: https://rasa.com/docs/rasa/1.1.8/user-guide/installation/

Basically I want to start up RestaurantBot example Bot, but ran into spacy problem, which appears to be related to spacy version.

Thanks, steve

[stevenev]

I also observed the error when I ran this as noted in the documentation:

$ pip install -r alt_requirements/requirements_full.txt

[stevenev]

In related errors, I cloned RestaurantBot and when I run "rasa train", I get:

rasa.nlu.model.InvalidModelError: Model 'en' is not a linked spaCy model. Please download and/or link a spaCy model, e.g. by running:
python -m spacy download en_core_web_md
python -m spacy link en_core_web_md en

Please help. It has been 4 days since I asked for help.

[stevenev]

My problem is similar to
https://forum.rasa.com/t/oserror-e050-cant-find-model-en-it-doesnt-seem-to-be-a-shortcut-link-a-python-package-or-a-valid-path-to-a-data-directory/19707

which is unresolved. At this point, I don't know what I can do since a very basic Rasa command, below, gives error, and I hope developers can take a quick look:
$ pip install -r alt_requirements/requirements_full.txt

[stevenev]

My previous post references an error that is also on Windows. So my best guess at this point is that this is a Windows-related issue.

[JulianGerhard21]

Hi @stevenev ,

could you please open a Windows Terminal with Administrator privileges and use:

python -m spacy download en

after downloading, please use:

python -m spacy validate

and then please post the output of the downloading process and the validation here. It might be the case, that there is something like "Download was succesfull but could'nt link model" printed - if that'S the case, this indeed is a Windows related thing.

If you see something like:

TYPE      NAME                      MODEL                     VERSION
package   en-core-web-sm            en_core_web_sm            2.2.0     ✔
package   de-trf-bertbasecased-lg   de_trf_bertbasecased_lg   2.2.0     ✔
package   de-core-news-sm           de_core_news_sm           2.2.0     ✔
package   de-core-news-md           de_core_news_md           2.2.0     ✔
link      en                        en_core_web_sm            2.2.0     ✔

you can rasa train again.

Regards
Julian

[stevenev]

Thank you @JulianGerhard21 for the detailed instructions.

Actually first I had to do (using CMD Admin):
pip install spacy

and then I did:

python -m spacy download en
python -m spacy validate

I got:
====================== Installed models (spaCy v2.2.3) ======================
ℹ spaCy installation: C:\Users\steve\Anaconda3\lib\site-packages\spacy

TYPE NAME MODEL VERSION
package en-core-web-sm en_core_web_sm 2.2.5 ✔
package en-core-web-md en_core_web_md 2.2.5 ✔
link en en_core_web_sm 2.2.5 ✔

Still cannot train - same error:
rasa.nlu.model.InvalidModelError: Model 'en' is not a linked spaCy model. Please download and/or link a spaCy model, e.g. by running:
python -m spacy download en_core_web_md
python -m spacy link en_core_web_md en

My guess is that symlinking on Windows is very different from Mac/Linux. Unless and until this is fixed, I will be installing a VirtualBox/Linux and see if I can get Rasa[spacy] up and running there.

If you guys can spare some time testing Windows, it is appreciated, as I have seen others run into the same issue, or otherwise, maybe docs can clearly state there is a problem with spacy right now. Tnx.

[stevenev]

After some digging, looks like symlinking of spacy in Windows is a known issue: https://stackoverflow.com/questions/53505068/couldn-t-link-model-to-en-core-web-md-on-a-windows-10

I will test the approach shown in stackoverflow and report back here.

[JulianGerhard21]

Hi @stevenev ,

first of all, this documentation is for an old version of Rasa. I'd recommend to use the latest version since you mentioned that you are actually using version 1.6.1.

I tried to reproduce the error but everything worked pretty well including the training. What I did:

1. I used this tutorial for installing a virtualenv properly on Windows 10
2. I used pip install rasa[spacy] to install every dependency
3. I used python -m spacy download en for the model
4. I used rasa init and rasa train for the bot

The output was:

2020-01-14 18:07:32 INFO     rasa.utils.train_utils  - Finished training embedding policy, train loss=1.139, train accuracy=1.000
2020-01-14 18:07:32 INFO     rasa.nlu.model  - Finished training component.
2020-01-14 18:07:32 INFO     rasa.nlu.model  - Successfully saved model into 'C:\Users\xx\AppData\Local\Temp\tmp7pky_7c_\nlu'
NLU model training completed.
Your Rasa model is trained and saved at 'C:\Users\xx\PycharmProjects\test\models\20200114-180709.tar.gz'.

Actually there might be some kind of symlinking problems, but if you follow my guideline, everything should work fine.

Regards
Julian

[stevenev]

Hi Julian@JulianGerhard21, PROBLEM SOLVED!!! THANKS!!

The problem related to me running:
pip install rasa[spacy]

from Anaconda Powershell, NOT from CMD with Admin Privileges.

So my process with conda installed:

1. Create virtual environment in conda
   C:> python -m venv --system-site-packages ./venv

1. go to CMD with Admin and:
   .\venv\Scripts\activate
   pip install rasa[spacy]
   python -m spacy download en
   rasa init

2. Ok now to go back to Conda Powershell:

rasa train

So you were correct when you stated I needed to run from CMD with Admin, and I like to add to that to run
pip install rasa[spacy]
from the same CMD as well, which I had not done.

Again, million thanks. It was a long thread, but I think any Window user who is using Anaconda and installing Rasa from Anaconda Powershell and not CMD with Admin will experience the same issue.

[JulianGerhard21]

Hi @stevenev ,

glad that I could help!

Regards
Julian