Rasa: Using rasa_nlu[spacy] with spacy 2.1

[bastbnl]

Hi,

Newby question here. I'm trying to install rasa_nlu[spacy] onto my pre-existing development virtualenv. This fails because I'm currently using spacy==2.1.3 and I'd like to continue using that.

The rasa version I'm trying to install is 0.15.0 on python 3.7.3 which is running on Linux x64.

I imagine the installation fails due to https://github.com/RasaHQ/rasa/blob/master/setup.py#L87, which adds a dependency to at most spacy version 2.0.18. Is there a reason why the rasa is no using spacy 2.1 yet?

I've read the issue template and left out the yml sections since I didn't make it there yet.

Thanks! Really looking forward to getting started using this wonderful project!

[akelad]

Thanks for raising this issue, @tmbo will get back to you about it soon.

[tmbo]

I don't think there is an issue with 2.1, just checked the release notes. I think we can loosen that restriction

[tmbo]

https://github.com/RasaHQ/rasa/pull/3329

[bastbnl]

Thanks, @tmbo. I'm trying to install rasa as per the manual on a VM running Debian 9.9 with Python 3.7.3.

From the manual I'm assuming I should run this:
pip install rasa[spacy]

This fails:

env) bastb@bastb-vps:/var/www/brwnppr/dev/env$ BLAS=/usr/lib/libblas.so.3 LAPACK=/usr/lib/liblapack.so.3 ATLAS=/usr/lib/libatlas.so.3 TMPDIR=/var/tmp pip install --no-binary :all: 'rasa[spacy]'
Collecting rasa[spacy]
  Using cached https://files.pythonhosted.org/packages/35/7d/5507e5f18e9b07c8433d1afe7e311ae742078c3b97e776d82b50c2fce41f/rasa-1.0.7.tar.gz
Requirement already satisfied: requests~=2.22 in ./lib/python3.7/site-packages (from rasa[spacy]) (2.22.0)
Requirement already satisfied: boto3~=1.9 in ./lib/python3.7/site-packages (from rasa[spacy]) (1.9.164)
Collecting matplotlib~=3.0 (from rasa[spacy])
  Using cached https://files.pythonhosted.org/packages/51/fe/84ab101f8ab543d89b6a128326f62adcdafd2781ab8362a737e6ce78eea7/matplotlib-3.1.0.tar.gz
Requirement already satisfied: simplejson~=3.16 in ./lib/python3.7/site-packages (from rasa[spacy]) (3.16.0)
Requirement already satisfied: attrs>=18 in ./lib/python3.7/site-packages (from rasa[spacy]) (19.1.0)
Collecting jsonpickle~=1.1 (from rasa[spacy])
  Using cached https://files.pythonhosted.org/packages/00/55/ce2cbc6d64034b30cad81a29ba61bdba456f190f5e83c09831304bf68d6b/jsonpickle-1.2.tar.gz
Requirement already satisfied: redis~=3.2 in ./lib/python3.7/site-packages (from rasa[spacy]) (3.2.1)
Collecting fakeredis~=1.0 (from rasa[spacy])
  Using cached https://files.pythonhosted.org/packages/a2/98/e522a70987b35052f2776f654b014c01218ce6a473711277d3a7c2570931/fakeredis-1.0.3.tar.gz
Collecting pymongo~=3.8 (from rasa[spacy])
  Using cached https://files.pythonhosted.org/packages/19/77/da358f5729ff046ceaf6c6a86755f9d8285719f93df6da15bb2440367d7e/pymongo-3.8.0.tar.gz
Requirement already satisfied: numpy~=1.16 in ./lib/python3.7/site-packages (from rasa[spacy]) (1.16.2)
Requirement already satisfied: scipy~=1.2 in ./lib/python3.7/site-packages (from rasa[spacy]) (1.3.0)
Collecting tensorflow~=1.13.0 (from rasa[spacy])
  ERROR: Could not find a version that satisfies the requirement tensorflow~=1.13.0 (from rasa[spacy]) (from versions: none)
ERROR: No matching distribution found for tensorflow~=1.13.0 (from rasa[spacy])

It seems like the setup.py requires tensorflow. But tensorflow is not available for python 3.7. The install further requires pymongo. And a lot of other packages. So now I'm wondering if I get it.

I want to use rasa for nlu with spacy. I don't think I need a mattermost client, pymongo and a slack client. Can it be done? Do I need to alter my pip install command line somehow to prevent the installation of the additional packages?

The idea is that I use NLU in an existing Django project.

[tmbo]

There is nothing about the tensorflow support - the only thing for py 3.7 you can do is to install the release candidate (1.14.0rc1) - as soon as this is released as a stable version we can properly support python 3.7

The additional dependencies are really just to make the setup easier as it would be hard to explain which dependencies need to be installed - are there any issues with having pymongo installed even though you do not need it? (I do know it is not ideal, but having lots of different ways to install the package isn't ideal either)

[akelad]

@tmbo what do you mean with tensorflow support? 1.13.1 exists and it supports py3.7

[tmbo]

looking at https://pypi.org/project/tensorflow/1.13.1/ it doesn't (left-hand side under "Programming Language")

[akelad]

but at least me and @Ghostvv are running py3.7 and it works :D
Also their release notes say they do: https://github.com/tensorflow/tensorflow/releases/tag/v1.13.1
I think they just forgot to update the supported languages on pypi?

[tmbo]

do you know if they added it to 1.13.1 or 1.13.0 - in any case it is unclear why installation fails if it is supported (I am not what pip is looking at to check if a python version is supported, if it is the labels then they missed to add them for 1.13.1). did you have any issues installing on py 3.7?

[Ghostvv]

Python 3.7.3 (default, Mar 27 2019, 09:23:15) 
[Clang 10.0.1 (clang-1001.0.46.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.13.1'

[Ghostvv]

I think they added with 1.13.0

[akelad]

nah i had no issues, it all worked fine

[bastbnl]

Thanks for getting back to me @tmbo Much appreciated!

There is nothing about the tensorflow support - the only thing for py 3.7 you can do is to install the release candidate (1.14.0rc1) - as soon as this is released as a stable version we can properly support python 3.7

I'm installing using pip --no-binary :all: install ..., so maybe it's caused by that. Surely hope not.

The additional dependencies are really just to make the setup easier as it would be hard to explain which dependencies need to be installed - are there any issues with having pymongo installed even though you do not need it? (I do know it is not ideal, but having lots of different ways to install the package isn't ideal either)
I do think I understand, but it opens a box of questions. For instance: I need to inspect the licenses for each of these products. I need to keep these packages up to date, as additional packages increase the attack vector. The system I'm working on is not very interesting right now, but it might be in the future.

I think you're aiming for a friction-free install of rasa, which is a very good idea. Maybe add a package rasa_base[spacy] which would be required by rasa and which has the nlu components and everything it needs, but doesn't include tensorflow, pymongo, ... . That way somebody going for a friction free installation would just install rasa, and somebody - like me - would install the base package for spacy - since I'm already using spacy - and benefit from the NLU parts in my pre-existing app.

This raises another question. Is that feasible? Can I help in any way?

[akelad]

i'm afraid that's not really feasible, because rasa.core requires tensorflow

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed due to inactivity. Please create a new issue if you need more help.