Ludwig: Unable to install ludwig

[harshitazilen]

Describe the bug
Unable to install ludwig by command "pip3 install ludwig"

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Termial'
2. Clone Plato library of Uber 'git clone https://github.com/uber-research/plato-research-dialogue-system.git'
3. Install pyaudio 'sudo apt-get install python3-pyaudio'
4. Go inside folder 'plato-research-dialogue-system'
5. install requirements using "pip3 install -r requirements.txt"

Please provide code, yaml definition file and a sample of data in order to entirely reproduce the issue.
Issues that are not reproducible will be ignored.

Expected behavior
It should complete installation

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• OS: Ubuntu
• Version 16.04
  
  
  
  • Python 3.6.7
  
  
  • Ludwig (latest)
  
  

Additional context
I face below error for fast api

  Could not find a version that satisfies the requirement fastapi (from ludwig->-r requirements.txt (line 3)) (from versions: )
No matching distribution found for fastapi (from ludwig->-r requirements.txt (line 3))

[msaisumanth]

You don't really need fastapi for experimenting. So you can comment it out and install Ludwig.

If you do need it, try installing it separately from Ludwig.

[harshitazilen]

@msaisumanth , Ludwig needs FastAPI. So installation process of Ludwig installs FastAPI and fails. Also I tried installing Fast API externally and then retried installing Ludwig, still the same error occurs.

Also if I check fastapi installation using command pip3 show fastapi , it shows below error:

Name: fastapi Version: 0.33.0 Summary: FastAPI framework, high performance, easy to learn, fast to code, ready for production Home-page: https://github.com/tiangolo/fastapi --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/__init__.py", line 996, in emit stream.write(msg) UnicodeEncodeError: 'ascii' codec can't encode character '\xe1' in position 15: ordinal not in range(128) Call stack: File "/mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/chatbots/bin/pip3", line 10, in <module> sys.exit(main()) File "/mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/chatbots/lib/python3.6/site-packages/pip/_internal/__init__.py", line 78, in main return command.main(cmd_args) File "/mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/chatbots/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 178, in main status = self.run(options, args) File "/mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/chatbots/lib/python3.6/site-packages/pip/_internal/commands/show.py", line 47, in run results, list_files=options.files, verbose=options.verbose): File "/mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/chatbots/lib/python3.6/site-packages/pip/_internal/commands/show.py", line 145, in print_results logger.info("Author: %s", dist.get('author', '')) Message: 'Author: %s' Arguments: ('Sebasti\xe1n Ram\xedrez',) Author-email: [email protected] License: UNKNOWN Location: /mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/chatbots/lib/python3.6/site-packages Requires: starlette, pydantic Required-by:

[msaisumanth]

@harshitazilen FastAPI is required for serving models. If you only want to experiment, you don't need it. So you can comment it out in the requirements and install Ludwig from source code.

[harshitazilen]

@msaisumanth , Thanks. finally I am able to install plato. I did with new virtual environment of python3.5. A virtual environment with python3.6 does not work (Error while installing httptools). Also there was one more error with "pydantic" and I fixed it. The requirement.txt file was asking for pydantic==0.30.0 which is not available so I installed latest version of pydantic.

[harshitazilen]

Unfortunately uber plato code does not work with python 3.5. It needs python 3.6 and while installing Ludwig from source in python 3.6, I face error in installing httptools , can you please guide me how to complete installation.

Please find error description as below:

Building wheels for collected packages: httptools
  Building wheel for httptools (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/uberplato/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-e7dr9ejv/httptools/setup.py'"'"'; __file__='"'"'/tmp/pip-install-e7dr9ejv/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-t81h0say --python-tag cp36
       cwd: /tmp/pip-install-e7dr9ejv/httptools/
  Complete output (29 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.6
  creating build/lib.linux-x86_64-3.6/httptools
  copying httptools/__init__.py -> build/lib.linux-x86_64-3.6/httptools
  creating build/lib.linux-x86_64-3.6/httptools/parser
  copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.6/httptools/parser
  copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.6/httptools/parser
  running egg_info
  writing httptools.egg-info/PKG-INFO
  writing dependency_links to httptools.egg-info/dependency_links.txt
  writing top-level names to httptools.egg-info/top_level.txt
  reading manifest file 'httptools.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  writing manifest file 'httptools.egg-info/SOURCES.txt'
  copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.6/httptools/parser
  running build_ext
  building 'httptools.parser.parser' extension
  creating build/temp.linux-x86_64-3.6
  creating build/temp.linux-x86_64-3.6/httptools
  creating build/temp.linux-x86_64-3.6/httptools/parser
  creating build/temp.linux-x86_64-3.6/vendor
  creating build/temp.linux-x86_64-3.6/vendor/http-parser
  x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -I/mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/uberplato/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
  httptools/parser/parser.c:4:20: fatal error: Python.h: No such file or directory
  compilation terminated.
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for httptools
  Running setup.py clean for httptools
Failed to build httptools
Installing collected packages: Cython, h5py, pyparsing, cycler, kiwisolver, matplotlib, PyWavelets, decorator, networkx, pillow, cloudpickle, toolz, dask, scikit-image, seaborn, wasabi, cymem, plac, preshed, murmurhash, srsly, tqdm, blis, thinc, spacy, tabulate, absl-py, protobuf, markdown, werkzeug, grpcio, tensorboard, tensorflow-estimator, wrapt, astor, keras-preprocessing, google-pasta, keras-applications, gast, termcolor, tensorflow, PyYAML, zipp, importlib-metadata, py, attrs, packaging, atomicwrites, wcwidth, pluggy, more-itertools, pytest, dataclasses, pydantic, httptools, uvicorn, python-multipart
  Running setup.py install for httptools ... error
    ERROR: Command errored out with exit status 1:
     command: /mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/uberplato/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-e7dr9ejv/httptools/setup.py'"'"'; __file__='"'"'/tmp/pip-install-e7dr9ejv/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-wda99v31/install-record.txt --single-version-externally-managed --compile --install-headers /mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/uberplato/include/site/python3.6/httptools
         cwd: /tmp/pip-install-e7dr9ejv/httptools/
    Complete output (29 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.6
    creating build/lib.linux-x86_64-3.6/httptools
    copying httptools/__init__.py -> build/lib.linux-x86_64-3.6/httptools
    creating build/lib.linux-x86_64-3.6/httptools/parser
    copying httptools/parser/__init__.py -> build/lib.linux-x86_64-3.6/httptools/parser
    copying httptools/parser/errors.py -> build/lib.linux-x86_64-3.6/httptools/parser
    running egg_info
    writing httptools.egg-info/PKG-INFO
    writing dependency_links to httptools.egg-info/dependency_links.txt
    writing top-level names to httptools.egg-info/top_level.txt
    reading manifest file 'httptools.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'httptools.egg-info/SOURCES.txt'
    copying httptools/parser/parser.c -> build/lib.linux-x86_64-3.6/httptools/parser
    running build_ext
    building 'httptools.parser.parser' extension
    creating build/temp.linux-x86_64-3.6
    creating build/temp.linux-x86_64-3.6/httptools
    creating build/temp.linux-x86_64-3.6/httptools/parser
    creating build/temp.linux-x86_64-3.6/vendor
    creating build/temp.linux-x86_64-3.6/vendor/http-parser
    x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.6m -I/mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/uberplato/include/python3.6m -c httptools/parser/parser.c -o build/temp.linux-x86_64-3.6/httptools/parser/parser.o -O2
    httptools/parser/parser.c:4:20: fatal error: Python.h: No such file or directory
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/uberplato/bin/python3.6 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-e7dr9ejv/httptools/setup.py'"'"'; __file__='"'"'/tmp/pip-install-e7dr9ejv/httptools/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-wda99v31/install-record.txt --single-version-externally-managed --compile --install-headers /mnt/1b974144-c202-4151-8390-a9df3154fd1a/harshit/RnD/VirtualEnvironments/uberplato/include/site/python3.6/httptools Check the logs for full command output.

[vaopaso]

@harshitazilen I solved it doing sudo apt-get install python3.6-dev
After that, I was able to install httptools.

[w4nderlust]

@harshitazilen can you try with python3.6-dev and confirm it works for you too?

[harshitazilen]

@w4nderlust , I tried installing python3.6-dev in my machine. Still facing the same issue.

[msaisumanth]

@harshitazilen Do you plan to use the model serving functionality? If not, remove fastapi from the requirements and install from master. I think that would solve the problem.

[Chee-Foong]

try

pip install ludwig==0.1.2

[w4nderlust]

In the new version in master there is a new way to specify dependencies: pip install ludwig will not install many additional packages, that are installable like pip install ludwig[serve]. This should solve your issues, as you can avoid installing the serve packages.
Closing it for now, let us know if you have further problems.