Fasttext: UnicodeDecodeError on pip install

[theo-m]

Description

I used to pull fastText with pip install git+git://github.com/facebookresearch/fastText.git which was working fine.
Now it fails with:

user@c17bd96d6458:/app$ pip install git+git://github.com/facebookresearch/fastText.git
Collecting git+git://github.com/facebookresearch/fastText.git
  Cloning git://github.com/facebookresearch/fastText.git to /tmp/pip-req-build-61go2tgp
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-req-build-61go2tgp/setup.py", line 173, in <module>
        long_description=_get_readme(),
      File "/tmp/pip-req-build-61go2tgp/setup.py", line 164, in _get_readme
        return fid.read()
      File "/home/user/miniconda/envs/py36/lib/python3.6/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1102: ordinal not in range(128)

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-req-build-61go2tgp/

I have the same result with pip install fasttext (I saw your announcement).

user@2b14c6d1ac0d:/app$ pip install fasttext
Collecting fasttext
  Downloading https://files.pythonhosted.org/packages/40/bc/cbb93d0696c068a780a1d144a5ca6185d11f272fb8a32252e54db633fe94/fasttext-0.9.tar.gz (57kB)
    100% |################################| 61kB 3.0MB/s
    Complete output from command python setup.py egg_info:
    Collecting pybind11
      Downloading https://files.pythonhosted.org/packages/5d/85/c7a8dffda52ce25a8bcfe9a28b6861bdd52da59ae001fdd4173e054b7d9b/pybind11-2.3.0-py2.py3-none-any.whl (147kB)
    Installing collected packages: pybind11
    Successfully installed pybind11-2.3.0
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-ewt93cc9/fasttext/setup.py", line 173, in <module>
        long_description=_get_readme(),
      File "/tmp/pip-install-ewt93cc9/fasttext/setup.py", line 164, in _get_readme
        return fid.read()
      File "/home/user/miniconda/envs/py36/lib/python3.6/encodings/ascii.py", line 26, in decode
        return codecs.ascii_decode(input, self.errors)[0]
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 1102: ordinal not in range(128)

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-ewt93cc9/fasttext/
user@2b14c6d1ac0d:/app$

Reproducing

You should be able to reproduce, these two results were obtained following docker image (python 3.6)

docker run -it anibali/pytorch:no-cuda /bin/bash

Observations

However if I create locally a new miniconda environment on python 3.6 it works 🤔

conda create -n fasttext-test python=3.6
source activate fasttext-test
pip install fasttext

Thanks

---

Edits:

1. Digging in, the docker image is on 3.6.5 while my local conda env is 3.6.8 but I'd be surprised if it was the reason. Offending line is here: https://github.com/facebookresearch/fastText/blame/master/python/README.rst#L31
2. It's not python versions of course, local conda install with 3.6.5 works as well

[theo-m]

Found the issue, the docker image has no locale set, using export LC_ALL=C.UTF-8 fixes the problem. See https://github.com/docker-library/python/issues/13

[Celebio]

Hi @theo-m ,
Thank you for reporting the problem. This should be fixed with this pull request.

Best regards,
Onur