Smart_open: UnicodeEncodeError on latest docker debian image

[slimm609]

Getting this error on the latest docker debian image with python-dev installed

``` Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-build-omJzX6/smart-open/setup.py", line 72, in
'Topic :: Database :: Front-Ends',
File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py", line 129, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/usr/lib/python2.7/distutils/command/install.py", line 613, in run
self.run_command(cmd_name)
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/install_egg_info.py", line 34, in run
self.run_command('egg_info')
File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
self.distribution.run_command(command)
File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 271, in run
writer(self, ep.name, os.path.join(self.egg_info, ep.name))
File "/usr/local/lib/python2.7/dist-packages/setuptools/command/egg_info.py", line 604, in write_pkg_info
metadata.write_pkg_info(cmd.egg_info)
File "/usr/lib/python2.7/distutils/dist.py", line 1106, in write_pkg_info
self.write_pkg_file(pkg_info)
File "/usr/local/lib/python2.7/dist-packages/setuptools/dist.py", line 76, in write_pkg_file
file.write('%s: %s\n' % (field, attr_val))
UnicodeEncodeError: 'ascii' codec can't encode character u'\u0158' in position 14: ordinal not in range(128)

Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-omJzX6/smart-open/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-CmK_oj-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-omJzX6/smart-open/
```

[xCatG]

I am seeing this with regular pip install smart-open as well... :-(

[dannyOhNo]

This isn't exclusive to Docker... able to reproduce on Python 2.7.10 via virtualenv...

virtualenv venv
. venv/bin/activate
python setup.py install

For reference...
uname -a

Darwin dannys-MBP 17.4.0 Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64 x86_64

[mpenkov]

```$ python
Python 3.6.4 (default, Jan 6 2018, 11:51:59)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

print('\u0158')
Ř

$ grep -R "Ř" .
./issue152/so1.5.5/lib/python3.6/site-packages/smart_open-1.5.5.dist-info/DESCRIPTION.rst:Copyright (c) 2015-now Radim Řehůřek <http://radimrehurek.com>_.
./issue152/so1.5.5/lib/python3.6/site-packages/smart_open-1.5.5.dist-info/METADATA:Author: Radim Řehůřek
./issue152/so1.5.5/lib/python3.6/site-packages/smart_open-1.5.5.dist-info/METADATA:Copyright (c) 2015-now Radim Řehůřek <http://radimrehurek.com>_.
./issue152/so1.5.3/lib/python3.6/site-packages/smart_open-1.5.3.dist-info/DESCRIPTION.rst:Copyright (c) 2015-now Radim Řehůřek <http://radimrehurek.com>_.
./issue152/so1.5.3/lib/python3.6/site-packages/smart_open-1.5.3.dist-info/METADATA:Author: Radim Řehůřek
./issue152/so1.5.3/lib/python3.6/site-packages/smart_open-1.5.3.dist-info/METADATA:Copyright (c) 2015-now Radim Řehůřek <http://radimrehurek.com>_.
./issue152/so1.5.4/lib/python3.6/site-packages/smart_open-1.5.4.dist-info/DESCRIPTION.rst:Copyright (c) 2015-now Radim Řehůřek <http://radimrehurek.com>_.
./issue152/so1.5.4/lib/python3.6/site-packages/smart_open-1.5.4.dist-info/METADATA:Author: Radim Řehůřek
./issue152/so1.5.4/lib/python3.6/site-packages/smart_open-1.5.4.dist-info/METADATA:Copyright (c) 2015-now Radim Řehůřek <http://radimrehurek.com>_.
./LICENSE:Copyright (c) 2015 Radim Řehůřek
./smart_open.egg-info/PKG-INFO:Author: Radim Řehůřek
./smart_open.egg-info/PKG-INFO: Copyright (c) 2015-now Radim Řehůřek <http://radimrehurek.com>_.
./setup.py: author=u'Radim Řehůřek',
./setup.py: maintainer=u'Radim Řehůřek',
./README.rst:Copyright (c) 2015-now Radim Řehůřek <http://radimrehurek.com>_.
```

Sounds like Python 2 is having problems encoding the package info. It's strange that this hasn't come up before...

[slimm609]

not sure. sudo pip install on 2.7 works but pip install as a user does not work

[mpenkov]

@slimm609 Could you update setup.py with the following changes and let me know if the problem persists?

```diff --git a/setup.py b/setup.py
index 319b558..6e59568 100644
--- a/setup.py
+++ b/setup.py
@@ -31,9 +31,9 @@ setup(
packages=find_packages(),
package_data={"smart_open.tests": ["test_data/*gz"]},

• author=u'Radim Řehůřek',
• author=u'Radim Řehůřek'.encode('utf-8'),
  author_email='[email protected]',
• maintainer=u'Radim Řehůřek',

• maintainer=u'Radim Řehůřek'.encode('utf-8'),
  maintainer_email='[email protected]',

  url='https://github.com/piskvorky/smart_open',```

[dannyOhNo]

Issue resolved: https://github.com/RaRe-Technologies/smart_open/pull/177

cc @mpenkov

[slimm609]

177 does fix the issue

[dannyOhNo]

Seems to be an issue with python 3.3.6.. trying to resolve now

[mpenkov]

@baldwindc @slimm609 I think it's preferable to work around this issue without butchering the author's name. Is encoding it to UTF-8 not sufficient?

[menshikh-iv]

Thanks fresh setuptools==39.0.0 for this problem, they released new version yesterday https://pypi.python.org/pypi/setuptools

[schlueter]

Good find @menshikh-iv. Based on that hint, I found that the last few versions (all released this weekend) suffer this issue. I had to go back to setuptools==38.6.1, which was added to the changelog 2 days ago. That works perfectly though, just:

pip install -U setuptools==38.6.1
pip install smart_open

[menshikh-iv]

Connected with https://github.com/pypa/setuptools/issues/1297

[menshikh-iv]

@slimm609 @xCatG @baldwindc @schlueter fixed in #178, smart_open==1.5.7 (that contained this fix) already released https://pypi.python.org/pypi/smart_open.

[schlueter]

With the release of setuptools==39.0.1 today, smart_open==1.5.6 may be installed without issues. So if Radim Řehůřek would like to be referred to with his accents intact, that won't cause issues now.

https://github.com/pypa/setuptools/pull/1300 was ultimately the fix to the problem.

[menshikh-iv]

@schlueter don't worry, I asked him before a change