Isort: pie_slice in isort>=4.3.5 no longer sets sys.setdefaultencoding('utf-8')

[ericboyangan]

Hi! We're experiencing a new error in a Python 2.7 lint test that I believe was caused by bumping our isort from 4.3.4 to 4.3.5 (by way of flake8-isort's call to SortImports with both file_path and file_contents).

I see that the 4.3.5 version of pie_slice.py no longer calls sys.setdefaultencoding('utf-8') in python 2; which means source code containing non-ASCII now errors with UnicodeDecodeError on line 129 of isort.py. In general, IIUC, it seems file_contents cannot be a string with non-ASCII characters.

Wondering what you think would be the best resolution for this issue? And thank you so much for the great tool all these years!

[cleder]

File "/home/vagrant/venv/lib/python2.7/site-packages/isort/isort.py", line 108, in __init__
    if file_contents is None or ("isort:" + "skip_file") in file_contents:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 1988: ordinal not in range(128)

isort==4.3.9

[timothycrosley]

This has been fixed in the latest 4.3.10 release.
https://github.com/timothycrosley/isort/releases/tag/4.3.10

Thanks!

~Timothy

[bcroq]

I still get @cleder 's error using isort 4.3.10 when using flake8-isort:

  File "/home/bcq/.virtualenvs/.../lib/python2.7/site-packages/isort/isort.py", line 108, in __init__
    if file_contents is None or ("isort:" + "skip_file") in file_contents:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1050: ordinal not in range(128)

[bcroq]

@timothycrosley here is how I get this issue.

First, create a test.py Python file containing:

# coding: utf-8

x = 'testé'

Then run isort - < test.py.

Here is what I get with isort 4.3.5 to 4.3.10:

Traceback (most recent call last):
  File "/home/bcq/.../bin/isort", line 11, in <module>
    load_entry_point('isort', 'console_scripts', 'isort')()
  File "/home/bcq/git/isort/isort/main.py", line 331, in main
    SortImports(file_contents=sys.stdin.read(), write_to_stdout=True, **arguments)
  File "/home/bcq/git/isort/isort/isort.py", line 108, in __init__
    if file_contents is None or ("isort:" + "skip_file") in file_contents:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 26: ordinal not in range(128)

No problem with isort 4.3.4.

[timothycrosley]

Hi @bcroq,

Thanks for providing some additional repro steps! Can you try the just release 4.3.11 and see if it resolves this issue for you? https://github.com/timothycrosley/isort/releases/tag/4.3.11

Thanks!

~Timothy

[bcroq]

@timothycrosley 4.3.11 does not solve this issue.