Isort: Crash on pylint's functional test with a bogus encoding declared

[Pierre-Sassoulas]

This is voluntarily a hard case to handle, here's the file : https://github.com/PyCQA/pylint/blob/master/tests/functional/u/unknown_encoding_py29.py

ERROR: Unrecoverable exception thrown when parsing tests/functional/u/unknown_encoding_py29.py! This should NEVER happen.
If encountered, please open an issue: https://github.com/PyCQA/isort/issues/new
Traceback (most recent call last):
  File "/usr/lib/python3.8/tokenize.py", line 342, in find_cookie
    codec = lookup(encoding)
LookupError: unknown encoding: IBO-8859-1

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pierre/.cache/pre-commit/reposvah2k04/py_env-python3/bin/isort", line 8, in <module>
    sys.exit(main())
  File "/home/pierre/.cache/pre-commit/reposvah2k04/py_env-python3/lib/python3.8/site-packages/isort/main.py", line 886, in main
    for sort_attempt in attempt_iterator:
  File "/home/pierre/.cache/pre-commit/reposvah2k04/py_env-python3/lib/python3.8/site-packages/isort/main.py", line 875, in <genexpr>
    sort_imports(  # type: ignore
  File "/home/pierre/.cache/pre-commit/reposvah2k04/py_env-python3/lib/python3.8/site-packages/isort/main.py", line 94, in sort_imports
    incorrectly_sorted = not api.sort_file(
  File "/home/pierre/.cache/pre-commit/reposvah2k04/py_env-python3/lib/python3.8/site-packages/isort/api.py", line 300, in sort_file
    with io.File.read(filename) as source_file:
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/home/pierre/.cache/pre-commit/reposvah2k04/py_env-python3/lib/python3.8/site-packages/isort/io.py", line 48, in read
    stream = File._open(file_path)
  File "/home/pierre/.cache/pre-commit/reposvah2k04/py_env-python3/lib/python3.8/site-packages/isort/io.py", line 33, in _open
    encoding, _ = tokenize.detect_encoding(buffer.readline)
  File "/usr/lib/python3.8/tokenize.py", line 381, in detect_encoding
    encoding = find_cookie(second)
  File "/usr/lib/python3.8/tokenize.py", line 350, in find_cookie
    raise SyntaxError(msg)
SyntaxError: unknown encoding for '/home/pierre/pylint/tests/functional/u/unknown_encoding_py29.py': IBO-8859-1

I don't think if isort should do anything about it, except exiting gracefully instead.

[timothycrosley]

@Pierre-Sassoulas thanks for forwarding this issue along! I'm thinking it might make sense for isort to:
a) Warn if it's one of many files passed in
b) Error, but gracefully with an encoding related error, if it is the only file passed in?

Thanks!
~Timothy

[Pierre-Sassoulas]

Thank you for the quick reply. This sound good, I think the main thing to consider is not blocking a batch treatment, I think that you could have the same behavior for a single file to make your life easier.

[anirudnits]

Opened a PR https://github.com/PyCQA/isort/pull/1496 trying to solve this issue. Its not complete and not clearing all the tests, have a look and help me out :)

Thanks

[timothycrosley]

Update: this has just been released to PyPI in 5.6.0 release of isort: https://pycqa.github.io/isort/CHANGELOG/#560-october-7-2020

Thanks!

~Timothy

[Pierre-Sassoulas]

Thanks for the update, and congrats on the fast fix :) !