Dvc: Unknown encoding error (cp65001) in windows exe version

Created on 21 Jun 2019 · 11Comments · Source: iterative/dvc

DVC version:
dvc-0.41.3.exe windows exe

Error message:

C:\Users\LexSong> dvc

C:\PROGRA~2\DATAVE~1\aliyunsdkcore\vendored\requests\packages\urllib3\contrib\socks.py:37: DependencyWarning: SOCKS support in urllib3 requires the installation of optional dependencies: specifically, PySocks.  For more information, see https://urllib3.readthedocs.io/en/latest/contrib.html#socks-proxies
Traceback (most recent call last):
  File "logging\__init__.py", line 868, in emit
  File "logging\__init__.py", line 741, in format
  File "dvc\logger.py", line 51, in format
  File "dvc\logger.py", line 149, in _progress_aware
  File "dvc\progress.py", line 36, in clearln
  File "dvc\progress.py", line 117, in _print
  File "site-packages\colorama\ansitowin32.py", line 41, in write
  File "site-packages\colorama\ansitowin32.py", line 162, in write
  File "site-packages\colorama\ansitowin32.py", line 187, in write_and_convert
  File "site-packages\colorama\ansitowin32.py", line 195, in write_plain_text
LookupError: unknown encoding: cp65001
Logged from file cli.py, line 83
Traceback (most recent call last):
  File "logging\__init__.py", line 868, in emit
  File "logging\__init__.py", line 741, in format
  File "dvc\logger.py", line 51, in format
  File "dvc\logger.py", line 149, in _progress_aware
  File "dvc\progress.py", line 36, in clearln
  File "dvc\progress.py", line 117, in _print
  File "site-packages\colorama\ansitowin32.py", line 41, in write
  File "site-packages\colorama\ansitowin32.py", line 162, in write
  File "site-packages\colorama\ansitowin32.py", line 187, in write_and_convert
  File "site-packages\colorama\ansitowin32.py", line 195, in write_plain_text
LookupError: unknown encoding: cp65001
Logged from file main.py, line 48

There is an encoding error in colorama when the code page is cp65001(UTF-8).
There is no issue when I install with pip install on Anaconda.

bug c1-quick-fix p0-critical

Source

LexSong

All 11 comments

@LexSong could you try to do set PYTHONIOENCODING=UTF-8 in your terminal before running dvc?

shcheklein on 21 Jun 2019

👍1

Hi @LexSong ! You are running a default windows terminal, right? And when running on Anaconda, you are using their own terminal? @shcheklein is right, setting that env var should help. Also, if it does, then it is most likely windows-terminal-specific error. To solve it we would need to include https://github.com/Drekin/win-unicode-console into our dependencies when running/building on windows.

Thanks for the feedback! :)

efiop on 21 Jun 2019

And when running on Anaconda, you are using their own terminal?

No, I'm using the default windows 10 terminal, too.

set PYTHONIOENCODING=UTF-8

I'll see if this works.

LexSong on 21 Jun 2019

👍1

set PYTHONIOENCODING=UTF-8 doesn't work.

LexSong on 21 Jun 2019

😕1

@LexSong I've added a package to our dependencies in https://github.com/iterative/dvc/commit/c7ca83e226f5b10c2b986307820a74ba846027cc , which might help with that. Would you be ok with using anaconda for now? If so, we will soon build a new windows package with that package included and will notify you to try it out, so you could confirm or deny that it works for you.

efiop on 21 Jun 2019

By the way, my anaconda environment is here:

name: dvc
channels:
  - defaults
  - conda-forge
dependencies:
  - asn1crypto=0.24.0=py37_0
  - ca-certificates=2019.5.15=0
  - certifi=2019.6.16=py37_0
  - cffi=1.12.3=py37h7a1dbc1_0
  - cryptography=2.7=py37h7a1dbc1_0
  - dulwich=0.19.11=py37hfa6e2cd_0
  - idna=2.8=py37_0
  - openssl=1.1.1c=he774522_1
  - pip=19.1.1=py37_0
  - pycparser=2.19=py37_0
  - pyopenssl=19.0.0=py37_0
  - pysocks=1.7.0=py37_0
  - python=3.7.3=h8c8aaf0_1
  - setuptools=41.0.1=py37_0
  - six=1.12.0=py37_0
  - sqlite=3.28.0=he774522_0
  - urllib3=1.25.3=py37_0
  - vc=14.1=h0510ff6_4
  - vs2015_runtime=14.15.26706=h3a45250_4
  - wheel=0.33.4=py37_0
  - win_inet_pton=1.1.0=py37_0
  - wincertstore=0.2=py37_0
  - pip:
    - appdirs==1.4.3
    - asciimatics==1.11.0
    - chardet==3.0.4
    - colorama==0.4.1
    - configobj==5.0.6
    - configparser==3.7.4
    - contextlib2==0.5.5
    - decorator==4.4.0
    - distro==1.4.0
    - dvc==0.41.3
    - future==0.17.1
    - gitdb2==2.0.5
    - gitpython==2.1.11
    - grandalf==0.6
    - humanize==0.5.1
    - inflect==2.1.0
    - jsonpath-ng==1.4.3
    - nanotime==0.5.2
    - networkx==2.3
    - pillow==6.0.0
    - ply==3.11
    - pyasn1==0.4.5
    - pyfiglet==0.8.post1
    - pyparsing==2.4.0
    - pypiwin32==223
    - pywin32==224
    - requests==2.22.0
    - ruamel-yaml==0.15.97
    - schema==0.7.0
    - smmap2==2.0.5
    - treelib==1.5.5
    - wcwidth==0.1.7
    - zc-lockfile==1.4
prefix: C:\Miniconda3\envs\dvc

Maybe it will be helpful.

LexSong on 21 Jun 2019

❤1

OK, I think the issue may be caused by Python 2.7. Good luck to fix it. I'll keep using Python 3 for now.

LexSong on 21 Jun 2019

@LexSong Oh, good point! We are still using python2 to build binaries with pyinstaller. I'll adjust it to use python3 shortly too. Thanks! :)

efiop on 21 Jun 2019