Black: Struggling to run within pycharm

[kindjacket]

Black works totally fine from the command line - my issue is running it from within pycharm. I've followed the instructions to install for pycharm in the README.

I get the following error when I run it

```Traceback (most recent call last):
File "/usr/local/bin/black", line 11, in
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__
return self.main(args, *kwargs)
File "/usr/local/lib/python3.6/site-packages/click/core.py", line 676, in main
_verify_python3_env()
File "/usr/local/lib/python3.6/site-packages/click/_unicodefun.py", line 118, in _verify_python3_env
'for mitigation steps.' + extra)
RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult http://click.pocoo.org/python3/for mitigation steps.

This system lists a couple of UTF-8 supporting locales that
you can pick from. The following suitable locales where
discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, ca_ES.UTF-8, cs_CZ.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_CH.UTF-8, de_DE.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_CA.UTF-8, en_GB.UTF-8, en_IE.UTF-8, en_NZ.UTF-8, en_US.UTF-8, es_ES.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fi_FI.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, he_IL.UTF-8, hr_HR.UTF-8, hu_HU.UTF-8, hy_AM.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, ja_JP.UTF-8, kk_KZ.UTF-8, ko_KR.UTF-8, lt_LT.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, no_NO.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8

I've tried fixed like such but I still get the same error 

export LC_ALL=en_us.UTF-8
export LANG=en_us.UTF-8

If I run the following from within the shell of my virtualenv or my global python3 shell everything looks fine.

```➜  ~ python3
Python 3.6.5 (default, Apr 25 2018, 14:23:58)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getlocale()
('en_US', 'UTF-8')

Operating system: Mac OS High Sierra 10.13.4
Python version: 3.6.5
Black version: black 18.5b1

[ambv]

@vlasovskikh, do you have a suggestion how this can be fixed within PyCharm?

[vlasovskikh]

I've tried installing Black to my Python 3 interpreter on macOS and setting it up in PyCharm as an external tool. It went well, so I cannot reproduce it at the moment.

Make sure that the interpreter you've installed Black into has locale set properly. Check the first line of your black executable and run the following testing program with that interpreter by its full path:

environment_test.py

import locale
import os
from pprint import pprint

pprint(dict(os.environ))
pprint(locale.getlocale())

import click


@click.command()
def main():
    print("Click works well")


if __name__ == "__main__":
    main()

Then you can set up this testing program as an external tool in PyCharm in order to test your environment variables and your locale.

Did it help you with finding the source of the problem?

[vlasovskikh]

@ambv BTW there is a third-party black-pycharm plugin for PyCharm, but apparently it has the same issue at least on some environments, see https://github.com/pablogsal/black-pycharm/issues/2.

[ambv]

@kindjacket we are waiting for your report on whether you found the source of your issue.

[danigosa]

I had the same issue, and now solved. The steps were the suspected ones:

1. Install python 3.6 through brew: brew update && brew install python3
2. sudo pip3 install black
3. Check black command works with No paths given. Nothing to do 😴
4. Create external tool in PyCharm (see below)

As an external tool it works:

With pycharm-black extension it happens as described here: https://github.com/pablogsal/black-pycharm/issues/2, it picks Java's Runtime to perform it what it seems to not work well in MacOS: https://github.com/pablogsal/black-pycharm/blob/master/src/ReformatCode.java#L53 (God's know why, even if you have generally set LANG and LC's well configured). The only workaround to make the extension work is to modify /usr/local/bin/black and set locales in Python by hand 😅 I did it and then uninstalled the extension then put the original black code back 😎.

[vlasovskikh]

@danigosa Could you try my testing script for your Python installed via brew please?

[danigosa]

@vlasovskikh

 $ python3                            
Python 3.6.5 (default, May 19 2018, 06:01:45)
[GCC 4.2.1 Compatible Apple LLVM 9.1.0 (clang-902.0.39.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
 $ python3 test.py                   
{'Apple_PubSub_Socket_Render': '/private/tmp/com.apple.launchd.6Af6MzDAF0/Render',
 'COLORFGBG': '15;0',
 'COLORTERM': 'truecolor',
 'DISPLAY': '/private/tmp/com.apple.launchd.bB5Y1FhpDw/org.macosforge.xquartz:0',
 'HOME': '/Users/demo',
 'ITERM_PROFILE': 'Default',
 'ITERM_SESSION_ID': 'w1t0p1:995496BD-4C52-4BDA-9A1D-0C0B01B34ED1',
 'LANG': 'en_US.UTF-8',
 'LC_ALL': '',
 'LESS': '-R',
 'LOGNAME': 'demo',
 'LSCOLORS': 'Gxfxcxdxbxegedabagacad',
 'OLDPWD': '/Users/demo/Dropbox/workspace/robbie/robbie-services/robbie-profiles-services',
 'PAGER': 'less',
 'PATH': '/Users/demo/tmp/google-cloud-sdk/bin:/Users/demo/bin:/usr/local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/opt/X11/bin',
 'PWD': '/Users/demo/Dropbox/workspace/robbie/robbie-services/robbie-profiles-services',
 'SHELL': '/bin/zsh',
 'SHLVL': '1',
 'SSH_AUTH_SOCK': '/private/tmp/com.apple.launchd.ihIWNbgFnF/Listeners',
 'TERM': 'xterm-256color',
 'TERM_PROGRAM': 'iTerm.app',
 'TERM_PROGRAM_VERSION': '3.1.6beta5',
 'TERM_SESSION_ID': 'w1t0p1:995496BD-4C52-4BDA-9A1D-0C0B01B34ED1',
 'TMPDIR': '/var/folders/wh/5r8gfrpx07z_rdwtsr8q3l9h0000gn/T/',
 'USER': 'demo',
 'XPC_FLAGS': '0x0',
 'XPC_SERVICE_NAME': '0',
 'ZSH': '/Users/demo/.oh-my-zsh',
 '_': '/usr/local/bin/python3',
 '__CF_USER_TEXT_ENCODING': '0x1F5:0x0:0x0',
 '__PYVENV_LAUNCHER__': '/usr/local/bin/python3'}
('en_US', 'UTF-8')
Click works well

[vlasovskikh]

@danigosa Have you set up this testing script as an _external tool_ in PyCharm (as the instructions for Black recommend)? Based on your output I guess you've tried it in an external terminal or the PyCharm console or the PyCharm terminal.

This testing script acts as a substitute for Black so we can learn what's wrong with your environment when you run Black instead of it in the exactly the same manner.

[jacobzweig]

I was able to fix this by explicitly setting the LANG environment variable in the black file (e.g., /usr/local/bin/black)

# -*- coding: utf-8 -*-
import re
import sys
import os

from black import main

if __name__ == '__main__':

    os.environ['LANG'] = 'en_US.utf-8'  
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(main())

[ambv]

@jacobzweig, this silences the exception but it doesn't fix the problem that Click is pointing out. You will not be able to properly reformat files with non-ASCII letters in the filename.

[danigosa]

@vlasovskikh nope, I ran it directly from iTerm (with zsh). It works for me as an External Tool and in the command-line, what it does not work is the plugin for Pycharm.

[swilcox]

I had a similar issue running black under the latest python extension for VSC (now that it supports it). And I run zsh as my shell. Once I modified my .zshrc file by adding (or uncommenting) the line export LANG=en_US.UTF-8 ... I restarted VSC and black began to work (no more click errors). So perhaps for those with PyCharm and zsh etc... maybe this will help.

[danigosa]

@vlasovskikh impossible to reproduce in the PyCharm plugin this is all I got from running (even with your script):

BlackPycharm: Traceback (most recent call last): File "/usr/local/bin/black", line 18, in   main() File "/usr/local/lib/python3.6/site-packages/click/core.py", line 722, in __call__ return self.main(*args, **kwargs) File "/usr/local/lib/python3.6/site-packages/click/core.py", line 676, in main _verify_python3_env() File "/usr/local/lib/python3.6/site-packages/click/_unicodefun.py", line 118, in _verify_python3_env 'for mitigation steps.' + extra) RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult http://click.pocoo.org/python3/for mitigation steps.  This system lists a couple of UTF-8 supporting locales that you can pick from. The following suitable locales where discovered: af_ZA.UTF-8, am_ET.UTF-8, be_BY.UTF-8, bg_BG.UTF-8, ca_ES.UTF-8, cs_CZ.UTF-8, da_DK.UTF-8, de_AT.UTF-8, de_CH.UTF-8, de_DE.UTF-8, el_GR.UTF-8, en_AU.UTF-8, en_CA.UTF-8, en_GB.UTF-8, en_IE.UTF-8, en_NZ.UTF-8, en_US.UTF-8, es_ES.UTF-8, et_EE.UTF-8, eu_ES.UTF-8, fi_FI.UTF-8, fr_BE.UTF-8, fr_CA.UTF-8, fr_CH.UTF-8, fr_FR.UTF-8, he_IL.UTF-8, hr_HR.UTF-8, hu_HU.UTF-8, hy_AM.UTF-8, is_IS.UTF-8, it_CH.UTF-8, it_IT.UTF-8, ja_JP.UTF-8, kk_KZ.UTF-8, ko_KR.UTF-8, lt_LT.UTF-8, nl_BE.UTF-8, nl_NL.UTF-8, no_NO.UTF-8, pl_PL.UTF-8, pt_BR.UTF-8, pt_PT.UTF-8, ro_RO.UTF-8, ru_RU.UTF-8, sk_SK.UTF-8, sl_SI.UTF-8, sr_YU.UTF-8, sv_SE.UTF-8, tr_TR.UTF-8, uk_UA.UTF-8, zh_CN.UTF-8, zh_HK.UTF-8, zh_TW.UTF-8

[jeremy886]

+1 same problem for my Mac (used to work) but it works on my Windows 10 machine

[yunti]

Just tested with pycharm (2018.1.4 and macOS High sierra 10.13.5)

works perfectly in pycharm using black 18.6b2
didn't install the pycharm plugin

installed via:
(python3 already installed with brew)
Install via
$ pip3 install black

Test in terminal with:
$ black

No paths given. Nothing to do 😴

Add to pycharm via setting-> external tool

[slochower]

As another data point, I'm having the same problem on macOS 10.13.15 with Visual Studio Code 1.24.1 using the new built-in formatter option for black. The output of the test script by @vlasovskikh looks normal.

[jeremy886]

@yunti thanks. I tried again using "external tool" and it works for now. 👍👍

I dug into the error messages and found the problem is from "click".
It seems nothing will be done from the click side according to the discussions in its issue tracker. 🙀💀 If Python3 is the problem, is it possible to find an alternative (for click, not python 🤗)?

[ambv]

The thing is, Click is right in the fact that if LC_ALL is misconfigured then Python cannot open files with paths that are outside of the misconfigured ASCII charset.

Python 3.7 fixes this by defaulting to UTF-8 anyway.

And since it's unlikely people will use Unicode names in Python paths, I will just silence this warning for the next release of Black.

[rupert]

Another option for PyCharm users is to use the LSP Support plugin (untested) with the Python Language Server and pyls-black plugin.

[DylanMuir]

I have this problem as well. The locale test script from @vlasovskikh raised an error "ValueError: unknown locale: UTF-8".

The fix was to add these lines to .bash_profile to explicitly define the locale:

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

I noticed in my environment that LC_CTYPE was set to en_CH.UTF-8. Just including this in case the non-US locale might have caused problems.

[vlasovskikh]

@DylanMuir I believe the most recent version of Black already includes the workaround mentioned above and implemented by @ambv.