Click: Remove unicode_literals `warning` or some easier way to globally disable

[cooperlees]

We (@ Facebook) have been using click a lot with Python 3 and Python 2 with

from __future__ import unicode_literals

and yes, if used incorrectly unicode can blow up click, so I understand the warning, but feel we should fix the code and take this away.

Can we look at fixing code to be more unicode friendly (maybe based on 'utf-8'). Today when ever we pull in click we have to modify it and this is painful as another upgrade happened and we forgot about this modification we made to set:

click.disable_unicode_literals_warning = True

globally.

Thoughts?

[dsully]

We (@ LinkedIn) have the same problem. +1 to this. I do hear that Python 3.7 may address this.

[davidism]

The problem is not with Unicode support in Click, it's with the consequences of that future in other libraries. In Python 2, if you use that future, it's really easy to mistakenly pass Unicode data to libraries that don't understand it.

This warning is explained in more depth in the docs, but maybe I'm misunderstanding what you're describing.

[warsaw]

Yes, it's true the future import of unicode_literals can cause problems, especially in straddling code, and particularly (IME) when migrating unittests. But sometimes it's fine, and I'm not aware of any automated way to make that determination. We have to trust the developer to make that determination, and generally it'll be the leaf application developer who is in the best position to decide.

Apropos of Python 3.7, these two PEPs are relevant for the UTF-8 runtime environment:

• PEP 538, legacy C locale coercion
• PEP 540, forced UTF-8 runtime mode

I'd be interested in experiences using click with Python 3.7 in non-UTF-8 locales, and whether these two PEPs make any difference.

[bt-thiago]

Related to this, I'm interested to know if the recommendation to use Python 2 is still current:

it is strongly recommended to use Python 2 for Click utilities unless Python 3 is a hard requirement.

I've been holding-out on using Click because of the strongly worded recommendation above as Python 2 is not an option I can take.

Thanks!

[davidism]

No, the docs need to be updated to remove all mentions of Python 2 over Python 3. PRs welcome.

[davidism]

As to the actual issue, I have no idea where I would start with it, but PRs are welcome. Given that January 1, 2020 is getting closer, I'm not sure how important it is anymore though.

[bt-thiago]

I'd say removing the recommendation to use Python 2 is a good start because people who try to run Click in an environment with LANG unset are directed to https://click.palletsprojects.com/en/7.x/python3/#python-3-support

I'll give the doco PR a crack later this week. Thanks for your quick reply btw.

[davidism]

@cooperlees @dsully can you comment on whether you still need this? Python 2 is EOL, and maybe the projects you wanted this for have upgraded? I never got clarification if the issue you were referring to was Click's use of Unicode (which as far as I know is correct), or if you just didn't want the warning (which can be disabled either by the disable_unicode_literals_warning flag or with the warnings module).

I agree the warning is heavy handed if the project knows what they're doing when enabling that future. Since Click itself should handle unicode or str values on Python 2 transparently, a PR to remove the warning is probably fine.

[dsully]

We are Python 3.6+ only now, and soon to be 3.7+ shortly (in the next week or so @warsaw ?)

I don't think we need it anymore.

[warsaw]

We should have 3.8 available by then, but it will be a little longer before we can drop 3.6.

[davidism]

This is now removed because Click 8 (master) is Python 3 only.