Homebrew-core: Python formula does not include TK - python uses default Apple TK 8.5 which causes bugs/crashes

[gwpantazes]

Please note we will close your issue without comment if you delete, do not read or do not fill out the issue checklist below and provide ALL the requested information. If you repeatedly fail to use the issue template, we will block you from ever submitting issues to Homebrew again.

• [x] are reporting a bug others will be able to reproduce and not asking a question or requesting software. If you're not sure or want to ask a question do so on our Discourse: https://discourse.brew.sh. To get software added or changed in Homebrew please file a Pull Request
• [x] have a problem with brew install (or upgrade, reinstall) a single, Homebrew/homebrew-core formula (not cask) on macOS? If it's a general brew problem please file this issue at Homebrew/brew: https://github.com/Homebrew/brew/issues/new/choose. If it's a Linux problem please file this issue at https://github.com/Homebrew/linuxbrew-core/issues/new/choose. If it's a brew cask problem please file this issue at https://github.com/Homebrew/homebrew-cask/issues/new/choose. If it's a tap (e.g. Homebrew/homebrew-php) problem please file this issue at the tap.
• [x] ran brew update and can still reproduce the problem?
• [x] ran brew doctor, fixed all issues and can still reproduce the problem?
  
  
  
  • See brew doctor output in this gist. I saw Python linking needed fixing, so addressed that. Did not mess with the python config files (I'll do that only if I have to).
  
  
• [x] ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link? https://gist.github.com/40d4b8564da9ad23a1e9b141d3917a10
• [ ] if brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

To help us debug your issue please explain:

tl;dr: Python formula does not include TK, relies on old Apple TK version (8.5), so causes TK bugs.

This issue (python formula missing up-to-date TK) has already been reported before (#26880), but the solution given in a comment there is not valid since there is no longer an --with-tcl-tk option to install with TK.

Suggested solution is to bundle up-to-date TK with python formula just like python.org bundles TK with their installations.

What you were trying to do (and why)

Install python via hombrew, then use the IDLE app (which is included in the python brew installation).

What happened (include command output):

IDLE warns about an out-of-date TK version. See the IDLE boot text.

Python 3.7.3 (default, Jun 19 2019, 07:38:49)
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license()" for more information.
WARNING: The version of Tcl/Tk (8.5.9) in use may be unstable. Visit
http://www.python.org/download/mac/tcltk/ for current information.
>>>

Then, scrolling causes the IDLE GUI window to crash.
If running IDLE directly from its .app, the app crashes/quits. Since we ran from the .app, there's no exception/strack trace to look at.
If running IDLE from the command line, this is the output upon the scroll-crash:

➜  ~ idle
2019-06-25 15:41:14.272 Python[46084:348809] ApplePersistenceIgnoreState: Existing state will not be touched. New state will be written to (null)
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/__main__.py", line 7, in <module>
    idlelib.pyshell.main()
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/idlelib/pyshell.py", line 1572, in main
    root.mainloop()
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tkinter/__init__.py", line 1283, in mainloop
    self.tk.mainloop(n)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte

I first reported this over in the Python bug tracker, and it was found to be an installation issue with setting up an up-to-date version of TK for the python installation. Thus, that issue was closed and I am filing this homebrew issue.

The specific issue is that brew install python does not come with TK, so brew's python installation relies on the default Apple TK 8.5 version which has serious bugs compared to the up-to-date 8.6 TK versions. python.org's installation of Python comes bundled with TK 8.6.8 and does not reproduce the IDLE scroll crash bug.

This issue (python formula missing up-to-date TK) has already been reported before (#26880), but the solution given in a comment there is not valid since there is no longer an --with-tcl-tk option to install with TK.

What you expected to happen:

In order for TK-dependent things to work correctly, an up-to-date TK version would come bundled with the python formula, just like it comes with the python.org installation.

Thus, IDLE that comes with brew install python would be able to scroll without crashing.

Step-by-step reproduction instructions (by running brew install commands)

1. Open terminal
2. Run brew update && brew install python
3. Run idle or python -m idlelib
4. Switch to the IDLE app window
   
   
   
   1. At this point, observe the warning message about the TK version printed by the IDLE start text.
   
   
   2. Optionally check IDLE > "About IDLE" via the MacOS top bar to see the TK version in use.
   
   
5. Attempt to scroll the IDE using mouse wheel or touchpad scrolling
6. Observe that the IDLE GUI window crashed

System Info: MacOS Mojave 10.14.5 (18F132)
Python 3.7.3 (default, Jun 19 2019, 07:38:49)
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Tk version: 8.5.9
IDLE version: 3.7.3

[fxcoudert]

Starting with macOS Catalina, Apple's Tcl actually says:

WARNING: This version of tcl is included in macOS for compatibility with legacy software. 
In future versions of macOS the tcl runtime will not be available by 
default, and may require you to install an additional package.

which sounds like a clear sign that we need to move away from system Tcl/Tk.

[jonchang]

Sounds good. How do you think we should accomplish this? Checking for linkage against system tcl/tk in an audit?

[fxcoudert]

@jonchang I'd say let's not be systematic at first, and start working with formulas where there is breakage… like python.

[MikeMcQuaid]

This issue (python formula missing up-to-date TK) has already been reported before (#26880), but the solution given in a comment there is not valid since there is no longer an --with-tcl-tk option to install with TK.

Suggested solution is to bundle up-to-date TK with python formula just like python.org bundles TK with their installations.

It's worth looking at the commit where this was removed: https://github.com/Homebrew/homebrew-core/commit/a377978583a538110a1e6e2fd95530ec6f6e7abe

A patch (unmerged upstream) was needed for Tcl/tk to be used with our Python formula. This patch would need to be submitted and merged upstream before we would consider including it again in future.

If running IDLE directly from its .app, there is no error message on the crash.

Note https://docs.brew.sh/Acceptable-Formulae#stuff-that-builds-an-app

This is another reason against making this change. I would instead propose that these be removed from the Python formula and instead pushed into a Homebrew Cask for idle or similar instead.

[MikeMcQuaid]

This is another reason against making this change. I would instead propose that these be removed from the Python formula and instead pushed into a Homebrew Cask for idle or similar instead.

Note: we should add caveats if we do this and point people in the right direction to an alternative.

[gwpantazes]

@MikeMcQuaid Just in case this line was being misinterpreted because of the confusing way I wrote it:

If running IDLE directly from its .app, there is no error message on the crash.

The IDLE.app does indeed crash. I was just pointing out it doesn't have error output like if IDLE was run by other means, like if it was started from the command line as a python module (which prints the exception trace).

Sorry if that was already understood / unrelated and this was just extra noise. I've edited things to be more clear.

[alexchandel]

@MikeMcQuaid The best solution for Homebrew users is to reintroduce those patches and build python/python3 with them by default, rather than shipping the current buggy python build. We can submit the patches to python separately, and remove them when they're accepted.

[MikeMcQuaid]

@alexchandel Feel free to submit those patches to Python and once they are merged in an upstream branch we'll consider including them again.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.