Also tried (to no avail - same result):
pacman -S mingw-w64-i686-python3
pacman -S mingw-w64-i686-python3-pip
pip install https://github.com/pyinstaller/pyinstaller/archive/develop.zip

LeighKorbel on 14 Jul 2020

And this works just fine in a MINGW shell, just not MSYS2.....

https://stackoverflow.com/questions/62884958/issues-compiling-gtk-software-in-msys2-mingw-with-pyinstaller

LeighKorbel on 14 Jul 2020

I found this module: pipwin, that has unofficial compiled binaries for windows. Do the bellow.

pip install pipwin
pipwin install pyinstaller

kethan1 on 15 Jul 2020

👍3

Running the above completes without error, but it doesn't actually install pyinstaller:

# pipwin install pyinstaller /usr/lib/python3.8/site-packages/pipwin/command.py:65: UserWarning: Found a non Windows system. Package installation might not work. warn("Found a non Windows system. Package installation might not work.") Building cache. Hang on . . . Done Packagepyinstallerfound in cache Downloading package . . . https://download.lfd.uci.edu/pythonlibs/w3jqiv8s/PyInstaller-3.6-py2.py3-none-any.whl PyInstaller-3.6-py2.py3-none-any.whl [*] 2.8 MB / 2.8 MB @ 1.4 MB/s [##################] [100%, 0s left] Processing ./pipwin/PyInstaller-3.6-py2.py3-none-any.whl Collecting pefile>=2017.8.1 Downloading pefile-2019.4.18.tar.gz (62 kB) |████████████████████████████████| 62 kB 315 kB/s Requirement already satisfied: setuptools in /usr/lib/python3.8/site-packages (from PyInstaller==3.6) (47.1.1) Collecting pywin32-ctypes>=0.2.0 Downloading pywin32_ctypes-0.2.0-py2.py3-none-any.whl (28 kB) Collecting altgraph Downloading altgraph-0.17-py2.py3-none-any.whl (21 kB) Collecting future Downloading future-0.18.2.tar.gz (829 kB) |████████████████████████████████| 829 kB 3.4 MB/s Using legacy setup.py install for pefile, since package 'wheel' is not installed. Using legacy setup.py install for future, since package 'wheel' is not installed. Installing collected packages: future, pefile, pywin32-ctypes, altgraph, PyInstaller Running setup.py install for future ... done Running setup.py install for pefile ... done Successfully installed PyInstaller-3.6 altgraph-0.17 future-0.18.2 pefile-2019.4.18 pywin32-ctypes-0.2.0

And when I enter python and try to import pyinstaller I get:

python

Python 3.8.3 (default, May 18 2020, 08:39:23)
[GCC 9.3.0] on msys
Type "help", "copyright", "credits" or "license" for more information.

import pyinstaller
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'pyinstaller'

LeighKorbel on 15 Jul 2020

You are not supposed to import it, you are supposed to convert python files to exes to other file executables (such as .app).

kethan1 on 15 Jul 2020

Yes, but the point was to show that its not installed. When I go to convert the python scripts into executables using my makefile, it fails because PyInstaller is not installed. Anyway, I have PyInstaller working using Python3.7 using the normal route of just pip installing, but it clearly doesn't work for Python3.8 and I am not sure why jumping through hoops like this is necessary?

LeighKorbel on 15 Jul 2020

I got it working this way for python 3.8:

Install it from github (as a zip) and unzip it.
Then do cd bootloader
and then python setup.py install

kethan1 on 15 Jul 2020

There is no setup.py script in the bootloader folder... There is one in the root directory though. Running python setup.py install makes it seem like it is installed (the same as pipwin install pyinstaller), but when I type pyinstaller into the MSYS shell I get the all too familiar:

# pyinstaller
Your platform is not yet supported. Please define constant PYDYLIB_NAMES for your platform.

LeighKorbel on 15 Jul 2020

@LeighKorbel your import test didn't work; you have the wrong name. Use python -c "import PyInstaller" to test the install.

Legorooj on 16 Jul 2020

@LeighKorbel @kethan1 the issue here isn't the bootloader or PYDYLIB_NAMES actually; it's the platform test. We already semi-support MSYS2, except that we aren't testing for whether we're on MSYS2 properly it appears. Please submit a pull request that modifies compat.py so it uses the _Windows_ PYDYLIB_NAMES definition if running on MSYS2.

Legorooj on 16 Jul 2020

@Legorooj I'm with @LeighKorbel , but we're not entirely sure how to implement the changes you're mentioning, or even where. Would this be a simple fix, or something more involved? Thanks!

alexeast99 on 16 Jul 2020

@alexeast99 @LeighKorbel

On the _face_ of it, this shouldn't be too difficult a fix, but in attempting it I've hit unexpected snags. I'm hoping perhaps @Legorooj or someone else has some insights.

@Legorooj pointed at compat.py as the primary location for the changes that would need to be made, and indeed I found the issue there easily enough. Running the following in an MSYS2 session inside a Win7 VM produces the following, for me:

$ python3 -c 'import sys; import platform; print(sys.platform); 
print(sys.version_info[:2]); print(platform.architecture()); 
print(platform.win32_ver()); print(platform.system())'
msys
(3, 8)
('64bit', 'WindowsPE')
('', '', '', '')
MSYS_NT-6.1-7601

The platform.system() value is a bit odd, compared to other systems where it's simply "Windows" or "Linux". But other than that, pretty standard. The same thing in a MinGW64 shell produces this:

$  python3 -c 'import sys; import platform; print(sys.platform); 
print(sys.version_info[:2]); print(platform.architecture()); 
print(platform.win32_ver()); print(platform.system())'
win32
(3, 8)
('64bit', 'WindowsPE')
('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Windows

So, the platforms are clearly distinct as well, which is great.

As you can see in the msys-compat branch of my fork, I added detection of sys.platform.startswith('msys') to compat.py, which does allow the build process to proceed a bit farther. However, it still dies, and I'm having a bit of trouble figuring out _why_ (or even how) it's producing the traceback encountered. Here's the output of a build attempt, post- my changes to compat.py:

$ python3 ./setup.py build
running build
running build_bootloader
No precompiled bootloader found. Trying to compile it for you ...
Setting top to                           : /home/ferd/pyinstaller/bootloader
Setting out to                           : /home/ferd/pyinstaller/bootloader/build
Python Version                           : 3.8.4 (default, Jul 16 2020, 11:10:12) [GCC 9.3.0]
Checking for 'clang' (C compiler)        : not found
Checking for 'gcc' (C compiler)          : /usr/bin/gcc
Traceback (most recent call last):
  File "/home/ferd/pyinstaller/bootloader/.waf3-2.0.20-a3a5949fb45bcd86794262f86d7977a7/waflib/Scripting.py", line 119, in waf_entry_point
    run_commands()
  File "/home/ferd/pyinstaller/bootloader/.waf3-2.0.20-a3a5949fb45bcd86794262f86d7977a7/waflib/Scripting.py", line 182, in run_commands
    ctx=run_command(cmd_name)
  File "/home/ferd/pyinstaller/bootloader/.waf3-2.0.20-a3a5949fb45bcd86794262f86d7977a7/waflib/Scripting.py", line 173, in run_command
    ctx.execute()
  File "/home/ferd/pyinstaller/bootloader/.waf3-2.0.20-a3a5949fb45bcd86794262f86d7977a7/waflib/Configure.py", line 85, in execute
    super(ConfigurationContext,self).execute()
  File "/home/ferd/pyinstaller/bootloader/.waf3-2.0.20-a3a5949fb45bcd86794262f86d7977a7/waflib/Context.py", line 92, in execute
    self.recurse([os.path.dirname(g_module.root_path)])
  File "/home/ferd/pyinstaller/bootloader/.waf3-2.0.20-a3a5949fb45bcd86794262f86d7977a7/waflib/Context.py", line 133, in recurse
    user_function(self)
  File "/home/ferd/pyinstaller/bootloader/wscript", line 403, in configure
    DESTOS_TO_SYSTEM[ctx.env.DEST_OS])
KeyError: 'cygwin'
ERROR: Failed compiling the bootloader. Please compile manually and rerun setup.py

I can't figure out where the cygwin key is even coming from, to be raising a KeyError exception in the DESTOS_TO_SYSTEM lookup. There's nothing in the actual MSYS2 Python environment parameters that I can find that would return "cygwin", so I'm kind of at a loss.

I could certainly try adding cygwin to DESTOS_TO_SYSTEM, the same way I've already added msys, but I don't know what value I'd even set for that key and it just feels like I'm flailing around in the dark.

Any thoughts?

ferdnyc on 16 Jul 2020

Well, I updated DESTOS_TO_SYSTEM with an additional mapping from cygwin to platform.system() , despite having no idea why that would or should work, but it does appear to nevertheless. The build completes successfully in an MSYS2 shell with the latest Python 3.8.4:

$ python3 ./setup.py build
running build
running build_bootloader
No precompiled bootloader found. Trying to compile it for you ...
Setting top to                           : /home/ferd/pyinstaller/bootloader
Setting out to                           : /home/ferd/pyinstaller/bootloader/build
Python Version                           : 3.8.4 (default, Jul 16 2020, 11:10:12) [GCC 9.3.0]
Checking for 'clang' (C compiler)        : not found
Checking for 'gcc' (C compiler)          : /usr/bin/gcc
System                                   : Assuming cross-compilation for MSYS_NT-6.1-7601
Checking size of pointer                 : 8
Platform                                 : MSYS_NT-6.1-7601-64bit detected based on compiler
Checking for compiler flags -m64         : yes
Checking for linker flags -m64           : yes
Checking for library dl                  : yes
Checking for library m                   : yes
Checking for library z                   : yes
Checking for library cmocka              : not found
Checking for function unsetenv           : yes
Checking for function mkdtemp            : yes
Checking for function dirname            : yes
Checking for function basename           : yes
Checking for function strndup            : yes
Checking for function strnlen            : yes
Checking for program '/usr/bin/strip'    : /usr/bin/strip
Checking for program 'strip'             : /usr/bin/strip
'configure' finished successfully (13.396s)
'all' finished successfully (0.001s)
'distclean' finished successfully (0.017s)
Setting top to                           : /home/ferd/pyinstaller/bootloader
Setting out to                           : /home/ferd/pyinstaller/bootloader/build
Python Version                           : 3.8.4 (default, Jul 16 2020, 11:10:12) [GCC 9.3.0]
Checking for 'clang' (C compiler)        : not found
Checking for 'gcc' (C compiler)          : /usr/bin/gcc
System                                   : Assuming cross-compilation for MSYS_NT-6.1-7601
Checking size of pointer                 : 8
Platform                                 : MSYS_NT-6.1-7601-64bit detected based on compiler
Checking for compiler flags -m64         : yes
Checking for linker flags -m64           : yes
Checking for library dl                  : yes
Checking for library m                   : yes
Checking for library z                   : yes
Checking for library cmocka              : not found
Checking for function unsetenv           : yes
Checking for function mkdtemp            : yes
Checking for function dirname            : yes
Checking for function basename           : yes
Checking for function strndup            : yes
Checking for function strnlen            : yes
Checking for program '/usr/bin/strip'    : /usr/bin/strip
Checking for program 'strip'             : /usr/bin/strip
'configure' finished successfully (6.759s)
'make_all' finished successfully (0.030s)
Waf: Entering directory `/home/ferd/pyinstaller/bootloader/build/debug'
[ 1/12] Compiling src/pyi_pythonlib.c
[ 2/12] Compiling src/pyi_launch.c
[ 3/12] Compiling src/pyi_python.c
[ 4/12] Compiling src/pyi_main.c
[ 5/12] Compiling src/pyi_utils.c
[ 6/12] Compiling src/pyi_global.c
[ 7/12] Compiling src/pyi_archive.c
[ 8/12] Compiling src/pyi_path.c
[ 9/12] Compiling src/pyi_win32_utils.c
[10/12] Compiling src/main.c
[11/12] Linking build/debug/run_d.exe
[12/12] Processing build/debug/run_d.exe
Waf: Leaving directory `/home/ferd/pyinstaller/bootloader/build/debug'
'build_debug' finished successfully (5.120s)
Waf: Entering directory `/home/ferd/pyinstaller/bootloader/build/release'
[ 1/12] Compiling src/pyi_pythonlib.c
[ 2/12] Compiling src/pyi_launch.c
[ 3/12] Compiling src/pyi_archive.c
[ 4/12] Compiling src/pyi_python.c
[ 5/12] Compiling src/pyi_utils.c
[ 6/12] Compiling src/pyi_global.c
[ 7/12] Compiling src/pyi_main.c
[ 8/12] Compiling src/pyi_path.c
[ 9/12] Compiling src/pyi_win32_utils.c
[10/12] Compiling src/main.c
[11/12] Linking build/release/run.exe
[12/12] Processing build/release/run.exe
Waf: Leaving directory `/home/ferd/pyinstaller/bootloader/build/release'
'build_release' finished successfully (2.908s)
Waf: Entering directory `/home/ferd/pyinstaller/bootloader/build/debug'
+ install /home/ferd/pyinstaller/PyInstaller/bootloader/MSYS_NT-6.1-7601-64bit/run_d.exe (from build/debug/run_d.exe)
[13/13] Processing build/debug/run_d.exe
Waf: Leaving directory `/home/ferd/pyinstaller/bootloader/build/debug'
'install_debug' finished successfully (0.206s)
Waf: Entering directory `/home/ferd/pyinstaller/bootloader/build/release'
[12/13] Processing build/release/run.exe
+ install /home/ferd/pyinstaller/PyInstaller/bootloader/MSYS_NT-6.1-7601-64bit/run.exe (from build/release/run.exe)
Waf: Leaving directory `/home/ferd/pyinstaller/bootloader/build/release'
'install_release' finished successfully (0.192s)
running build_py
creating build
creating build/lib
creating build/lib/PyInstaller
copying PyInstaller/compat.py -> build/lib/PyInstaller
copying PyInstaller/config.py -> build/lib/PyInstaller
copying PyInstaller/configure.py -> build/lib/PyInstaller
copying PyInstaller/exceptions.py -> build/lib/PyInstaller
copying PyInstaller/log.py -> build/lib/PyInstaller
copying PyInstaller/__init__.py -> build/lib/PyInstaller
copying PyInstaller/__main__.py -> build/lib/PyInstaller
running egg_info
creating pyinstaller.egg-info
writing pyinstaller.egg-info/PKG-INFO
writing dependency_links to pyinstaller.egg-info/dependency_links.txt
writing entry points to pyinstaller.egg-info/entry_points.txt
writing requirements to pyinstaller.egg-info/requires.txt
writing top-level names to pyinstaller.egg-info/top_level.txt
writing manifest file 'pyinstaller.egg-info/SOURCES.txt'
reading manifest file 'pyinstaller.egg-info/SOURCES.txt'

(...etc, etc.)

So, with the (very minor) changes in my msys-compat branch, that's available for anyone looking to make use of it. I'm not real wild about the idea of submitting a _pull request_ when I don't even remotely understand why the changes work in the first place, but the branch is there and you're welcome to pick out any or all of the four lines of changes, if they can be of any help.

ferdnyc on 16 Jul 2020

(Hmm, I just noticed from the output that it thinks it's cross-compiling. I wonder if that'll be a problem?)

ferdnyc on 16 Jul 2020

Could cygwin be coming from the MSYS2 runtime library msys2.0-dll?

From: https://www.msys2.org/wiki/How-does-MSYS2-differ-from-Cygwin/

MSYS2 tries to provide an environment for building native Windows software. MSYS2 provides a large collection of packages containing such software, and libraries for their development. As a large portion of the software uses GNU build tools which are tightly coupled to the unix world, this environment is also POSIX-compatible, and is in fact based on Cygwin.

In any case, yes, I will take a look at your branch this evening and report back regarding the functionality. Many thanks!

LeighKorbel on 17 Jul 2020

Huh. Well I am not quite sure what to make of this. I used your branch and was able to successfully compile PyInstaller in MSYS2 using Python 3.8. Everything seemed to look good. I can see that it is installed in the correct location:

C:\msys64\usr\lib\python3.8\site-packages\pyinstaller-4.0.dev0-py3.8.egg\PyInstaller

When I type in pyinstaller I get:

# pyinstaller
usage: pyinstaller [-h] [-v] [-D] [-F] [--specpath DIR] [-n NAME] [--add-data <SRC;DEST or SRC:DEST>] [--add-binary <SRC;DEST or SRC:DEST>]
[-p DIR] [--hidden-import MODULENAME] [--additional-hooks-dir HOOKSPATH] [--runtime-hook RUNTIME_HOOKS]
[--exclude-module EXCLUDES] [--key KEY] [-d {all,imports,bootloader,noarchive}] [-s] [--noupx] [--upx-exclude FILE] [-c] [-w]
[-i <FILE.ico or FILE.exe,ID or FILE.icns>] [--version-file FILE] [-m <FILE or XML>] [-r RESOURCE] [--uac-admin]
[--uac-uiaccess] [--win-private-assemblies] [--win-no-prefer-redirects] [--osx-bundle-identifier BUNDLE_IDENTIFIER]
[--runtime-tmpdir PATH] [--bootloader-ignore-signals] [--distpath DIR] [--workpath WORKPATH] [-y] [--upx-dir UPX_DIR] [-a]
[--clean] [--log-level LEVEL]
scriptname [scriptname ...]
pyinstaller: error: the following arguments are required: scriptname

So everything looks good there. But then i run my normal makefile and get this:

The version of MSYS you are running is 3 (0 meaning not MSYS at all) cmd //C pyinstaller src/python/plotter.py -y -w 'pyinstaller' is not recognized as an internal or external command, operable program or batch file. make: *** [makefile:73: dist/plotter/plotter.exe] Error 1

Which is a complete headscratcher, all things considered. The compile line from the makefile that has always done the job is:

ifeq ($(msys_version), 0)
define GEN_RULE
dist/$(script)/$(script)$(PY_EXT) : src/python/$(script).py
python3.6 -m PyInstaller src/python/$(script).py -y
rm -rf build
endef
else
define GEN_RULE
dist/$(script)/$(script)$(PY_EXT) : src/python/$(script).py
cmd //C pyinstaller src/python/$(script).py -y -w
rm -rf build
endef
endif

So I'm not sure what to make of this. I expected it to run, and the fact that it is complaining that there is no command pyinstaller when I can run it in the MSYS2 shell is confounding...

LeighKorbel on 17 Jul 2020

Derp. Looks like I need to update PATH in .bashrc:

https://stackoverflow.com/questions/45951964/pyinstaller-is-not-recognized-as-internal-or-external-command

LeighKorbel on 17 Jul 2020

Yeah. I am stuck. Normally the $PATH would be where the pyinstaller.exe is installed, which if it were installed via pip to a Windows install it would be:

export PYTHON_HOME=C:\\Users\\Leigh\\AppData\\Local\\Programs\\Python\\Python38
export PATH="$PYTHON_HOME:$PYTHON_HOME\\Scripts:$PATH"

There is not an executable for pyinstaller in C:\mys64\usr\bin, just a 1kb file. But that has to be it, so I:

# export PATH=C:\\msys64\\usr\\bin
# echo $PATH
C:\msys64\usr\bin

And still get the same issue. If that isn't it, I am not sure what $PATH could possibly be...

LeighKorbel on 17 Jul 2020

The issue was with the makefile. It starts to run now but fails:

9275 INFO: Python library not in binary dependencies. Doing additional searching...
Traceback (most recent call last):
  File "\msys64\usr\bin/pyinstaller", line 11, in <module>
    load_entry_point('pyinstaller==4.0.dev0', 'console_scripts', 'pyinstaller')()
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/building/build_main.py", line 716, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/building/build_main.py", line 663, in build
    exec(code, spec_namespace)
  File "/home/Leigh/Cybercyte/CyberSolver/plotter.spec", line 6, in <module>
    a = Analysis(['src/python/plotter.py'],
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/building/build_main.py", line 241, in __init__
    self.__postinit__()
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/building/build_main.py", line 474, in assemble
    self._check_python_library(self.binaries)
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/building/build_main.py", line 563, in _check_python_library
    python_lib = bindepend.get_python_library_path()
  File "/usr/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg/PyInstaller/depend/bindepend.py", line 948, in get_python_library_path
    raise IOError(msg)
OSError: Python library not found: python38.dll, libpython3.8.dll, libpython3.8m.dll, libpython38m.dll, libpython38.dll
    This would mean your Python installation doesn't come with proper library files.
    This usually happens by missing development package, or unsuitable build parameters of Python installation.

    * On Debian/Ubuntu, you would need to install Python development packages
      * apt-get install python3-dev
      * apt-get install python-dev
    * If you're building Python by yourself, please rebuild your Python with `--enable-shared` (or, `--enable-framework` on Darwin)](url)

This looks similar to issue https://github.com/pyinstaller/pyinstaller/issues/3167 from some time back with Python 3.6. There are no development packages for Python available in MSYS2, and trying to compile python 3.8 from source failed at make:

ModuleNotFoundError: No module named '_sysconfigdata__msys_nt-10'

So I imagine there is more involved than just the updates to compat.py and wscript...

LeighKorbel on 17 Jul 2020

@LeighKorbel

Sorry, that's my fault. The MSYS Python library names have the form /usr/bin/msys-python3.8.dll. I just updated compat.py in my branch, with the latest changes pyinstaller -d all script.py successfully creates a dist/script/ directory containing the packaged executable files.

There is the wrinkle that, in the MSYS environment, many Python modules aren't available. For example, I tested with an app that has PyQt5 dependencies, but there's no MSYS PyQt5. There's a MinGW64 PyQt5 package available through pacman, which installs to /mingw64/lib/python3.8/site-packages/PyQt5, but that can't be used to build MSYS packages. So I imagine the Windows PyQt5 distribution would have to be installed, and the PYTHON_PATH set accordingly. Ditto for any other dependencies.

But pyinstaller itself _seems_ functional, after my latest commit.

ferdnyc on 18 Jul 2020

Oh, I should also mention how I installed it:

python3 ./setup.py bdist_wheel
pip3 install dist/pyinstaller-4.0.dev0-py3-none-any.whl

That correctly installed the console entry_point scripts as /usr/bin/pyinstaller and friends.

ferdnyc on 18 Jul 2020

@ferdnyc Thanks again for your continued assistance. Unfortunately, this still failed, but this time I am confused as to why. I would have suspected the most recent commit on your branch would have resolved this issue, but I am still met with less similar issues. The first time I got:

Python library not found: python38.dll, libpython3.8.dll, libpython3.8m.dll, libpython38m.dll, libpython38.dll

After your most recent patch I got:

Python library not found: libpython38.dll, libpython38m.dll, python38.dll

So now there are two fewer dll's missing. I am confused as to why libpython38.dll and libpython38m.dll were not captured by your most recent definition for PYDYLIB_NAMESwhen is_msys. I naively tried to tack on the three from is_win or is_cygwin but that also did not do the trick:

elif is_msys:
    # For MSYS2 environment
    PYDYLIB_NAMES = {
        'python%d%d.dll' % _pyver,
        'libpython%d%d.dll' % _pyver,
        'libpython%d%dm.dll' % _pyver,
        'msys-python%d%d.dll' % _pyver,
        'msys-python%d.%d.dll' % _pyver,
        'msys-python%d%dm.dll' % _pyver,
        'msys-python%d.%dm.dll' % _pyver,
    }

I also found that it didn't matter if I created a wheel and installed from that or just simply did python setup.py install.

LeighKorbel on 18 Jul 2020

@LeighKorbel

The shorter list makes sense, as I reduced the list of names matches for is_win / is_cygwin when I split out is_msys. Your output would seem to indicate that the build is still using the is_win list, not is_msys, oddly.

Can you share the output of this Python command in your MSYS shell? Maybe the detection is different.

$ python3 -c "import sys; import platform; print(sys.platform);"\
"print(sys.version_info[:2]); print(platform.architecture());"\
"print(platform.win32_ver()); print(platform.system())"
msys
(3, 8)
('64bit', 'WindowsPE')
('', '', '', '')
MSYS_NT-6.1-7601

ferdnyc on 18 Jul 2020

Also note, PyInstaller is looking for ANY of the listed libraries, not all of them. The message indicates that located none of listed names.

Various different builds of libpython are named differently, but all will work
So it searches for all possibilities, and only needs to find one of the names listed.

ferdnyc on 18 Jul 2020

Not much difference from yours @ferdnyc:

# python3 -c "import sys; import platform; print(sys.platform);"\
> "print(sys.version_info[:2]); print(platform.architecture());"\
> "print(platform.win32_ver()); print(platform.system())"
msys
(3, 8)
('64bit', 'WindowsPE')
('', '', '', '')
MSYS_NT-10.0-18362

LeighKorbel on 18 Jul 2020

Clearly it would seem that there is a difference in the MSYS2 shell. I am using the most up to date version from https://www.msys2.org/. Perhaps there are now differences in the runtime library?

LeighKorbel on 18 Jul 2020

Mmm, I'm running Windows 7 in a VM. But the thing that matters is how Python identifies itself, and on that front I agree, it seems the same.

Unless maybe the Win10 detection is overriding the MSYS detection. I'm on my phone right now, but when I get home I'll take another look at compat.py and see if I can find anything.

ferdnyc on 18 Jul 2020

Nope, just checked, Win10 depends on Windows which based on your Python environment wouldn't be detected. I did update my branch to add an is_py38 variable, but it's not used anywhere currently so it shouldn't matter.

The only thing I can think of is, maybe because I added that is_cygwin mapping to DESTOS_TO_SYSTEM, maybe it's causing your environment to be detected as cygwin instead of MSYS, but I don't know why that would happen for you and not me.

Here's what I'd suggest: do a clean rebuild and install. In particular, uninstall PyInstallet, and make sure that when you're done, typing pyinstaller at a shell prompt results in "command not found". Then, clone my msys-compat branch to a new directory, build PyInstaller as a wheel, install it with pip, and try it again.

Oh, if you're going to be using that Makefile you mentioned, don't run PyInstaller with CMD. That WILL cause the system to be identified as Windows, which will break things.

ferdnyc on 18 Jul 2020

If you want to build a Windows PyInstaller, then you should execute setup.py using a Windows Python binary, like C:\Program Files\Python 3.8\bin\python.exe or whatever, and none of this MSYS stuff will matter at all. Just make sure that libpython38.dll from that distribution is somewhere on the path.

(EDIT: Note that I haven't tried the above yet, as I'm still in transit. Details may be off.)

ferdnyc on 18 Jul 2020

No, that is exactly what I am trying to avoid. I want everything to be contained within the MSYS2 environment. The last oversight with CMD was, in fact, due to me previously pointing to a Windows Python binary in $PATH. Everything defaults to Python 3.8 currently in MSYS2, and that is the precise reason why I would like to see PyInstaller working there, since everything else Python related should become hopefully trivial after that with it all installing to MSYS2. The particular problem is pygobject. From what I can gather, I can't simply install that to Windows (so what I did before without it is no longer valid), and I need the MSYS2 environment and the pygobject library to pull that off. So the only thing holding me back is PyInstaller working correctly with the current default distro of Python with MSYS2. If I had that, then I can have everything self contained in the MSYS2 environment. I expect that would allow me to get my GTK app running in Windows.

I will try your suggestions shortly. Everything you have said so far seems spot on, so it could be an oversight on my part. I'll report back, and again, I appreciate the assistance!

LeighKorbel on 18 Jul 2020

Yes, that worked! It was clearly a mistake on my end. This is running with Python 3.8 in MSYS2 now. Well done! I really appreciate you tackling this in such a timely fashion. I can't thank you enough. I just need to figure out how to get pygobject working with it now, but that's another matter. If you can flesh out the details that you are unsure about, I absolutely think this branch is worthy of a pull request. I have come across more than a handful of posts on Stack Overflow with people trying to get PyInstaller working in MSYS2 with Python 3.8, so the demand is there.

LeighKorbel on 18 Jul 2020

@ferdnyc yes, please submit a PR.

Legorooj on 19 Jul 2020

@LeighKorbel @Legorooj Will do, at least for the compat.py changes. The wscript changes... I want to look those over a bit more. Whether or not I really understand why we're forced down the cygwin path there, it seems like there are other things in wscript that are currently limited to the win32 path, but should probably be included in the MSYS path as well. (Or the cygwin path, as the case may be.)

I'd also _like_ to move the build_bootloader output files to the /build/ directory in the root of the project, instead of building in /bootloader/ and installing into /PyInstaller/bootloader/ (polluting the source tree, basically). I'd do something that big as a separate PR, of course, but would you be amenable to that change, @Legorooj ?

ferdnyc on 20 Jul 2020

@ferdnyc the second change I don't think is necessary; we want the bootloaders in the source tree.

As for the other changes, just open a PR, mark it as a draft, and ask for reviews. It's easier to review a PR then a branch

Legorooj on 20 Jul 2020

@ferdnyc As I mentioned before, the need for PyInstaller with Python 3.8 was all in regards to compiling software containing some Python GTK code in it (which requires using a MinGW shell, unfortunately). I had to build the bootloader prior to your previous instructions in a MinGW shell:

cd pyinstaller/bootloader
python3 ./waf --gcc distclean all
cd ..
python3 ./setup.py bdist_wheel
pip3 install dist/pyinstaller-4.0.dev0-py3-none-any.whl

Which got me past the issue of MinGW complaining about not having one. However, PyInstaller gets to the point where it is loading the module hook "hook-gi.repository.Gtk.py" and then fails right after. I am almost certain that it successfully loads the hooks, even though it takes an eternity, as I threw a print statement at the end of the python script and it came up. The failure is:

4391 INFO: Processing pre-safe import module hook gi.repository.Gtk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gtk.py'.
4391 INFO: Processing module hooks...
4391 INFO: Loading module hook 'hook-distutils.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
4391 INFO: Loading module hook 'hook-encodings.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
4470 INFO: Loading module hook 'hook-gi.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
4485 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
Unable to find "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Anaconda3/pkgs/backports.shutil_get_terminal_size-1.0.0-py36h79ab834_2/Lib/site-packages/backports.shutil_get_terminal_size-1.0.0-py3.6.egg-info/dependency_links.txt" when adding binary and data files.
make: *** [makefile:73: dist/plotter/plotter.exe] Error 1

The reason I am asking this here is because I am not sure if this has anything to do with system definitions in the module hooks. For example in hook-gi.repository.Gtk.py there is:

from PyInstaller.compat import is_win

# these only seem to be required on Windows
if is_win:
    datas += collect_glib_etc_files('fonts')
    datas += collect_glib_etc_files('pango')
    datas += collect_glib_share_files('fonts')

Also the fact that PyInstaller says it cannot find that file is a little confusing. If I go into Windows Command Prompt I can locate that exact file in that exact path. Since ProgramData/Application Data is a junction point, that was the only way I could see what existed there.

Long story short. I am not sure if this issue is a PyInstaller issue on your branch or if this is something altogether different?

LeighKorbel on 21 Jul 2020

@LeighKorbel

OK, yeah, that's exactly the kind of stuff I was afraid of:

it seems like there are other things in wscript that are currently limited to the win32 path, but should probably be included in the MSYS path as well. (Or the cygwin path, as the case may be.)

So it sounds like it's not even just wscript, but the _rest_ of pyinstaller that's doing certain things only when is_win is true. _Possibly_ is_win just should be set true, on MSYS, but that assumes that _all_ of the is_win actions are also valid in MSYS. I'd be surprised if that's the case, and if it's _not_ then the ones that do apply to both will need to be changed to if is_win or is_msys:.

ferdnyc on 22 Jul 2020

👍1

@ferdnyc I should stress that this is occurring now using a MinGW shell (as opposed to the MSYS2 shell I was working in before), because pygobject requires this:

https://pygobject.readthedocs.io/en/latest/getting_started.html

So I suppose it should come as no surprise that is_win triggers here given the shell that I am using. I do not understand the subtle connections between the MSYS2 and MinGW shells. MinGW does not contain Linux system headers and my code will not compile in there as a result, but they do exist with MSYS2. If I compile in MSYS2 first, then it will suddenly compile in MinGW. Why that is the case is unbeknownst to me. I mentioned adding print statements to the python hook for GTK before. This looks like:

# these only seem to be required on Windows
if is_win:
    print("Begin is_win")
    datas += collect_glib_etc_files('fonts')
    datas += collect_glib_etc_files('pango')
    datas += collect_glib_share_files('fonts')
    print("End is_win")

print("GTK hook complete")

And the output I get when running my makefile is:

5047 INFO: Loading module hook 'hook-gi.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
5078 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
Begin is_win
End is_win
GTK hook complete
Unable to find "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Anaconda3/pkgs/backports.shutil_get_terminal_size-1.0.0-py36h79ab834_2/Lib/site-packages/backports.shutil_get_terminal_size-1.0.0-py3.6.egg-info/dependency_links.txt" when adding binary and data files.
make: *** [makefile:73: dist/plotter/plotter.exe] Error 1

I would assume that there are a bunch of other module hooks that need to be loaded, but I suspect there would be another print to the console from PyInstaller stating that it is attempting to load one if a fail was coming from within a hook. This appears to be happening in between loads of module hooks where the failure to find a file occurs. The path is quite strange with the repetition of "Application Data" subdirectories, but I can navigate that exact path in Windows CMD and find that exact file there, so I am not sure why PyInstaller says it can't find it.

So in summary, this error might not have anything to do with system paths.

LeighKorbel on 22 Jul 2020

Ah, thanks @LeighKorbel , I hadn't caught on that the latest results were from MinGW.

I may have an idea what's happening here.

$ echo "C:/ProgramData/Application Data/Application Data/Application Data/\
Application Data/Application Data/Anaconda3/pkgs/\
backports.shutil_get_terminal_size-1.0.0-py36h79ab834_2/Lib/\
site-packages/backports.shutil_get_terminal_size-1.0.0-py3.6.egg-info/\
dependency_links.txt" | wc -c
266

Heh. Yup.

Windows file-access APIs, the older pre-universal-whatever ones that didn't show up until like Windows 7 or 8 I think, had a hard cap of 256 characters for a complete file path. MinGW is clearly using those older APIs. there is simply no way it can read a file located at "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Anaconda3/pkgs/backports.shutil_get_terminal_size-1.0.0-py36h79ab834_2/Lib/site-packages/backports.shutil_get_terminal_size-1.0.0-py3.6.egg-info/dependency_links.txt".

ferdnyc on 23 Jul 2020

MinGW is clearly using those older APIs.

MinGW, or MinGW Python, I should say. It's possible there's some alternate file-access module that supports the ~~UWP (that's what they're called!!)~~ (so, nothing to do with UWP really) APIs and doesn't have restrictions on path length, though you'd think if there were it would be used by default in Python 3.8...

Hmm. I was wrong. It's 260 characters, it wasn't fixed until Windows 10, and apparently it's a _filesystem_ setting that can be changed in the Registry!? Presumably, if you can navigate to that file in Explorer, the default has already been changed by Microsoft, but here are some three-year-old instructions on how to overcome this, in case you want to double-check your configuration: https://www.howtogeek.com/266621/how-to-make-windows-10-accept-file-paths-over-260-characters/

ferdnyc on 23 Jul 2020

Clever. I had no idea. So it would seem that this is a MinGW issue and not a PyInstaller issue? This is the last hurdle holding me up and if I can get past this it would make my week. I am going to attempt your suggestion now and see what I can do. I can't thank you enough for your help so far @ferdnyc!

LeighKorbel on 23 Jul 2020

Yeah, I can't imagine how it's PyInstaller's fault, if the path can't be read the path can't be read. Finding some way to shorten it would be the most effective approach to solving this. It sounds like something went wrong with that Anaconda installation, if it's really located its packages directory at "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Anaconda3/" That's just craziness. Maybe there's some way to convince it that "C:/ProgramData/Application Data/Anaconda3/" would be a perfectly fine path to use?

(Edit: Can I just say, as an aside, that 260 is one of the most BIZARRE values I have _ever_ seen, for a maximum number of something.)

ferdnyc on 23 Jul 2020

I pretty much avoid Windows for coding as a policy, but I did install Anaconda Spyder on here (Windows 10) in case I didn't want to switch over to the Linux partition for anything Python related. I completely agree that the path is insane. I would think that the file path would be good on Windows 10 based on what you are telling me, so I don't know if the howtogeek link is relevant?

LeighKorbel on 23 Jul 2020

Also, because ProgramData/Application Data is a system junction I can't seem to navigate that path outside of CMD...

LeighKorbel on 23 Jul 2020

I would think that the file path would be good on Windows 10 based on what you are telling me, so I don't know if the howtogeek link is relevant?

Yeah, if you can read it from Explorer then I assume that Microsoft has already pushed the change described there out to everyone as the new default, so it shouldn't be necessary to change anything. I imagine if it _weren't_ already changed then you wouldn't be able to reach the file period.

Also, because ProgramData/Application Data is a system junction I can't seem to navigate that path outside of CMD...

Oh, now THAT'S interesting. Hmm. Wonder if the problem is a circular junction mapping? Can you try just running type "C:\ProgramData\Application Data\Anaconda3\pkgs\backports.shutil_get_terminal_size-1.0.0-py36h79ab834_2\Lib\site-packages\backports.shutil_get_terminal_size-1.0.0-py3.6.egg-info/dependency_links.txt" in a CMD window and see if it finds the same file? If it _does_, then the problem is just that your C:\ProgramData\Application Data\ has gotten looped around to reference _itself_. (Most likely, the C:\ProgramData\Application Data\ junction point was accidentally copied to C:\ProgramData\Application Data\Application Data\, and became an endless black hole). I wouldn't have the first clue how to fix that, though.

ferdnyc on 23 Jul 2020

The system cannot find the file specified.
So... a couple of things come to mind here. First, I wonder if this dependency would be resolved by something else? What would happen if I were to uninstall Anaconda from my computer? Also, I could try repeating this process from another computer that does not have Anaconda installed?

LeighKorbel on 23 Jul 2020

There are a lot of subtleties going on here that I did not anticipate. I was really hoping to have everything self contained within the MSYS2 environment (which is why I needed PyInstaller working with the native Python 3.8 there) and not have to depend on anything within Windows, but the python GTK requirement of MinGW really put a stop to that.

LeighKorbel on 23 Jul 2020

Or... actually, wait. Is there _supposed to be_ a C:\ProgramData\Application Data\ at all?

Answer, apparently, is:

Yes, it's "normal" (as normal as Windows gets)
It's actually just a circular reference back to C:\ProgramData\
That is incredibly stupid
But, we're back to this maybe being PyInstaller's fault, since to fall down that hole it would have to be recursively scanning C:\ProgramData\ or something like that.

Unless maybe the path is stored somewhere in its expanded form, and could be shortened at the source. But basically wherever you see C:\ProgramData\[Application Data\ × N], _all_ of the Application Data repetitions can be removed. So Anaconda3 is really just at C:\ProgramData\Anaconda3.

ferdnyc on 23 Jul 2020

I mentioned previously that I can navigate that exact path and find that exact file in that location within CMD. Is it Anaconda that placed it there, or something else?

LeighKorbel on 23 Jul 2020

That's one question, the other one is why PyInstaller needs to read it at all. I need to go back over your earlier post on this, I've sort of been reacting to details without an understanding of the bigger picture. Hopefully I can find some clues in the process that led you to this point.

ferdnyc on 23 Jul 2020

It seems that the Anaconda Spyder install falls under "Python 3.6.4 (Anaconda3 5.1.0 64-bit) under Apps & Features. I am going to uninstall it now and see what happens.

LeighKorbel on 23 Jul 2020

It says it will be uninstalling from: C:\ProgramData\Anaconda3\
That's interesting.

LeighKorbel on 23 Jul 2020

Ugh. So I uninstalled that. Then I uninstalled PyInstaller via pip in MinGW, then remade the wheel and installed PyInstaller again. Now when I go to compile my app I get this:

5246 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
Unable to find "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Apple/Installer Cache/Apple Mobile Device Support 11.0.0.30/AppleMobileDeviceSupport6464.msi" when adding binary and data files.

Apple Mobile Device Support....?

LeighKorbel on 23 Jul 2020

And an MSI is an installer for Windows. Why in the world would that come up? This is making less sense by the minute.

LeighKorbel on 23 Jul 2020

I'm going to try this same process on three more computers tomorrow. I'll report back.

LeighKorbel on 23 Jul 2020

Yeah, it really seems like PyInstaller is picking up paths from _somewhere_ and falling down the circular reference hole created by the Application Data junction. Like, I bet there really _is_ an Apple\Installer Cache\Apple Mobile Device Support 11.0.0.30\AppleMobileDeviceSupport6464.msi _somewhere_, either in C:\ProgramData or your C:\Users\username\Application Data\{Local|Roaming|the-third-one}\. But there still must be something weird going on with "The Process" if it feels the need to look up completely unrelated files in the Application Data cache.

I haven't had a chance to look at your original info about the MinGW attempt, I was grabbing dinner and whatnot. But I'll take a look at that now, along with the relevant PyInstaller code that would be executing in that scenario. Hopefully something will jump out at me.

ferdnyc on 23 Jul 2020

(1) Another computer on Windows 10 that also had Anaconda installed:

15731 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
Unable to find "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Anaconda3/pkgs/backports.shutil_get_terminal_size-1.0.0-py36h79ab834_2/Lib/site-packages/backports.shutil_get_terminal_size-1.0.0-py3.6.egg-info/dependency_links.txt" when adding binary and data files.
make: *** [makefile:73: dist/plotter/plotter.exe] Error 1

Removed Anaconda:

23294 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
Unable to find "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Adobe/Setup/{AC76BA86-7AD7-1033-7B44-AC0F074E4100}/AcroRdrDCUpd1801120035.msp" when adding binary and data files.
make: *** [makefile:73: dist/plotter/plotter.exe] Error 1

Removed Adobe Acrobat Reader:

12349 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
Unable to find "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Microsoft.SqlServer.Compact.400.32.bc" when adding binary and data files.
make: *** [makefile:73: dist/plotter/plotter.exe] Error 1

(2) Another computer on Windows 10 that did not have Anaconda installed:

5210 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
Unable to find "C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Apple/Installer Cache/Apple Software Update 2.2.0.150/AppleSoftwareUpdate.msi" when adding binary and data files.

Note that this seems to be going in Alphabetical order. Anaconda3, then either Adobe or Apple. The Microsoft SQL server thing is the outlier.

(3) A fourth computer on Windows 10 that did not have Anaconda installed. This one acted differently. I repeated it twice to make sure it was the same. In both instances it got past the module hook for GTK and got all the way to the final step with PyInstaller before failing. The first run gave:

3223 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
189902 INFO: Processing pre-safe import module hook gi.repository.GObject from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GObject.py'.
189906 INFO: Processing pre-safe import module hook gi.repository.GLib from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GLib.py'.
189910 INFO: Processing pre-safe import module hook gi.repository.Gio from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gio.py'.
189911 INFO: Processing pre-safe import module hook gi.repository.Gdk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gdk.py'.
189912 INFO: Processing pre-safe import module hook gi.repository.Atk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Atk.py'.
189915 INFO: Loading module hook 'hook-lib2to3.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
189971 INFO: Loading module hook 'hook-sysconfig.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
189972 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
189973 INFO: Loading module hook 'hook-xml.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
190007 INFO: Loading module hook 'hook-_tkinter.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
190098 INFO: checking Tree
190098 INFO: Building Tree because Tree-00.toc is non existent
190098 INFO: Building Tree Tree-00.toc
190130 INFO: checking Tree
190130 INFO: Building Tree because Tree-01.toc is non existent
190130 INFO: Building Tree Tree-01.toc
190246 INFO: Loading module hook 'hook-gi.repository.Atk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
190403 INFO: Loading module hook 'hook-gi.repository.Gdk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
190531 INFO: Processing pre-safe import module hook gi.repository.cairo from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.cairo.py'.
190532 INFO: Processing pre-safe import module hook gi.repository.Pango from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Pango.py'.
190532 INFO: Processing pre-safe import module hook gi.repository.GdkPixbuf from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GdkPixbuf.py'.
190533 INFO: Loading module hook 'hook-gi.repository.GdkPixbuf.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
190771 INFO: Processing pre-safe import module hook gi.repository.GModule from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GModule.py'.
190772 INFO: Loading module hook 'hook-gi.repository.Gio.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
190955 INFO: Loading module hook 'hook-gi.repository.GLib.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
191196 INFO: Loading module hook 'hook-gi.repository.GModule.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
191306 INFO: Loading module hook 'hook-gi.repository.GObject.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
191440 WARNING: Hidden import "gi._gobject" not found!
191440 INFO: Loading module hook 'hook-gi.repository.Pango.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
191558 INFO: Loading module hook 'hook-gi.repository.cairo.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
192161 INFO: Looking for ctypes DLLs
192178 INFO: Analyzing run-time hooks ...
192180 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth__tkinter.py'
192184 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
192190 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gtk.py'
192194 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_glib.py'
192198 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gdkpixbuf.py'
192202 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gio.py'
192206 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gi.py'
192212 INFO: Looking for dynamic libraries
192878 INFO: Looking for eggs
192878 INFO: Using Python library c:/msys64/mingw64/bin/libpython3.8.dll
192878 INFO: Found binding redirects:
[]
192881 INFO: Warnings written to C:/msys64/home/alexe/app/build/plotter/warn-plotter.txt
192911 INFO: Graph cross-reference written to C:/msys64/home/alexe/app/build/plotter/xref-plotter.html
199125 INFO: checking PYZ
199125 INFO: Building PYZ because PYZ-00.toc is non existent
199125 INFO: Building PYZ (ZlibArchive) C:/msys64/home/alexe/app/build/plotter/PYZ-00.pyz
199599 INFO: Building PYZ (ZlibArchive) C:/msys64/home/alexe/app/build/plotter/PYZ-00.pyz completed successfully.
199628 INFO: checking PKG
199628 INFO: Building PKG because PKG-00.toc is non existent
199628 INFO: Building PKG (CArchive) PKG-00.pkg
C:/msys64/home/alexe/app/src/python/plotter.py:181: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if event.button is 'up':
C:/msys64/home/alexe/app/src/python/plotter.py:184: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if event.button is 'down':
199649 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
199650 INFO: Bootloader C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/bootloader/Windows-64bit/runw.exe
199650 INFO: checking EXE
199650 INFO: Building EXE because EXE-00.toc is non existent
199650 INFO: Building EXE from EXE-00.toc
199650 INFO: Appending archive to EXE C:/msys64/home/alexe/app/build/plotter/plotter.exe
199693 INFO: Building EXE from EXE-00.toc completed successfully.
199885 INFO: checking COLLECT
199885 INFO: Building COLLECT because COLLECT-00.toc is non existent
199885 INFO: Removing dir C:/msys64/home/alexe/app/dist/plotter
202126 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:/msys64/mingw64/bin/pyinstaller.exe/__main__.py", line 7, in <module>
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 716, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 663, in build
    exec(code, spec_namespace)
  File "C:/msys64/home/alexe/app/plotter.spec", line 30, in <module>
    coll = COLLECT(exe,
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/api.py", line 698, in __init__
    self.__postinit__()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/api.py", line 742, in assemble
    shutil.copy(fnm, tofnm)
  File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 261, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Microsoft/Windows/SystemData/S-1-5-18/ReadOnly/LockScreen_Z/LockScreen___1920_1080_notdimmed.jpg'

And the second time it was:

3345 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
193798 INFO: Processing pre-safe import module hook gi.repository.GObject from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GObject.py'.
193801 INFO: Processing pre-safe import module hook gi.repository.GLib from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GLib.py'.
193805 INFO: Processing pre-safe import module hook gi.repository.Gio from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gio.py'.
193806 INFO: Processing pre-safe import module hook gi.repository.Gdk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gdk.py'.
193807 INFO: Processing pre-safe import module hook gi.repository.Atk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Atk.py'.
193810 INFO: Loading module hook 'hook-lib2to3.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
193870 INFO: Loading module hook 'hook-sysconfig.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
193871 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
193872 INFO: Loading module hook 'hook-xml.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
193903 INFO: Loading module hook 'hook-_tkinter.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
193988 INFO: checking Tree
193988 INFO: Building Tree because Tree-00.toc is non existent
193988 INFO: Building Tree Tree-00.toc
194019 INFO: checking Tree
194019 INFO: Building Tree because Tree-01.toc is non existent
194019 INFO: Building Tree Tree-01.toc
194134 INFO: Loading module hook 'hook-gi.repository.Atk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
194293 INFO: Loading module hook 'hook-gi.repository.Gdk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
194416 INFO: Processing pre-safe import module hook gi.repository.cairo from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.cairo.py'.
194417 INFO: Processing pre-safe import module hook gi.repository.Pango from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Pango.py'.
194417 INFO: Processing pre-safe import module hook gi.repository.GdkPixbuf from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GdkPixbuf.py'.
194418 INFO: Loading module hook 'hook-gi.repository.GdkPixbuf.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
194664 INFO: Processing pre-safe import module hook gi.repository.GModule from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GModule.py'.
194665 INFO: Loading module hook 'hook-gi.repository.Gio.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
194853 INFO: Loading module hook 'hook-gi.repository.GLib.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
195091 INFO: Loading module hook 'hook-gi.repository.GModule.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
195204 INFO: Loading module hook 'hook-gi.repository.GObject.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
195319 WARNING: Hidden import "gi._gobject" not found!
195319 INFO: Loading module hook 'hook-gi.repository.Pango.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
195434 INFO: Loading module hook 'hook-gi.repository.cairo.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
196042 INFO: Looking for ctypes DLLs
196059 INFO: Analyzing run-time hooks ...
196061 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth__tkinter.py'
196062 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
196065 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gtk.py'
196066 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_glib.py'
196066 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gdkpixbuf.py'
196068 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gio.py'
196068 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gi.py'
196072 INFO: Looking for dynamic libraries
196859 INFO: Looking for eggs
196859 INFO: Using Python library c:/msys64/mingw64/bin/libpython3.8.dll
196859 INFO: Found binding redirects:
[]
196862 INFO: Warnings written to C:/msys64/home/alexe/app/build/plotter/warn-plotter.txt
196892 INFO: Graph cross-reference written to C:/msys64/home/alexe/app/build/plotter/xref-plotter.html
203157 INFO: checking PYZ
203157 INFO: Building PYZ because PYZ-00.toc is non existent
203157 INFO: Building PYZ (ZlibArchive) C:/msys64/home/alexe/app/build/plotter/PYZ-00.pyz
203573 INFO: Building PYZ (ZlibArchive) C:/msys64/home/alexe/app/build/plotter/PYZ-00.pyz completed successfully.
203581 INFO: checking PKG
203581 INFO: Building PKG because PKG-00.toc is non existent
203581 INFO: Building PKG (CArchive) PKG-00.pkg
C:/msys64/home/alexe/app/src/python/plotter.py:181: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if event.button is 'up':
C:/msys64/home/alexe/app/src/python/plotter.py:184: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if event.button is 'down':
203601 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
203602 INFO: Bootloader C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/bootloader/Windows-64bit/runw.exe
203602 INFO: checking EXE
203602 INFO: Building EXE because EXE-00.toc is non existent
203602 INFO: Building EXE from EXE-00.toc
203602 INFO: Appending archive to EXE C:/msys64/home/alexe/app/build/plotter/plotter.exe
203647 INFO: Building EXE from EXE-00.toc completed successfully.
203837 INFO: checking COLLECT
203837 INFO: Building COLLECT because COLLECT-00.toc is non existent
203837 INFO: Removing dir C:/msys64/home/alexe/app/dist/plotter
206374 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:/msys64/mingw64/bin/pyinstaller.exe/__main__.py", line 7, in <module>
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 716, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 663, in build
    exec(code, spec_namespace)
  File "C:/msys64/home/alexe/app/plotter.spec", line 30, in <module>
    coll = COLLECT(exe,
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/api.py", line 698, in __init__
    self.__postinit__()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/api.py", line 742, in assemble
    shutil.copy(fnm, tofnm)
  File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 261, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Microsoft/Search/Data/Applications/Windows/edb.jtx'
make: *** [makefile:73: dist/plotter/plotter.exe] Error 1

Both of these attempts progressed much further than the other computers but failed at the end in Microsoft folders with Permission errors. On every computer, the MinGW shell was ran as an administrator.

LeighKorbel on 23 Jul 2020

Ohh-kay, I've come up with... _some_ ideas, more about how to _find out_ what's going on, than actual theories about what's going on. But, hopefully the former will produce the latter.

`msys-compat` branch updated

First off, I discovered in testing that, if you're using PyInstaller from my branch under MinGW, even if all of the hooks had run successfully PyInstaller wouldn't have worked properly. You'd have found that when it came time to assemble the .exe, the process fails when the Python libraries can't be found.

Remember a ways back when I said I'd "shortened the list" of library matches, in adding MSYS support? Well, it turns out the library names I _removed_ from the is_win list (which were labeled "for MSYS") were _actually_ "for MinGW", so without them PyInstaller in a MinGW environment can't link executables.

I've put those names back on the is_win list (and updated the associated comment, so that it's accurate as to their purpose); if you're working from the msys-compat branch you'll want to pull the new commit, then rebuild and reinstall PyInstaller.

However, that definitely has _nothing_ to do with your infinite-directory-recursion issues.

Directory searches

PyInstaller's code structure is... complex, to say the least. Necessarily so, as the complexity is largely due to abstractions that allow the hooks system to account for all manner of special cases in packaging different types of modules, libraries, applications, frameworks, etc. But it makes it somewhat difficult, even in the face of knowing _what's_ going wrong, to divine _where_ the wrong-going actually lives in the code. Which is to say, I still really have very little clue what could be causing that descent into C:\ProgramData\Application Data\ hell.

But I have some ideas on where we _must_ be able to find it, if we poke around enough.

There are only three possible ways that path could get into the packaging process:

It's something on the user $PATH (or %PATH% in Windows terminology), which PyInstaller is chasing in an attempt to locate dependencies
It's something in the Python sys.path, which yadda yadda same thing
It's something being erroneously injected by the Makefile that executes PyInstaller.

So, here's how we can track down which.

User program search path

Could you post the contents of your PATH variable from _both_ the MinGW shell _and_ a Windows CMD shell? (You'll be surprised how different they are; at least, I was.)

In MinGW's bash shell:
command $ echo $PATH /mingw64/bin:/mingw64/bin/site_perl/5.28.0:/mingw64/bin/vendor_perl :/mingw64/bin/core_perl:/usr/local/bin:/usr/bin:/bin:/c/Windows/System32:/c/Windows :/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/
In a Windows Command Prompt:
command C:\> echo %PATH% C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\ ;C:\Program Files\Python 3.7.7;C:\Program Files\Python 3.7.7\Scripts ;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem ;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\ProgramData\chocolatey\bin ;C:\Program Files\Git\cmd;C:\Program Files\Windows NT\Accessories

Any paths that point inside C:\ProgramData\ would obviously be the prime suspects, and barring that anything that seems like it might have indirect ties to something in C:\ProgramData\.

Python's `sys.path`

Back in a MinGW bash shell, can you run this command and provide the results?
command $ python3 -c 'import sys; import pprint; pprint.pprint(sys.path)' ['', 'C:/msys64/mingw64/lib/python38.zip', 'C:/msys64/mingw64/lib/python3.8', 'C:/msys64/mingw64/lib/python3.8/lib-dynload', 'C:/Users/ferd/.local/lib/python3.8/site-packages', 'C:/Users/ferd/.local/lib/python3.8/site-packages/pyinstaller-4.0.dev0-py3.8.egg', 'C:/msys64/mingw64/lib/python3.8/site-packages']

As you can see, PyInstaller is present on my sys.path, because I installed it using python3 setup.py install --user which installs the .egg distribution at the path you see there. Installing it from a wheel you won't see that, but my worry is that there may possibly be _other_ installed packages polluting your sys.path and possibly sending you down into C:\ProgramData. (Especially since it _was_ a file path inside of the Anaconda installation that was residing in there, when we first encountered this issue.)

Inspecting the build process

Needless to say, if any suspect path elements come up in any of the previous checks, the thing to do would be to get them _out_ of the relevant path list, and see if perhaps that solves the problem.

But at heart I'm more pragmatist than optimist, so I'm just going to charge ahead assuming we haven't gotten that lucky.

So, assuming your paths are all clean and unlikely to be causing this, we'll need to look at the process by which PyInstaller is both (a) set in motion, and then (b) goes about its tasks.

Getting an idea of what's going on during the parent Makefile build may help, if nothing else knowing exactly how it's running PyInstaller means that we can experiment with running it _ourselves_ and making adjustments to the command lines in use.

Most Makefiles generated from any sort of build system, whether it's autotools, CMake, the Linux kernel tree, etc., if their build process is by default at all "quiet" (meaning it suppresses some or all of Make's normal verbosity in executing commands), will typically support a "verbose build" flag of some sort. It nearly always takes the form of an argument to the make command, like make V=1 (autoconf, kernel tools) or make VERBOSE=1 (CMake).

If you could run a make V=1 or make VERBOSE=1 build of the software you're trying to package, from start to finish — well, failure — in a _clean_ build tree, that will hopefully provide some insights. I find the tee command especially handy for this sort of thing. If you pipe the output of the make command into it, tee will display all of the lines just like normal, but _also_ copy them all to whatever filename you give it.

So, if you run:

$ make VERBOSE=1 | tee make.txt

then at the end of the build run you'll have a file make.txt containing all of the build output.

It'd be great if you could do that and post as much of the _entire_ output as possible. I understand there may be sensitive data that you'd rather not share publicly, and if that's the case then of course please don't include those portions. But err on the side of including as much as you're comfortable sharing, as it's much easier to comprehend a system from afar when you can see its _entire_ lifecycle. Small snippets of an error message or whatever often lack the necessary context. If you don't want to dump the whole output into a comment that could end up being nearly as long as this one, you can upload the file as a GitHub comment attachment and the only thing inserted into your comment will be a download link.

Inspecting PyInstaller

Assuming all of _that_ still gets us no closer, then the next step would be to inspect the PyInstaller run itself. That process starts with the Python -v flag, which will trace all of the module imports performed as each script is executed.

However your Makefile is invoking PyInstaller now (presumably it's running the pyinstaller shell command, which is just a wrapper to the Python module), that wrapper can be replaced with python3 -m PyInstaller (the module name is case-sensitive) to explicitly run the Python interpreter, at which point we can add the -v flag as well.

So, if you could edit/configure your Makefile or its generator so that the command that probably looks like:

pyinstaller <rest of the command line>

and replace that with:

python3 -v -m PyInstaller <same rest of command line>

Then you'll get a _very_ detailed look at all of the module interdependencies in play as PyInstaller runs. If you do that from the Makefile, you can use tee to capture it as well, with the one wrinkle that Python's debug output is normally directed to the STDERR output, not STDOUT, so tee won't capture any of it. To correct that, you can redirect STDERR to STDOUT so it all goes to the same file:

# With the Makefile
$ make 2>&1 | tee make.txt
# Or, running PyInstaller standalone:
$ python3 -v -m PyInstaller <command line> 2>&1 | tee pyinstaller.txt

And, same deal, please post as much of the complete output as you're able.

ferdnyc on 23 Jul 2020

In MinGW shell:

# echo $PATH
/mingw64/bin:/usr/local/bin:/usr/bin:/bin:/c/Windows/System32:/c/Windows:/c/Windows/System32/Wbem:/c/Windows/System32/WindowsPowerShell/v1.0/:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl

In CMD:

C:\Users\Leigh>echo %PATH%
C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\ProgramData\Oracle\Java\javapath;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\WINDOWS\System32\WindowsPowerShell\v1.0\;C:\Program Files\MiKTeX 2.9\miktex\bin\x64\;C:\Program Files (x86)\Wolfram Research\WolframScript\;C:\WINDOWS\System32\OpenSSH\;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\GTK3-Runtime Win64\bin;C:\Users\Leigh\AppData\Local\Microsoft\WindowsApps;

Python sys.path:

# python3 -c 'import sys; import pprint; pprint.pprint(sys.path)'
['',
 'C:/msys64/mingw64/lib/python38.zip',
 'C:/msys64/mingw64/lib/python3.8',
 'C:/msys64/mingw64/lib/python3.8/lib-dynload',
 'C:/msys64/mingw64/lib/python3.8/site-packages']

So at first glance it seems as though your pessimism was well founded. I am using GNU make on a self constructed makefile. You are looking for the output of make -n then?

LeighKorbel on 23 Jul 2020

Otherwise I think GNU make is verbose by default.

LeighKorbel on 23 Jul 2020

Inspecting PyInstaller:

# make
The version of MSYS you are running is 3 (0 meaning not MSYS at all)
python3 -v m PyInstaller src/python/plotter.py -y -w
import _frozen_importlib # frozen
import _imp # builtin
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
import '_io' # <class '_frozen_importlib.BuiltinImporter'>
import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
import 'nt' # <class '_frozen_importlib.BuiltinImporter'>
import _thread # previously loaded ('_thread')
import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
import _weakref # previously loaded ('_weakref')
import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
import 'winreg' # <class '_frozen_importlib.BuiltinImporter'>
# installing zipimport hook
import 'time' # <class '_frozen_importlib.BuiltinImporter'>
import 'zipimport' # <class '_frozen_importlib.FrozenImporter'>
# installed zipimport hook
# C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/__init__.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/encodings/__init__.py
# code object from 'C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/__init__.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/codecs.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/codecs.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/codecs.cpython-38.pyc'
import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007a0970>
# C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/aliases.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/encodings/aliases.py
# code object from 'C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/aliases.cpython-38.pyc'
import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007bef40>
import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007a06d0>
# C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/utf_8.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/encodings/utf_8.py
# code object from 'C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/utf_8.cpython-38.pyc'
import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007a07f0>
# C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/cp1252.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/encodings/cp1252.py
# code object from 'C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/cp1252.cpython-38.pyc'
import 'encodings.cp1252' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007bef70>
import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
# C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/latin_1.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/encodings/latin_1.py
# code object from 'C:/msys64/mingw64/lib/python3.8/encodings/__pycache__/latin_1.cpython-38.pyc'
import 'encodings.latin_1' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007d7400>
# C:/msys64/mingw64/lib/python3.8/__pycache__/io.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/io.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/io.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/abc.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/abc.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/abc.cpython-38.pyc'
import '_abc' # <class '_frozen_importlib.BuiltinImporter'>
import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007d7850>
import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007d7580>
# C:/msys64/mingw64/lib/python3.8/__pycache__/site.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/site.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/site.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/os.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/os.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/os.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/stat.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/stat.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/stat.cpython-38.pyc'
import '_stat' # <class '_frozen_importlib.BuiltinImporter'>
import 'stat' # <_frozen_importlib_external.SourceFileLoader object at 0x000000000254d220>
# C:/msys64/mingw64/lib/python3.8/__pycache__/_collections_abc.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/_collections_abc.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/_collections_abc.cpython-38.pyc'
import '_collections_abc' # <_frozen_importlib_external.SourceFileLoader object at 0x000000000254d2e0>
# C:/msys64/mingw64/lib/python3.8/__pycache__/ntpath.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/ntpath.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/ntpath.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/genericpath.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/genericpath.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/genericpath.cpython-38.pyc'
import 'genericpath' # <_frozen_importlib_external.SourceFileLoader object at 0x0000000002572e80>
import 'ntpath' # <_frozen_importlib_external.SourceFileLoader object at 0x000000000254dbe0>
import 'os' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007e1940>
# C:/msys64/mingw64/lib/python3.8/__pycache__/_sitebuiltins.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/_sitebuiltins.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/_sitebuiltins.cpython-38.pyc'
import '_sitebuiltins' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000025454f0>
# C:/msys64/mingw64/lib/python3.8/__pycache__/sysconfig.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/sysconfig.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/sysconfig.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/textwrap.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/textwrap.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/textwrap.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/re.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/re.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/re.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/enum.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/enum.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/enum.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/types.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/types.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/types.cpython-38.pyc'
import 'types' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000025b2160>
import 'enum' # <_frozen_importlib_external.SourceFileLoader object at 0x0000000002598b20>
# C:/msys64/mingw64/lib/python3.8/__pycache__/sre_compile.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/sre_compile.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/sre_compile.cpython-38.pyc'
import '_sre' # <class '_frozen_importlib.BuiltinImporter'>
# C:/msys64/mingw64/lib/python3.8/__pycache__/sre_parse.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/sre_parse.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/sre_parse.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/sre_constants.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/sre_constants.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/sre_constants.cpython-38.pyc'
import 'sre_constants' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000026c3fa0>
import 'sre_parse' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000026c3580>
import 'sre_compile' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000025b2c40>
# C:/msys64/mingw64/lib/python3.8/__pycache__/functools.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/functools.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/functools.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/collections/__pycache__/__init__.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/collections/__init__.py
# code object from 'C:/msys64/mingw64/lib/python3.8/collections/__pycache__/__init__.cpython-38.pyc'
# C:/msys64/mingw64/lib/python3.8/__pycache__/operator.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/operator.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/operator.cpython-38.pyc'
import '_operator' # <class '_frozen_importlib.BuiltinImporter'>
import 'operator' # <_frozen_importlib_external.SourceFileLoader object at 0x0000000002703be0>
# C:/msys64/mingw64/lib/python3.8/__pycache__/keyword.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/keyword.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/keyword.cpython-38.pyc'
import 'keyword' # <_frozen_importlib_external.SourceFileLoader object at 0x0000000002703d00>
# C:/msys64/mingw64/lib/python3.8/__pycache__/heapq.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/heapq.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/heapq.cpython-38.pyc'
# extension module '_heapq' loaded from 'C:/msys64/mingw64/lib/python3.8/lib-dynload/_heapq-cpython-38.dll'
# extension module '_heapq' executed from 'C:/msys64/mingw64/lib/python3.8/lib-dynload/_heapq-cpython-38.dll'
import '_heapq' # <_frozen_importlib_external.ExtensionFileLoader object at 0x000000000270d670>
import 'heapq' # <_frozen_importlib_external.SourceFileLoader object at 0x0000000002703e80>
import 'itertools' # <class '_frozen_importlib.BuiltinImporter'>
# C:/msys64/mingw64/lib/python3.8/__pycache__/reprlib.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/reprlib.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/reprlib.cpython-38.pyc'
import 'reprlib' # <_frozen_importlib_external.SourceFileLoader object at 0x0000000002703d90>
import '_collections' # <class '_frozen_importlib.BuiltinImporter'>
import 'collections' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000026e5dc0>
import '_functools' # <class '_frozen_importlib.BuiltinImporter'>
import 'functools' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000026c32b0>
import '_locale' # <class '_frozen_importlib.BuiltinImporter'>
# C:/msys64/mingw64/lib/python3.8/__pycache__/copyreg.cpython-38.pyc matches C:/msys64/mingw64/lib/python3.8/copyreg.py
# code object from 'C:/msys64/mingw64/lib/python3.8/__pycache__/copyreg.cpython-38.pyc'
import 'copyreg' # <_frozen_importlib_external.SourceFileLoader object at 0x0000000002710190>
import 're' # <_frozen_importlib_external.SourceFileLoader object at 0x000000000257ffa0>
import 'textwrap' # <_frozen_importlib_external.SourceFileLoader object at 0x000000000257fa30>
import 'sysconfig' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000025456d0>
import 'site' # <_frozen_importlib_external.SourceFileLoader object at 0x00000000007e1220>
# clear sys.audit hooks
# clear builtins._
# clear sys.path
# clear sys.argv
# clear sys.ps1
# clear sys.ps2
# clear sys.last_type
# clear sys.last_value
# clear sys.last_traceback
# clear sys.path_hooks
# clear sys.path_importer_cache
# clear sys.meta_path
# clear sys.__interactivehook__
# restore sys.stdin
# restore sys.stdout
# restore sys.stderr
# cleanup[2] removing sys
# cleanup[2] removing builtins
# cleanup[2] removing _frozen_importlib
# cleanup[2] removing _imp
# cleanup[2] removing _warnings
# cleanup[2] removing _frozen_importlib_external
# cleanup[2] removing _io
# cleanup[2] removing marshal
# cleanup[2] removing nt
# cleanup[2] removing _thread
# cleanup[2] removing _weakref
# cleanup[2] removing winreg
# cleanup[2] removing time
# cleanup[2] removing zipimport
# destroy zipimport
# cleanup[2] removing _codecs
# cleanup[2] removing codecs
# cleanup[2] removing encodings.aliases
# cleanup[2] removing encodings
# destroy encodings
# cleanup[2] removing encodings.utf_8
# cleanup[2] removing encodings.cp1252
# cleanup[2] removing _signal
# cleanup[2] removing __main__
# destroy __main__
# cleanup[2] removing encodings.latin_1
# cleanup[2] removing _abc
# cleanup[2] removing abc
# cleanup[2] removing io
# cleanup[2] removing _stat
# cleanup[2] removing stat
# cleanup[2] removing _collections_abc
# cleanup[2] removing genericpath
# cleanup[2] removing ntpath
# cleanup[2] removing os.path
# cleanup[2] removing os
# cleanup[2] removing _sitebuiltins
# cleanup[2] removing types
# destroy types
# cleanup[2] removing enum
# cleanup[2] removing _sre
# cleanup[2] removing sre_constants
# destroy sre_constants
# cleanup[2] removing sre_parse
# cleanup[2] removing sre_compile
# cleanup[2] removing _operator
# cleanup[2] removing operator
# destroy operator
# cleanup[2] removing keyword
# destroy keyword
# cleanup[2] removing _heapq
# cleanup[2] removing heapq
# cleanup[2] removing itertools
# cleanup[2] removing reprlib
# destroy reprlib
# cleanup[2] removing _collections
# cleanup[2] removing collections
# destroy collections
# cleanup[2] removing _functools
# cleanup[2] removing functools
# cleanup[2] removing _locale
# cleanup[2] removing copyreg
# cleanup[2] removing re
# cleanup[2] removing textwrap
# cleanup[2] removing sysconfig
# destroy sysconfig
# cleanup[2] removing site
# destroy site
# destroy time
# destroy _signal
# destroy _sitebuiltins
# destroy io
# destroy abc
# destroy ntpath
# destroy _stat
# destroy genericpath
# destroy stat
# destroy os
# destroy textwrap
# destroy re
# destroy enum
# destroy sre_compile
# destroy functools
# destroy copyreg
# destroy sre_parse
# destroy _sre
# destroy _abc
# destroy _collections_abc
# destroy heapq
# destroy _operator
# destroy _heapq
# destroy itertools
# destroy _collections
# destroy _functools
# destroy _locale
# cleanup[3] wiping encodings.latin_1
# cleanup[3] wiping encodings.cp1252
Python 3.8.3 (default, Jun 17 2020, 06:11:06)  [GCC 10.1.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
C:\msys64\mingw64\bin\python3.exe: can't open file 'm': [Errno 2] No such file or directory
# cleanup[3] wiping encodings.utf_8
# cleanup[3] wiping encodings.aliases
# cleanup[3] wiping codecs
# cleanup[3] wiping _codecs
# cleanup[3] wiping winreg
# cleanup[3] wiping _weakref
# cleanup[3] wiping _thread
# cleanup[3] wiping nt
# cleanup[3] wiping marshal
# cleanup[3] wiping _io
# cleanup[3] wiping _frozen_importlib_external
# destroy io
# destroy nt
# destroy winreg
# destroy marshal
# cleanup[3] wiping _warnings
# cleanup[3] wiping _imp
# cleanup[3] wiping _frozen_importlib
# destroy _frozen_importlib_external
# destroy _imp
# destroy _thread
# destroy _warnings
# destroy _weakref
# cleanup[3] wiping sys
# cleanup[3] wiping builtins
# destroy _frozen_importlib
make: *** [makefile:73: dist/plotter/plotter.exe] Error 2

LeighKorbel on 23 Jul 2020

Also, I could just post the contents of the makefile if that is more useful. It's pretty straightforward and I don't think there is anything there that I would consider proprietary.

LeighKorbel on 23 Jul 2020

Sorry, the printout from inspecting PyInstaller before was not everything, as I just assumed both STDOUT and STDERR would be printed to the shell. There is a little bit more from the redirect of both to the file as you suggested, but I am not sure that it is any more informative. Also, let me know about the makefile and we can go from there. Thank you!

make.txt

LeighKorbel on 23 Jul 2020

So at first glance it seems as though your pessimism was well founded. I am using GNU make on a self constructed makefile. You are looking for the output of make -n then?

Yeah, seems that way, darn. If make runs verbosely, so that it echoes commands before running them, then the output of the actual make run would be best. make -n will show what it's trying to do, but not what happens when it does it, so there'd still be pieces missing from the picture.

The first pyinstaller run didn't actually do anything, buried in the middle is this error message:

C:\msys64\mingw64\bin\python3.exe: can't open file 'm': [Errno 2] No such file or directory

So, something in the command line there was no bueno. (Looks like maybe the dash was missing, before -m PyInstaller?)

I'll take a look at the make.txt now.

ferdnyc on 24 Jul 2020

Yeah, that has the full details... though they're not super enlightening. Still, I'll sum up what's there...

ferdnyc on 24 Jul 2020

So, hook-gi.repository.Gtk.py is indeed the very last import before it dies. Before that, only a few others had been run (in reverse order):

hook-gi.py
hook-encodings.py
hook-distutils.py
Before that, we had some pre-hooks:
pre_safe_import_module/hook-gi.repository.Gtk.py
pre_find_module_path/hook-distutils.py

And there are some utility imports and etc, but nothing that seems likely to be causing this. The ProgramData path never appears anywhere in the output until that error message indicating that it's trying to add the crazy-long path to the package's binary data.

Hmm... here's an idea. PyInstaller successfully runs its makespec module, because along the way it reports:

70 INFO: wrote C:/msys64/home/Leigh/Cybercyte/CyberSolver/plotter.spec

Is there anything useful in that file? It would be one of the sources for that "binary and data files" list that it's getting the crazy off of.

ferdnyc on 24 Jul 2020

Contents of plotter.spec:

# -*- mode: python ; coding: utf-8 -*-

block_cipher = None


a = Analysis(['src/python/plotter.py'],
             pathex=['C:/msys64/home/Leigh/Cybercyte/CyberSolver'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='plotter',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=False )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               upx_exclude=[],
               name='plotter')

So regarding the make then, I can either post the contents of the makefile itself, or the verbose output. I am still not sure what flag I would need to pass with GNU make to get verbose output though.

LeighKorbel on 24 Jul 2020

The output using make --debug=j is here:
make_output.txt

LeighKorbel on 24 Jul 2020

I could swear I did this earlier and the output was the same as make, but here is the output of make V=1:
make_verbose.txt

LeighKorbel on 24 Jul 2020

So I am guessing that I am dead in the water here, eh?

LeighKorbel on 28 Jul 2020

@LeighKorbel I can reproduce some of the issues you are having when using PyInstaller 4.0.

I'm not sure about all the errors you are getting when you have a computer with Anaconda installed, but this is the error that I am getting consistently with PyInstaller 4.0, that I wasn't getting in 3.6:

(3) A fourth computer on Windows 10 that did not have Anaconda installed. This one acted differently. I repeated it twice to make sure it was the same. In both instances it got past the module hook for GTK and got all the way to the final step with PyInstaller before failing. The first run gave:
File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 415, in copy copyfile(src, dst, follow_symlinks=follow_symlinks) File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 261, in copyfile with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst: PermissionError: [Errno 13] Permission denied: 'C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Microsoft/Search/Data/Applications/Windows/edb.jtx'
I'm trying to figure out a minimally reproducible program to use, a simple hello world program doesn't give the error, so maybe it is related to GTK.

danyeaw on 11 Aug 2020

👍1

@danyeaw Great! In the mean time I backtracked the commit on my project that was using PyGObject because of this issue. Let me know if you need anything on my end and I will get back promptly.

LeighKorbel on 11 Aug 2020

Latest development version. Ran pyinstaller --exclude-module tkinter --onedir --debug all --noupx gtktest.py
Minimum reproducible example:

import gi
gi.require_version("Gtk", "3.0")
from gi.repository import Gtk

window = Gtk.Window(title="Hello World")
window.show()
window.connect("destroy", Gtk.main_quit)
Gtk.main()

Output:

$ pyinstaller --exclude-module tkinter --onedir --debug all --noupx gtktest.py
78 INFO: PyInstaller: 4.1.dev0
78 INFO: Python: 3.8.5
78 INFO: Platform: Windows-10-10.0.18362-SP0
78 INFO: wrote C:/tools/msys64/home/dyeaw/pyinstaller-test/gtktest.spec
359 INFO: UPX is available.
359 INFO: Extending PYTHONPATH with paths
['C:/tools/msys64/home/dyeaw/pyinstaller-test',
 'C:/tools/msys64/home/dyeaw/pyinstaller-test']
359 INFO: checking Analysis
359 INFO: Building Analysis because Analysis-00.toc is non existent
359 INFO: Initializing module dependency graph...
359 INFO: Caching module graph hooks...
374 INFO: Analyzing base_library.zip ...
2312 INFO: Processing pre-find module path hook distutils from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
2312 INFO: distutils: retargeting to non-venv dir 'C:/tools/msys64/mingw64/lib/python3.8'
3799 INFO: Caching module dependency graph...
3887 INFO: running Analysis Analysis-00.toc
3887 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
  required by c:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/bin/python.exe
3927 INFO: Analyzing C:/tools/msys64/home/dyeaw/pyinstaller-test/gtktest.py
3983 INFO: Processing pre-safe import module hook gi.repository.Gtk from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gtk.py'.
3983 INFO: Processing module hooks...
3983 INFO: Loading module hook 'hook-distutils.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
3983 INFO: Loading module hook 'hook-encodings.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
4175 INFO: Loading module hook 'hook-gi.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
4183 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
37034 INFO: Processing pre-safe import module hook gi.repository.GObject from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GObject.py'.
37058 INFO: Processing pre-safe import module hook gi.repository.GLib from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GLib.py'.
37058 INFO: Processing pre-safe import module hook gi.repository.Gio from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gio.py'.
37066 INFO: Processing pre-safe import module hook gi.repository.Gdk from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gdk.py'.
37066 INFO: Processing pre-safe import module hook gi.repository.Atk from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Atk.py'.
37066 INFO: Loading module hook 'hook-lib2to3.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
37202 INFO: Loading module hook 'hook-sysconfig.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
37202 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
37202 INFO: Loading module hook 'hook-xml.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
37250 INFO: Loading module hook 'hook-gi.repository.Atk.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
37802 INFO: Loading module hook 'hook-gi.repository.Gdk.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
38337 INFO: Processing pre-safe import module hook gi.repository.cairo from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.cairo.py'.
38337 INFO: Processing pre-safe import module hook gi.repository.Pango from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Pango.py'.
38337 INFO: Processing pre-safe import module hook gi.repository.GdkPixbuf from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GdkPixbuf.py'.
38345 INFO: Loading module hook 'hook-gi.repository.GdkPixbuf.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
39600 INFO: Processing pre-safe import module hook gi.repository.GModule from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GModule.py'.
39600 INFO: Loading module hook 'hook-gi.repository.Gio.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
40488 INFO: Loading module hook 'hook-gi.repository.GLib.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
41751 INFO: Loading module hook 'hook-gi.repository.GModule.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
42297 INFO: Loading module hook 'hook-gi.repository.GObject.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
42889 WARNING: Hidden import "gi._gobject" not found!
42889 INFO: Loading module hook 'hook-gi.repository.Pango.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
43441 INFO: Loading module hook 'hook-gi.repository.cairo.py' from 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks'...
44136 INFO: Looking for ctypes DLLs
44160 INFO: Analyzing run-time hooks ...
44160 INFO: Including run-time hook 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
44168 INFO: Including run-time hook 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gtk.py'
44168 INFO: Including run-time hook 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_glib.py'
44168 INFO: Including run-time hook 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gdkpixbuf.py'
44168 INFO: Including run-time hook 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gio.py'
44168 INFO: Including run-time hook 'C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gi.py'
44176 INFO: Looking for dynamic libraries
45184 INFO: Looking for eggs
45184 INFO: Using Python library C:\tools\msys64\mingw64\bin/libpython3.8.dll
45184 INFO: Found binding redirects:
[]
45272 INFO: Warnings written to C:/tools/msys64/home/dyeaw/pyinstaller-test/build/gtktest/warn-gtktest.txt
45304 INFO: Graph cross-reference written to C:/tools/msys64/home/dyeaw/pyinstaller-test/build/gtktest/xref-gtktest.html
46047 INFO: checking PYZ
46047 INFO: Building PYZ because PYZ-00.toc is non existent
46047 INFO: Building PYZ (ZlibArchive) C:/tools/msys64/home/dyeaw/pyinstaller-test/build/gtktest/PYZ-00.pyz
46047 INFO: Building PYZ (ZlibArchive) C:/tools/msys64/home/dyeaw/pyinstaller-test/build/gtktest/PYZ-00.pyz completed successfully.
46047 INFO: checking PKG
46047 INFO: Building PKG because PKG-00.toc is non existent
46047 INFO: Building PKG (CArchive) PKG-00.pkg
46063 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
46063 INFO: Bootloader C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/bootloader/Windows-64bit/run_d.exe
46063 INFO: checking EXE
46063 INFO: Building EXE because EXE-00.toc is non existent
46063 INFO: Building EXE from EXE-00.toc
46063 INFO: Appending archive to EXE C:/tools/msys64/home/dyeaw/pyinstaller-test/build/gtktest/gtktest.exe
46063 INFO: Building EXE from EXE-00.toc completed successfully.
46119 INFO: checking COLLECT
46119 INFO: Building COLLECT because COLLECT-00.toc is non existent
46119 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "C:/tools/msys64/mingw64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:/tools/msys64/mingw64/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/bin/pyinstaller.exe/__main__.py", line 7, in <module>
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 720, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 667, in build
    exec(code, spec_namespace)
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/gtktest.spec", line 30, in <module>
    coll = COLLECT(exe,
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/building/api.py", line 699, in __init__
    self.__postinit__()
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "C:/tools/msys64/home/dyeaw/pyinstaller-test/.venv/lib/python3.8/site-packages/PyInstaller/building/api.py", line 743, in assemble
    shutil.copy(fnm, tofnm)
  File "C:/tools/msys64/mingw64/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:/tools/msys64/mingw64/lib/python3.8/shutil.py", line 261, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:/ProgramData/Symantec/Symantec Endpoint Protection/CurrentVersion/Data/Config/TrayPluginRegistration.dat'
(.venv)

danyeaw on 13 Aug 2020

Seems like you aren't falling down the circular reference hole created by the Application Data junction though. I'll try to reproduce this tomorrow on the computer that didn't have Anaconda installed on it.

LeighKorbel on 13 Aug 2020

Result running make on my project on the machine that produced the results that were similar to yours previously @danyeaw:

# make
The version of MSYS you are running is 3 (0 meaning not MSYS at all)
pyinstaller src/python/plotter.py -y -w
57 INFO: PyInstaller: 4.0.dev0
57 INFO: Python: 3.8.3
57 INFO: Platform: Windows-10-10.0.18362-SP0
58 INFO: wrote C:/msys64/home/Alex/Cybercyte/CyberSolver/plotter.spec
59 INFO: UPX is not available.
64 INFO: Extending PYTHONPATH with paths
['C:/msys64/home/Alex/Cybercyte/CyberSolver/src/python',
 'C:/msys64/home/Alex/Cybercyte/CyberSolver']
66 INFO: checking Analysis
66 INFO: Building Analysis because Analysis-00.toc is non existent
66 INFO: Initializing module dependency graph...
68 INFO: Caching module graph hooks...
78 INFO: Analyzing base_library.zip ...
1636 INFO: Processing pre-find module path hook distutils from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
1637 INFO: distutils: retargeting to non-venv dir 'C:/msys64/mingw64/lib/python3.8'
3218 INFO: Caching module dependency graph...
3292 INFO: running Analysis Analysis-00.toc
3297 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
  required by c:/msys64/mingw64/bin/python.exe
3330 INFO: Analyzing C:/msys64/home/Alex/Cybercyte/CyberSolver/src/python/plotter.py
C:/msys64/home/Alex/Cybercyte/CyberSolver/src/python/plotter.py:187: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if event.button is 'up':
C:/msys64/home/Alex/Cybercyte/CyberSolver/src/python/plotter.py:190: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if event.button is 'down':
3428 INFO: Processing pre-safe import module hook gi.repository.Gtk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gtk.py'.
3429 INFO: Processing module hooks...
3429 INFO: Loading module hook 'hook-distutils.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
3431 INFO: Loading module hook 'hook-encodings.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
3485 INFO: Loading module hook 'hook-gi.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
3492 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
393583 INFO: Processing pre-safe import module hook gi.repository.GObject from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GObject.py'.
393587 INFO: Processing pre-safe import module hook gi.repository.GLib from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GLib.py'.
393591 INFO: Processing pre-safe import module hook gi.repository.Gio from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gio.py'.
393593 INFO: Processing pre-safe import module hook gi.repository.Gdk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gdk.py'.
393593 INFO: Processing pre-safe import module hook gi.repository.Atk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Atk.py'.
393601 INFO: Loading module hook 'hook-lib2to3.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
393675 INFO: Loading module hook 'hook-sysconfig.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
393676 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
393677 INFO: Loading module hook 'hook-xml.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
393731 INFO: Loading module hook 'hook-_tkinter.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
393919 INFO: checking Tree
393919 INFO: Building Tree because Tree-00.toc is non existent
393919 INFO: Building Tree Tree-00.toc
394024 INFO: checking Tree
394024 INFO: Building Tree because Tree-01.toc is non existent
394024 INFO: Building Tree Tree-01.toc
394250 INFO: Loading module hook 'hook-gi.repository.Atk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
394449 INFO: Loading module hook 'hook-gi.repository.Gdk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
394585 INFO: Processing pre-safe import module hook gi.repository.cairo from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.cairo.py'.
394586 INFO: Processing pre-safe import module hook gi.repository.Pango from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Pango.py'.
394587 INFO: Processing pre-safe import module hook gi.repository.GdkPixbuf from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GdkPixbuf.py'.
394587 INFO: Loading module hook 'hook-gi.repository.GdkPixbuf.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
395239 INFO: Processing pre-safe import module hook gi.repository.GModule from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GModule.py'.
395239 INFO: Loading module hook 'hook-gi.repository.Gio.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
395445 INFO: Loading module hook 'hook-gi.repository.GLib.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
395718 INFO: Loading module hook 'hook-gi.repository.GModule.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
395841 INFO: Loading module hook 'hook-gi.repository.GObject.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
395974 WARNING: Hidden import "gi._gobject" not found!
395974 INFO: Loading module hook 'hook-gi.repository.Pango.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
396171 INFO: Loading module hook 'hook-gi.repository.cairo.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
397262 INFO: Looking for ctypes DLLs
397280 INFO: Analyzing run-time hooks ...
397282 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth__tkinter.py'
397287 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
397289 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gtk.py'
397290 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_glib.py'
397292 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gdkpixbuf.py'
397297 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gio.py'
397301 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gi.py'
397309 INFO: Looking for dynamic libraries
398110 INFO: Looking for eggs
398110 INFO: Using Python library c:/msys64/mingw64/bin/libpython3.8.dll
398110 INFO: Found binding redirects:
[]
398113 INFO: Warnings written to C:/msys64/home/Alex/Cybercyte/CyberSolver/build/plotter/warn-plotter.txt
398146 INFO: Graph cross-reference written to C:/msys64/home/Alex/Cybercyte/CyberSolver/build/plotter/xref-plotter.html
410769 INFO: checking PYZ
410769 INFO: Building PYZ because PYZ-00.toc is non existent
410769 INFO: Building PYZ (ZlibArchive) C:/msys64/home/Alex/Cybercyte/CyberSolver/build/plotter/PYZ-00.pyz
411238 INFO: Building PYZ (ZlibArchive) C:/msys64/home/Alex/Cybercyte/CyberSolver/build/plotter/PYZ-00.pyz completed successfully.
411248 INFO: checking PKG
411249 INFO: Building PKG because PKG-00.toc is non existent
411249 INFO: Building PKG (CArchive) PKG-00.pkg
C:/msys64/home/Alex/Cybercyte/CyberSolver/src/python/plotter.py:187: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if event.button is 'up':
C:/msys64/home/Alex/Cybercyte/CyberSolver/src/python/plotter.py:190: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if event.button is 'down':
411274 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
411275 INFO: Bootloader C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/bootloader/Windows-64bit/runw.exe
411275 INFO: checking EXE
411275 INFO: Building EXE because EXE-00.toc is non existent
411275 INFO: Building EXE from EXE-00.toc
411276 INFO: Appending archive to EXE C:/msys64/home/Alex/Cybercyte/CyberSolver/build/plotter/plotter.exe
411336 INFO: Building EXE from EXE-00.toc completed successfully.
411722 INFO: checking COLLECT
411722 INFO: Building COLLECT because COLLECT-00.toc is non existent
411723 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:/msys64/mingw64/bin/pyinstaller.exe/__main__.py", line 7, in <module>
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 716, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 663, in build
    exec(code, spec_namespace)
  File "C:/msys64/home/Alex/Cybercyte/CyberSolver/plotter.spec", line 30, in <module>
    coll = COLLECT(exe,
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/api.py", line 698, in __init__
    self.__postinit__()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/api.py", line 742, in assemble
    shutil.copy(fnm, tofnm)
  File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 261, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Microsoft/Windows Defender/Scans/mpcache-D197A58A47B6D6C0E51AA4D49CFAE9B250397A36.bin.7E'
make: *** [makefile:72: dist/plotter/plotter.exe] Error 1

Results on the same machine running your minimal example:

# pyinstaller --exclude-module tkinter --onedir --debug all --noupx gtktest.py
55 INFO: PyInstaller: 4.0.dev0
55 INFO: Python: 3.8.3
55 INFO: Platform: Windows-10-10.0.18362-SP0
56 INFO: wrote C:/msys64/home/Alex/Cybercyte/gtktest.spec
57 INFO: UPX is not available.
62 INFO: Extending PYTHONPATH with paths
['C:/msys64/home/Alex/Cybercyte', 'C:/msys64/home/Alex/Cybercyte']
64 INFO: checking Analysis
64 INFO: Building Analysis because Analysis-00.toc is non existent
65 INFO: Initializing module dependency graph...
66 INFO: Caching module graph hooks...
72 INFO: Analyzing base_library.zip ...
1450 INFO: Processing pre-find module path hook distutils from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
1451 INFO: distutils: retargeting to non-venv dir 'C:/msys64/mingw64/lib/python3.8'
3265 INFO: Caching module dependency graph...
3339 INFO: running Analysis Analysis-00.toc
3340 INFO: Adding Microsoft.Windows.Common-Controls to dependent assemblies of final executable
  required by c:/msys64/mingw64/bin/python.exe
3372 INFO: Analyzing C:/msys64/home/Alex/Cybercyte/gtktest.py
3440 INFO: Processing pre-safe import module hook gi.repository.Gtk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gtk.py'.
3441 INFO: Processing module hooks...
3441 INFO: Loading module hook 'hook-distutils.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
3443 INFO: Loading module hook 'hook-encodings.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
3497 INFO: Loading module hook 'hook-gi.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
3504 INFO: Loading module hook 'hook-gi.repository.Gtk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
400045 INFO: Processing pre-safe import module hook gi.repository.GObject from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GObject.py'.
400048 INFO: Processing pre-safe import module hook gi.repository.GLib from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GLib.py'.
400052 INFO: Processing pre-safe import module hook gi.repository.Gio from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gio.py'.
400054 INFO: Processing pre-safe import module hook gi.repository.Gdk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Gdk.py'.
400054 INFO: Processing pre-safe import module hook gi.repository.Atk from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Atk.py'.
400061 INFO: Loading module hook 'hook-lib2to3.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
400134 INFO: Loading module hook 'hook-sysconfig.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
400136 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
400136 INFO: Loading module hook 'hook-xml.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
400396 INFO: Loading module hook 'hook-gi.repository.Atk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
400626 INFO: Loading module hook 'hook-gi.repository.Gdk.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
400776 INFO: Processing pre-safe import module hook gi.repository.cairo from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.cairo.py'.
400777 INFO: Processing pre-safe import module hook gi.repository.Pango from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.Pango.py'.
400778 INFO: Processing pre-safe import module hook gi.repository.GdkPixbuf from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GdkPixbuf.py'.
400778 INFO: Loading module hook 'hook-gi.repository.GdkPixbuf.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
401070 INFO: Processing pre-safe import module hook gi.repository.GModule from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GModule.py'.
401070 INFO: Loading module hook 'hook-gi.repository.Gio.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
401275 INFO: Loading module hook 'hook-gi.repository.GLib.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
401547 INFO: Loading module hook 'hook-gi.repository.GModule.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
401669 INFO: Loading module hook 'hook-gi.repository.GObject.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
401798 WARNING: Hidden import "gi._gobject" not found!
401798 INFO: Loading module hook 'hook-gi.repository.Pango.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
401919 INFO: Loading module hook 'hook-gi.repository.cairo.py' from 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks'...
402977 INFO: Looking for ctypes DLLs
402996 INFO: Analyzing run-time hooks ...
402998 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
403000 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gtk.py'
403001 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_glib.py'
403002 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gdkpixbuf.py'
403003 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gio.py'
403003 INFO: Including run-time hook 'C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/hooks/rthooks/pyi_rth_gi.py'
403008 INFO: Looking for dynamic libraries
403943 INFO: Looking for eggs
403943 INFO: Using Python library c:/msys64/mingw64/bin/libpython3.8.dll
403943 INFO: Found binding redirects:
[]
404783 INFO: Warnings written to C:/msys64/home/Alex/Cybercyte/build/gtktest/warn-gtktest.txt
404815 INFO: Graph cross-reference written to C:/msys64/home/Alex/Cybercyte/build/gtktest/xref-gtktest.html
417313 INFO: checking PYZ
417313 INFO: Building PYZ because PYZ-00.toc is non existent
417313 INFO: Building PYZ (ZlibArchive) C:/msys64/home/Alex/Cybercyte/build/gtktest/PYZ-00.pyz
417314 INFO: Building PYZ (ZlibArchive) C:/msys64/home/Alex/Cybercyte/build/gtktest/PYZ-00.pyz completed successfully.
417323 INFO: checking PKG
417323 INFO: Building PKG because PKG-00.toc is non existent
417323 INFO: Building PKG (CArchive) PKG-00.pkg
417339 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
417340 INFO: Bootloader C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/bootloader/Windows-64bit/run_d.exe
417340 INFO: checking EXE
417340 INFO: Building EXE because EXE-00.toc is non existent
417340 INFO: Building EXE from EXE-00.toc
417341 INFO: Appending archive to EXE C:/msys64/home/Alex/Cybercyte/build/gtktest/gtktest.exe
417360 INFO: Building EXE from EXE-00.toc completed successfully.
417736 INFO: checking COLLECT
417736 INFO: Building COLLECT because COLLECT-00.toc is non existent
417736 INFO: Building COLLECT COLLECT-00.toc
Traceback (most recent call last):
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:/msys64/mingw64/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:/msys64/mingw64/bin/pyinstaller.exe/__main__.py", line 7, in <module>
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/__main__.py", line 114, in run
    run_build(pyi_config, spec_file, **vars(args))
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/__main__.py", line 65, in run_build
    PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 716, in main
    build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/build_main.py", line 663, in build
    exec(code, spec_namespace)
  File "C:/msys64/home/Alex/Cybercyte/gtktest.spec", line 30, in <module>
    coll = COLLECT(exe,
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/api.py", line 698, in __init__
    self.__postinit__()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/datastruct.py", line 160, in __postinit__
    self.assemble()
  File "C:/msys64/mingw64/lib/python3.8/site-packages/PyInstaller/building/api.py", line 742, in assemble
    shutil.copy(fnm, tofnm)
  File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "C:/msys64/mingw64/lib/python3.8/shutil.py", line 261, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: 'C:/ProgramData/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Application Data/Microsoft/Windows Defender/Scans/mpcache-D197A58A47B6D6C0E51AA4D49CFAE9B250397A36.bin.83'

LeighKorbel on 13 Aug 2020

The above Permission denied errors are likely due to #5058.

rokm on 14 Aug 2020

Thanks @rokm, I agree this seems to be the same issue.

@LeighKorbel Can we close this one?

danyeaw on 15 Aug 2020

I would probably wait until a pull request from @ferdnyc is merged before this is closed, as that was the original problem. Once that is complete, then yes, you can absolutely close this out.

LeighKorbel on 17 Aug 2020

5110 was supposed to resolve this, but unfortunately I am still getting the same traceback I was getting before with the GTK hello world minimum example above in MSYS2.

danyeaw on 3 Oct 2020

5110 was supposed to resolve this, but unfortunately I am still getting the same traceback I was getting before with the GTK hello world minimum example above in MSYS2.

Hmm, building the simple Gtk test application under MSYS2 (mingw64) seems to work for me now that #5110 was merged.

I don't suppose you have an old build/dist directory lying around, and are accidentally pulling in old list of files to collect?

rokm on 3 Oct 2020

@rokm I had removed the build and dist directories, and I was pretty sure I installed the latest develop version. I tried again and it looks like everything is in fact working. 🎉

@rokm @Legorooj @htgoebel Thanks for your effort fixing this 😄

danyeaw on 3 Oct 2020

👍1

@LeighKorbel @ferdnyc does this work for you with the development head?

Legorooj on 4 Oct 2020

To be honest, I haven't looked at this for several weeks now. I'm not even caught up on the lastest activity, so I don't have anything useful to add. Sorry.

ferdnyc on 5 Oct 2020

👍1

Closing as appears fixed.

Legorooj on 6 Oct 2020

Pyinstaller: PyInstaller not installing with pip on Python3.8 and latest MSYS2 in Windows

Most helpful comment

All 83 comments

python

`msys-compat` branch updated

Directory searches

User program search path

Python's `sys.path`

Inspecting the build process

Inspecting PyInstaller

5110 was supposed to resolve this, but unfortunately I am still getting the same traceback I was getting before with the GTK hello world minimum example above in MSYS2.

5110 was supposed to resolve this, but unfortunately I am still getting the same traceback I was getting before with the GTK hello world minimum example above in MSYS2.

Related issues

Pyinstaller: PyInstaller not installing with pip on Python3.8 and latest MSYS2 in Windows

Most helpful comment

All 83 comments

python

msys-compat branch updated

Directory searches

User program search path

Python's sys.path

Inspecting the build process

Inspecting PyInstaller

5110 was supposed to resolve this, but unfortunately I am still getting the same traceback I was getting before with the GTK hello world minimum example above in MSYS2.

5110 was supposed to resolve this, but unfortunately I am still getting the same traceback I was getting before with the GTK hello world minimum example above in MSYS2.

Related issues

`msys-compat` branch updated

Python's `sys.path`