Pyinstaller: import error with very recent changes to pyinstaller 4.0-dev
I think there might be something broken around sys.meta_path since my project started to have the following error in the last few days - beforehand it was building with the 4.0-dev version just fine. So the error looks like this:
import 'distro' # <_frozen_importlib_external.SourcelessFileLoader object at 0x7f655bd2e5f8>
# xml.etree not found in PYZ
# code object from '/usr/local/bin/egnyte/xml/etree/__init__.pyc'
import 'xml.etree' # <_frozen_importlib_external.SourcelessFileLoader object at 0x7f655bd41390>
# xml.etree.cElementTree not found in PYZ
# code object from '/usr/local/bin/egnyte/xml/etree/cElementTree.pyc'
# xml.etree.ElementTree not found in PYZ
# code object from '/usr/local/bin/egnyte/xml/etree/ElementTree.pyc'
# xml.etree.ElementPath not found in PYZ
# code object from '/usr/local/bin/egnyte/xml/etree/ElementPath.pyc'
import 'xml.etree.ElementPath' # <_frozen_importlib_external.SourcelessFileLoader object at 0x7f655bd5d3c8>
import '_elementtree' # <class '_frozen_importlib.BuiltinImporter'>
import 'xml.etree.ElementTree' # <_frozen_importlib_external.SourcelessFileLoader object at 0x7f655bd41550>
import 'xml.etree.cElementTree' # <_frozen_importlib_external.SourcelessFileLoader object at 0x7f655bd41438>
# application not found in PYZ
# code object from '/usr/local/bin/egnyte/application.pyc'
import 'application' # <_frozen_importlib_external.SourcelessFileLoader object at 0x7f655bd416d8>
# client not found in PYZ
# code object from '/usr/local/bin/egnyte/client/__init__.pyc'
import 'client' # <_frozen_importlib_external.SourcelessFileLoader object at 0x7f655bcebda0>
# client.host_interface not found in PYZ
Traceback (most recent call last):
File "egnyte_local_cloud_client.py", line 42, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/synchronizer/python/src/SyncDB.py", line 18, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 728, in exec_module
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "/synchronizer/python/src/common.py", line 31, in <module>
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'client.host_interface'
[24073] Failed to execute script egnyte_local_cloud_client
[24073] LOADER: OK.
[24073] LOADER: Cleaning up Python interpreter.
# clear builtins._
The structure of the project:
python/
|- src/
|- app.py
|- client/
|- __init__.py
|- host_interface.py
...
...
The top-level files and functionality from them can be found just fine. Also if I specify the full path as if python.src.client.host_interface - it also sees the module and imports all the functionality just fine. It's kinda awkward to replace all the imports in the project since it's huge...
klasy
All 48 comments
ModuleNotFoundError in PyInstaller usually just means you need to pass --hidden-import=client.host_interface on build. Does that fix it?
If not then could you clarify which modules are importing each other? I'm guessing app.py is the top level module you are passing to PyInstaller? And it contains import client somewhere in it? And client.__init__ contains from . import host_interface?
And which is your cwd when PyInstaller-ing? And have you either used pip install . or altered sys.path/PYTHONPATH to when setting your project up?
bwoodsend
on 22 Jul 2020
@bwoodsend, thanks so much for suggestion.
tried that - does not fix the problem (just re-checked - in case)
yes, your assumption is correct - app is top level and then it goes down to client and does
from client.host_interface import xyz.
I'm not 100% sure, but from what I know we are not doing pip install since it's our own project - we are just put the exe file into the folder and call it from the systemd service.
oh, and forgot - __init__ is totally clean.
Actually, it's a good idea - let me check the PYTHONPATH.
klasy
on 22 Jul 2020
@bwoodsend - yes, PYTHONPATH is not set:
root@ehs-storagesync:~# echo $PYTHONPATH
root@ehs-storagesync:~# python3
Python 3.7.3 (default, Dec 20 2019, 18:57:59)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/usr/lib/python37.zip', '/usr/lib/python3.7', '/usr/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/dist-packages', '/usr/lib/python3/dist-packages']
Also with pyinstaller 3.6 it works fine... exept for the lorem ipsum bug, that was fixed in 4.0-dev
The entry point is /project/python/src/xyz.py and everything on the same level is imported just fine.
klasy
on 22 Jul 2020
OK, I'm still not entirely sure what's happening. For PyInstaller to find a module, that module needs it to be in sys.path and the only way it could find client is if your cwd, i.e. the '' in sys.path, is python/src (I think). What is your cwd when you run PyInstaller? Are you inside python/src and call PyInstaller app.py or at the root and call PyInstaller python/src/app.py?
bwoodsend
on 23 Jul 2020
ok, this is how the pyinstaller command looks like:
create_pyinstaller_binary() {
# This hook forces pyinstaller to run the application with the default encoding set to UTF8
local pyinstaller_hook=/project/python/src/sitecustomize.py
# This is the entrypoint of our application for which we want pyinstaller to build a standalone package
local entrypoint=/project/python/src/egnyte_local_cloud_client.py
# This will create the folder /project/dist/egnyte_local_cloud_client
pyinstaller --hidden-import=email.mime.multipart \
--hidden-import=email.mime.message \
--hidden-import=email.mime.base \
--hidden-import=email.mime.audio \
--hidden-import=email.mime.image \
--hidden-import=email.mime.nonmultipart \
--hidden-import=email.mime.text \
--hidden-import=pkg_resources.py2_warn \
--runtime-hook=$pyinstaller_hook \
$entrypoint
}
called in docker in sequence after copying the project to /project
the pyinstaller hook is about making sure the environment is utf-8 encoded
klasy
on 23 Jul 2020
I'm fairly certain this is just PYTHONPATH in that case. If you put PYTHONPATH=/project/python/src/ before the PyInstaller call does it change anything? (you may need to correct my bash syntax - it's not something I use often).
Also, when you run your PyInstaler ..... it should create a file called egnyte_local_cloud_client.spec which contains all your --hidden-import=... and other settings. In future you can just call PyInstaller egnyte_local_cloud_client.spec without all the arguments in a bash script.
bwoodsend
on 23 Jul 2020
added export of the PythonPath - same problem... + this same code was building fine just last week - somewhere in the middle of it. Oh, it might be that PYTHONPATH is set somewhere else - not in this particular function - checked the other outputs - it's always there, sorry for the confusion...
also some output to see if it took effect:
29 INFO: PyInstaller: 4.0.dev0
30 INFO: Python: 3.7.3
31 INFO: Platform: Linux-4.19.76-linuxkit-x86_64-with-debian-10.4
32 INFO: wrote /project/egnyte_local_cloud_client.spec
34 INFO: UPX is not available.
35 INFO: Extending PYTHONPATH with paths
['/project/python/src', '/project']
42 INFO: checking Analysis
42 INFO: Building Analysis because Analysis-00.toc is non existent
42 INFO: Initializing module dependency graph...
43 INFO: Caching module graph hooks...
49 INFO: Analyzing base_library.zip ...
2693 INFO: Caching module dependency graph...
I'm pretty sure there should be some changes in the code to cause such a behavior...
klasy
on 23 Jul 2020
Oh well - that's the more obvious causes ruled out. Looking through the git log I'm not seeing anything that seems like a likely candidate for this change. Would you be willing to track down the offending commit? You would use:
To set up:
git clone https://github.com/pyinstaller/pyinstaller.git
cd pyinstaller
pip3 install -e .
Choose a commit ID from git log - anything that isn't either hooks or READMEs.
Then to set a version use git reset --hard commit-id or git reset --hard commit-id~1 to get the version before a commit. There's no need to repeat the pip3 install -e . after this.
Be sure to add the --clean option to your PyInstaller build command.
bwoodsend
on 23 Jul 2020
Ok, let's do that - with docker build it will be relatively easy and fast. Let me get back to you.
klasy
on 23 Jul 2020
@bwoodsend, Some intermediate update:
I checked some commits. Until the 7a7eb11a7bed7024831694f973c3f2cf30fd359e the build fails for me with the following:
/synchronizer/resource/rpm/sync.spec: Replacing '_@sync_version_@' ---> '12.1.1'
32 INFO: PyInstaller: 4.0.dev0
32 INFO: Python: 3.7.3
33 INFO: Platform: Linux-4.19.76-linuxkit-x86_64-with-debian-10.4
33 INFO: wrote /synchronizer/egnyte_local_cloud_client.spec
36 INFO: UPX is not available.
37 INFO: Extending PYTHONPATH with paths
['/synchronizer/python/src', '/synchronizer']
44 INFO: checking Analysis
44 INFO: Building Analysis because Analysis-00.toc is non existent
44 INFO: Initializing module dependency graph...
45 INFO: Caching module graph hooks...
50 WARNING: Several hooks defined for module 'uniseg'.Please take care they do not conflict.
Traceback (most recent call last):
File "/usr/local/bin/pyinstaller", line 33, in <module>
sys.exit(load_entry_point('pyinstaller==4.0.dev0', 'console_scripts', 'pyinstaller')())
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/__main__.py", line 114, in run
run_build(pyi_config, spec_file, **vars(args))
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/__main__.py", line 65, in run_build
PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 716, in main
build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 663, in build
exec(code, spec_namespace)
File "/synchronizer/egnyte_local_cloud_client.spec", line 17, in <module>
noarchive=False)
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 241, in __init__
self.__postinit__()
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/building/datastruct.py", line 160, in __postinit__
self.assemble()
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 340, in assemble
excludes=self.excludes, user_hook_dirs=self.hookspath)
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/analysis.py", line 790, in initialize_modgraph
user_hook_dirs=user_hook_dirs,
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/analysis.py", line 117, in __init__
self._reset(user_hook_dirs)
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/analysis.py", line 136, in _reset
self._hooks = self._cache_hooks("")
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/analysis.py", line 267, in _cache_hooks
return ModuleHookCache(self, hook_dirs)
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/imphook.py", line 99, in __init__
self._cache_hook_dirs(hook_dirs)
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/imphook.py", line 136, in _cache_hook_dirs
hook_module_name_prefix=self._hook_module_name_prefix,
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/imphook.py", line 295, in __init__
'{hook}'.format(hook=self.hook_module_name)
AssertionError: Hook Conflict. Please remove the following custom hook: __PyInstaller_hooks_0_uniseg
Starting 7a7eb11a7bed7024831694f973c3f2cf30fd359e it builds fine, but I have this exception about a module can not be found. I'll try to dig further, but this build error that I mentioned starts with or before 8714423aa56803027b5a5585257392024ea9f7a0.
klasy
on 23 Jul 2020
Ok, any suggestions about this failure? because it goes back to 5feb4167cc55c49cd25f408b6d4b253759a735b9 which is May 21 - way far to go...
klasy
on 23 Jul 2020
I actually have a better idea... can we go through the builds for 4.0-dev - those should be made daily... or do you keep just the last build?
klasy
on 23 Jul 2020
ok, finally found later commit that builds... it has lorem ipsum problem... but :) - going up the stack now... that later commit is beginning of April... 27-th of April it's already broken with the above hook problem.. The earliest good one so far is c6a74f060eef182365d88473961b1b1ea1a8e8d7
klasy
on 23 Jul 2020
can we go through the builds for 4.0-dev
There's no daily build (at least not that I'm aware of). There's testing CI that runs each commit.
I find these lines from your last traceback odd. It appears to be running from an egg. Are you running a really old version of setuptools? I've switched over to my Linux machine and reset to that commit and I can't reproduce that failure.
File "/usr/local/lib/python3.7/dist-packages/pyinstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 241, in __init__
Maybe just use pip install . without the -e. You'll have to repeat this after every git reset ....
Also thinking you might be better off waiting for Legorooj to wake up as he's been on here longer. @Legorooj - Need help! :stuck_out_tongue:
bwoodsend
on 23 Jul 2020
Btw, when you say very recent, surely that's well after April?
bwoodsend
on 23 Jul 2020
Doing install without -e is fine - I'm building new container each and every time - wiping out everything for the cleanness of the experiment... Yes, when I say very recent - it means last week... (when I originally started experience the issue it was like 1 or 2 days, but after that I spent a lot of time googling, investigating and trying to fix it myself before I went here) - 1,5 weeks max...
klasy
on 23 Jul 2020
Ok, I think it's my chroot removal has to do with the problem of the path... is there a way to workaround lorem ipsum problem for a version 3.6 - just wanted to make absolutely sure?
klasy
on 23 Jul 2020
BTW, building without -e helps to remove that build problem ;D
klasy
on 23 Jul 2020
BTW, building without
-ehelps to remove that build problem ;D
Build problem being this one?
I don't actually know what you mean by the lorem ipsum problem?
bwoodsend
on 23 Jul 2020
no, the following one:
44 INFO: checking Analysis
44 INFO: Building Analysis because Analysis-00.toc is non existent
44 INFO: Initializing module dependency graph...
46 INFO: Caching module graph hooks...
Traceback (most recent call last):
File "/usr/local/bin/pyinstaller", line 33, in <module>
sys.exit(load_entry_point('PyInstaller==4.0.dev0', 'console_scripts', 'pyinstaller')())
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/__main__.py", line 114, in run
run_build(pyi_config, spec_file, **vars(args))
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/__main__.py", line 65, in run_build
PyInstaller.building.build_main.main(pyi_config, spec_file, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 735, in main
build(specfile, kw.get('distpath'), kw.get('workpath'), kw.get('clean_build'))
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 682, in build
exec(code, spec_namespace)
File "/synchronizer/egnyte_local_cloud_client.spec", line 17, in <module>
noarchive=False)
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 247, in __init__
self.__postinit__()
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/building/datastruct.py", line 160, in __postinit__
self.assemble()
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/building/build_main.py", line 346, in assemble
excludes=self.excludes, user_hook_dirs=self.hookspath)
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/analysis.py", line 783, in initialize_modgraph
user_hook_dirs=user_hook_dirs,
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/analysis.py", line 117, in __init__
self._reset(user_hook_dirs)
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/analysis.py", line 135, in _reset
self._hooks = self._cache_hooks("")
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/analysis.py", line 266, in _cache_hooks
return ModuleHookCache(self, hook_dirs)
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/imphook.py", line 99, in __init__
self._cache_hook_dirs(hook_dirs)
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/imphook.py", line 132, in _cache_hook_dirs
hook_module_name_prefix=self._hook_module_name_prefix,
File "/usr/local/lib/python3.7/dist-packages/PyInstaller-4.0.dev0-py3.7.egg/PyInstaller/depend/imphook.py", line 290, in __init__
'{hook}'.format(hook=self.hook_module_name)
AssertionError: Hook Conflict. Please remove the following custom hook: __PyInstaller_hooks_0_uniseg
Oh, the lorem ipsum is about jaraco.text hook that was missing for quite some time... It's just a sample file containing this student's hymn in latin Lorem ipsum - actually was real famous...
klasy
on 23 Jul 2020
Ok, so I think I'm pretty confident it's my environment problem... could you point me where to look for a possible problem? What I'm doing - I'm moving our project to py3 from py2, therefore I'm using all the libraries in new versions... So the problem might be the new python3 stuff actually (fundamentally wrong build process), because our project in py2 continues to build just fine with pyinstaller... I'm quite lost though... because chroot is a relative way to create a virtual env (not very good one, but still - it was invented long before docker)... so I would think everything should have been exactly same since I was using py3 to build in chroot and it was working just fine for me
klasy
on 23 Jul 2020
Ok, so I think I'm pretty confident it's my environment problem...
Just to clarify: You no longer think it's a recent change in PyInstaller's development branch that's causing the ModuleNotFoundError: No module named 'client.host_interface'?
bwoodsend
on 23 Jul 2020
I can't think of anything wrong with using chroot and certainly not with switching from 2 to 3.
Actually we're still not sure where this module is getting lost. It could be any of the following:
- PyInstaller doesn't realise we need the
clientmodule (usually solved by--hidden-importwhich didn't work so unlikely). - PyInstaller does realise we need it but can't find it when building (not in
sys.pathwhich we've also tried. Should emit a warning). - It gets bundled but under a wrong name e.g.
python.src.clientinstead of justclient. (I've never seen this happen but your module structure is a bit weird.)
Could we have a full build log so I can check for 1 and 2? Also check the file under build/xref-app.html which I think could tell us all 3 if client is in it.
bwoodsend
on 23 Jul 2020
Yes, I went down to April and was able to confirm that the problem is exhibited even there and then (which should not be true since I've been building the project with py3 and pyinstaller back then just fine... - that one is for sure). The reason I was thinking that might be a pyinstaller itself since I moved to no chroot commit and then rebased back since I had the build problem.. So even rebased back and having chroot in place did not fix it..
klasy
on 23 Jul 2020
Full build log - what do you mean here?
klasy
on 23 Jul 2020
Yeah - the structure is weird - there's probably a story to that since the project is more than 11 years old... hope I do not need to change this structure... That would be epic
klasy
on 23 Jul 2020
Oh, and you want everything in debug? Or just maybe imports?
klasy
on 24 Jul 2020
Ok, this is with no debug, but those debug messages do not tell much more.
klasy
on 24 Jul 2020
This is the bit I was after:
32 INFO: PyInstaller: 4.0.dev0
32 INFO: Python: 3.7.3
33 INFO: Platform: Linux-4.19.76-linuxkit-x86_64-with-debian-10.4
34 INFO: wrote /synchronizer/egnyte_local_cloud_client.spec
36 INFO: UPX is not available.
37 INFO: Extending PYTHONPATH with paths
['/synchronizer/python/src', '/synchronizer']
42 INFO: checking Analysis
43 INFO: Building Analysis because Analysis-00.toc is non existent
43 INFO: Initializing module dependency graph...
44 INFO: Caching module graph hooks...
50 INFO: Analyzing base_library.zip ...
2798 INFO: Caching module dependency graph...
2876 INFO: running Analysis Analysis-00.toc
2906 INFO: Analyzing /synchronizer/python/src/egnyte_local_cloud_client.py
3262 INFO: Processing pre-find module path hook distutils from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
3263 INFO: distutils: retargeting to non-venv dir '/usr/lib/python3.7'
4059 INFO: Processing pre-safe import module hook six.moves from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks/pre_safe_import_module/hook-six.moves.py'.
4793 INFO: Processing pre-safe import module hook urllib3.packages.six.moves from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks/pre_safe_import_module/hook-urllib3.packages.six.moves.py'.
7693 INFO: Processing pre-safe import module hook setuptools.extern.six.moves from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks/pre_safe_import_module/hook-setuptools.extern.six.moves.py'.
7957 INFO: Processing pre-find module path hook site from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks/pre_find_module_path/hook-site.py'.
7957 INFO: site: retargeting to fake-dir '/usr/local/lib/python3.7/dist-packages/PyInstaller/fake-modules'
14251 INFO: Analyzing hidden import 'pkg_resources.py2_warn'
14252 INFO: Processing module hooks...
14252 INFO: Loading module hook 'hook-certifi.py' from '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
14255 INFO: Loading module hook 'hook-passlib.py' from '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
14255 INFO: Loading module hook 'hook-cryptography.py' from '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
14381 INFO: Loading module hook 'hook-Cryptodome.py' from '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
14386 INFO: Loading module hook 'hook-Crypto.py' from '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
14391 INFO: Loading module hook 'hook-pycparser.py' from '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
14391 INFO: Loading module hook 'hook-jaraco.text.py' from '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
14393 INFO: Loading module hook 'hook-keyring.backends.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
14772 INFO: Loading module hook 'hook-sqlite3.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
14808 INFO: Loading module hook 'hook-pkg_resources.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15106 INFO: Processing pre-safe import module hook win32com from '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/pre_safe_import_module/hook-win32com.py'.
15194 INFO: Excluding import '__main__'
15196 INFO: Removing import of __main__ from module pkg_resources
15197 INFO: Loading module hook 'hook-encodings.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15244 INFO: Loading module hook 'hook-sysconfig.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15254 INFO: Loading module hook 'hook-importlib_metadata.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15256 INFO: Loading module hook 'hook-lib2to3.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15268 INFO: Loading module hook 'hook-distutils.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15269 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15269 INFO: Loading module hook 'hook-setuptools.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15673 INFO: Loading module hook 'hook-xml.py' from '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks'...
15706 INFO: checking Tree
15707 INFO: Building Tree because Tree-00.toc is non existent
15707 INFO: Building Tree Tree-00.toc
15708 INFO: Looking for ctypes DLLs
15842 INFO: Analyzing run-time hooks ...
15842 INFO: Including custom run-time hook '/synchronizer/python/src/sitecustomize.py'
15856 INFO: Including run-time hook '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks/rthooks/pyi_rth_pkgres.py'
15857 INFO: Including run-time hook '/usr/local/lib/python3.7/dist-packages/_pyinstaller_hooks_contrib/hooks/rthooks/pyi_rth_certifi.py'
15858 INFO: Including run-time hook '/usr/local/lib/python3.7/dist-packages/PyInstaller/hooks/rthooks/pyi_rth_multiprocessing.py'
15882 INFO: Looking for dynamic libraries
17665 INFO: Looking for eggs
17666 INFO: Using Python library /usr/lib/x86_64-linux-gnu/libpython3.7m.so.1.0
17681 INFO: Warnings written to /synchronizer/build/egnyte_local_cloud_client/warn-egnyte_local_cloud_client.txt
17783 INFO: Graph cross-reference written to /synchronizer/build/egnyte_local_cloud_client/xref-egnyte_local_cloud_client.html
17817 INFO: checking PYZ
17817 INFO: Building PYZ because PYZ-00.toc is non existent
17817 INFO: Building PYZ (ZlibArchive) /synchronizer/build/egnyte_local_cloud_client/PYZ-00.pyz
19264 INFO: Building PYZ (ZlibArchive) /synchronizer/build/egnyte_local_cloud_client/PYZ-00.pyz completed successfully.
19288 INFO: checking PKG
19288 INFO: Building PKG because PKG-00.toc is non existent
19288 INFO: Building PKG (CArchive) PKG-00.pkg
19355 INFO: Building PKG (CArchive) PKG-00.pkg completed successfully.
19356 INFO: Bootloader /usr/local/lib/python3.7/dist-packages/PyInstaller/bootloader/Linux-64bit/run
19356 INFO: checking EXE
19356 INFO: Building EXE because EXE-00.toc is non existent
19356 INFO: Building EXE from EXE-00.toc
19356 INFO: Appending archive to ELF section in EXE /synchronizer/build/egnyte_local_cloud_client/egnyte_local_cloud_client
19373 INFO: Building EXE from EXE-00.toc completed successfully.
19375 INFO: checking COLLECT
19375 INFO: Building COLLECT because COLLECT-00.toc is non existent
19375 INFO: Building COLLECT COLLECT-00.toc
19497 INFO: Building COLLECT COLLECT-00.toc completed successfully.
Building deb for buster...
dpkg-deb: building package 'sync' in 'sync-debian-12.1.1-1-590baa97-amd64.deb'.
Removing intermediate container 8cb6cbd6e5fd
---> 4e40747a251f
Step 19/19 : ENTRYPOINT ["bash", "-c", "cp /synchronizer/sync-* /build"]
---> Running in f13fe462cb60
Removing intermediate container f13fe462cb60
---> 6918f41f54ee
Successfully built 6918f41f54ee
Successfully tagged synchronizer_buster_deb:latest
Creating synchronizer_buster_deb_1 ... done
Attaching to synchronizer_buster_deb_1
synchronizer_buster_deb_1 exited with code 0
You're right - it doesn't say much of use. What about the xref file? Can you get at that?
bwoodsend
on 24 Jul 2020
I can copy it somewhere I guess... or... I can just go to the last container - it should be there in theory - let me check
klasy
on 24 Jul 2020
Don't suppose in future you could maybe have a simpler build process? You seem to have a venv Python environment running with chroot inside docker image, building a large Python app with an unconventional structure.
bwoodsend
on 24 Jul 2020
xref-egnyte_local_cloud_client.html.zip
Yay - was able to copy that
klasy
on 24 Jul 2020
oh, this is helpful - I can see where the problem might be :)
klasy
on 24 Jul 2020
oh, this is helpful - I can see where the problem might be :)
That's a relief - cause I can't.
bwoodsend
on 24 Jul 2020
There're a bunch of backports that I remember being there in chroot... and I can follow on :)
klasy
on 24 Jul 2020
You reckon you're sorted then? We can close this and I can get back to whatever dull task I'm supposed to be doing?
bwoodsend
on 24 Jul 2020
No... not just yet.. I can see here:
client.host_interface InvalidSourceModule
imports: client
imported by: client.client_utils • client.nas.dm2 • client.nas.mapping.local.ad • client.nas.mapping.local.native • client.nas.mapping.virtualization_migration • client.nas.mapping.workgroup • client.nas.permission_view • client.nas.ui.ad • client.platform_utils • client.synchronizer • client.synclock • common • common_controller • egnyte_local_cloud_client.py • modules.cert_manager • modules.health.doctor • modules.health.samplers • modules.host_config • permissions_processor • utils.access_utils • utils.web
But it does not say why? Why would it be invalid?
klasy
on 24 Jul 2020
Hmm, no clue. I'll have a rummage in the source code for that and see if there's anything more illuminating in there...
bwoodsend
on 24 Jul 2020
I don't suppose there is anything weird about that client/host_interface.py?
bwoodsend
on 24 Jul 2020
no... it's pretty ordinary... imports... classes.. nothing out of the blue... ok, yeah - I guess I should stay with this quite a bit longer. But thanks for your help
klasy
on 24 Jul 2020
It's a syntax error apparently.
Got any old print "hello"s in there still? Or \s followed by blank lines?
bwoodsend
on 24 Jul 2020
Thanks for the tip - I was looking for something like that... kinda hard to find... let me run 2to3 on this file and see if that is gonna be of any help
klasy
on 24 Jul 2020
@klasy did you fix this? IE was this an issue with PyInstaller or not?
Legorooj
on 31 Jul 2020
yes, I got everything working - it turned out to be the import issue actually in my code. But we were able to find it out only with the help of that xref-*.html file
klasy
on 29 Aug 2020
@klasy excellent - I think the documentation on how to debug issues like this needs updating. @bwoodsend could you either handle or delegate this task?
Legorooj
on 29 Aug 2020
It really does need doing. I've been meaning to write a more complete how to debug PyInstaller issues for a while. I'll create an issue for it...
bwoodsend
on 29 Aug 2020
See #5117
bwoodsend
on 29 Aug 2020
Related issues
Merkwurdichliebe
·
57Comments
tnightengale
·
42Comments
LeighKorbel
·
83Comments
embryo10
·
50Comments
AlexFDias
·
37Comments
Most helpful comment
This is the bit I was after: