Nuitka: Package all standalone resources into a single file option

[maaaaz]

Hello @kayhayen,

Let me first greet you for that project who opens up new horizons, really.

I would like to bring an idea to improve it (at least, for my usages): could you propose an option to pack all standalone resources into one single file like the --onefile option for pyinstaller ? IMHO it's even more convenient for users to manipulate a single file than rather use entire folders.

Pyinstaller packs all resources into a single zip file (which is executable) and unzip them at runtime (check one of my compiled tools, for instance https://github.com/maaaaz/tibcopasswordrevealer) to see what it looks like).

Cheers!

[maaaaz]

I agree with you, my wish is to have only one standalone file: indeed the zip solution is not as a elegant as the DLL-embedded one.

[kayhayen]

On the mailing list, there was once a discussion. There is a library that py2exe uses, and that could also be integrated with Nuitka. I won't be doing that though.

[kayhayen]

@brupelo There is no point is telling people to not want what they want, because they still are going to want it, until provided with a better alternative. So please do not discuss this issue in that way. Are you OK with me deleting the comments of yours, that I feel are distracting others from knowing that a solution is actually welcome?

For all I know, it's possible to package all in one and people do it. Clearly at least pyinstaller does it, and it makes life easy for deployment, surely.

On the mailing list back then, concerns were raised, if this is something in scope of Nuitka, or if it is not to be a third party tool instead. Actually, if somebody were to look at how pyinstaller or py2exe do it, we might well find they are doing the same thing, or using the same library. If integration with Nuitka has any benefits other than convenience, I don't know yet, I would have to see the solution.

But like I said, one of the people who wants to have it, needs to create the PR and implement it in Nuitka, not necessary by implementing the loading of DLLs, which I understand is duplicating Windows functionality, but by using existing solutions. Feel free to go ahead if you are one of them, that is what the "help_needed" tag is for.

My hope is that this is actually only a matter of research, and not so much going to be a burden for actual Nuitka maintenance, with this being generic work, and that there could be an upstream responsible for problems if there are any.

[pykong]

I wonder whether PEP441 could be of any use in this matter?
One implementation is shiv.

[brupelo]

I remember I had tried this twitter utility couple of years ago called PEX and it was also implementing PEP441. Here's a nice video from one of the authors that explains the underlying logic, it's actually quite simple (but interesting) stuff.

Not sure though whether that approach would allow you to load .pyd files though :/

Anyway, here's some links for those who want it to give it a shot to this feature (DISCLAIMER: windows stuff only):

• https://github.com/sudachen/Molebox <-- abandoned, doesn't build properly on vs2010 so it needs a little bit of hacking, ie. build system is broken
• enigma protector <-- commercial
• https://blog.kowalczyk.info/article/d7d08da8d2c643349463292a051293b5/embed-dll-in-an-exe.html

[brupelo]

I've made a very fast test, which it's adding to a .zip file multiple .pyd files. Python3.6.x won't be able to load them, as expected... so I guess it can only load .pyc/.pyo/.py files from zip files, feels reasonable btw.

Ps. https://utcc.utoronto.ca/~cks/space/blog/python/ZipimportAndNativeModules

[golyalpha]

I think UPX could be a possible solution here. While PyInstaller is capable of packing files without it, most projects I've seen end up using UPX anyways because of it's performance (fast and low overhead unpack, and great compression ratio).
UPX

Nevermind, I jumped the gun a little there. UPX only provides compression of single files (still a useful thing to have with a 20%+ compression ratio), It cannot pack multiple files into one exe.

I just went ahead to look at the --onefile code for PyInstaller, and it does seem it uses it's own stuff: link