Appimagekit: Ideas for proper multiarch/universal binary/whatever_the_name support.

[harrytuttle]

(I was confused by the existing feature/multiarch branch, then i understood that it is only useful to build standard appimages for foreign architectures)

What about real (but only optional) multiarch support? Only 1 appimage that runs on both i386 and amd64 (and possibly x32 if anybody cares?). Or maybe only 1 appimage that runs on armv5, raspberry1/armv6, armv7 and armv8. Or possibly some other more exotic cpu family with binary compatibility.

Of course this would only make sense for apps with large amount of arch-independent data compared to the binary code, but for example games old and new would benefit, even if the world is rapidly discarding i386.

How could it work? Right now, to give every user the traditional "double click to run experience", an arch-dependent, dinamically linked and necessarily tiny (32KB) "runtime" is injected in the iso header, then it calls a not necessarily tiny, not necessarily dinamically linked, not necessarily arch-dependent "AppRun" (11KB).

The easy part is patching AppRun to make it search the correct binary in bin/x86_64-linux-gnu/, or bin/i386-linux-gnu/, or defaulting to bin/ for arch-independent scripts and backward compatibility. Same with LD_LIBRARY_PATH.

The tricky part is running the same AppRun on both architectures, with no guarantee that a suitable libc will be there. At present AppRun is a very simple command line tool. If it's not supposed to grow with a gui, self-updating or other complex management stuff it could be a statically linked i386 binary, around 700KB. Zisofs compression or even better upx compression will bring it down to a reasonable 240KB. Linking it with uclibc (for example using the convenient openwrt toolchain) will bring it down to a 50KB uncompressed / 26KB upx-compressed. If appimages will one day require some kind of standard gui interface or management engine it could be provided by several relatively small dynamically linked binaries shipped within the multiarch directory structure and called accordingly. Or maybe by that time there will be widespread support for fatelf, or AppImages will be forgotten, or will have taken over the world and a proper sandbox/binfmt/mimetype handler will be installed on every distro under the sun, obsoleting the shipped runtime and AppRun entirely.

The other very tricky part is running the same "runtime" on both architectures. It's a much more complex beast: it requires signals, strings, zlib, pthread, plus its own iso9660 implementation, plus libfuse and according to ldd even libdl, i don't know the details. I can't even build it statically, let alone shrink it down to 32KB. I don't know if it's possible to reimplement it in a tiny static envelope, maybe not, and surely not by me. But several of the current tiny runtimes could be hidden uncompressed somewhere in the zisofs image (for example concatenated to AppRun, or as elf segments in AppRun, or elsewhere), and an ad-hoc tiny static executable could replace the single runtime in the image header, it could fopen argv[0], check the running architecture, seek to a known offset, extract the correct runtime in /tmp/.mount-xxxxxx/runtime or /tmp/.runtime-yyyyyy and execv it with the appimage path as argv[1]. (I doubt a file descriptor born out of a memfd_create syscall could be fexecv-ed, and it is only available from kernel 3.17).

If this scheme is not too crazy or naive to work the only visible change for the user would be the lack of a second download link for the 32 or 64bit version of the software, and only for those apps that would benefit from a unified appimage due to space, bandwidth or ease of use concerns. AppImageAssistant would need to carry as binary requirements a few versions of precompiled runtimes and AppRuns (and potentially upx) like it does with xorriso, or it would need to rely on qemu and docker or some other vm container, or it would need to cross compile them all (very handy on debian systems with all those easy to install gcc-4.7-arm-linux-gnueabi, gcc-4.8-powerpc-linux-gnu, and so on). It could also autodetect the presence of several executables from different architectures and embed the right number and kind of runtimes. It could also warn the image creator about currently unavailable combinations, like arm-linux-gnueabihf and x86_64-linux-gnu folders (or binaries) in the same appimage.

It is not currently possible to use an arch-indepentent shell script to extract the runtime (like those self-extracting archives) without triggering an ugly and inconvenient dialog on most desktop environments (but it could be possible to replace AppRun with a shell script). In the unlikely case that fatelf gains momentum the tiny runtime extractor could cover several cpu families (i'm unsure about fatelf overhead). And with cooperating downstream distributors all this could become redundant. But for now i don't see any other way of achieving a similar outcome.

[shoogle]

There have been steps towards universal binaries in the past, but there are many difficulties that still need to be overcome. However, it's worth mentioning that runtime and/or AppRun could be interpreted scripts (e.g. bash) rather than binary executables. Interpreted scripts would be architecture-independent.

[probonopd]

We had looked into this many years ago, and concluded that you had to either

• Use something like FatELF - we did not follow this since it is not in the upstream kernel
• Require some sort of runtime/agent to be installed on the host system before you would be able to use AppImages - we proudly did away with such a requirement earlier and don't want to re-introduce it
• Use a script instead of the runtime.c binary as the executable header of AppImages - we had this in the very beginning (when the project still was called klik) and file managers totally messed up the images, trying to open them in text editors which promptly crashed in return

Are there other options?

[TheAssassin]

As there are no "interpreter-less binaries", I guess we can close this as "unsolvable". Script based options can never be really reliable, as they have even more dependencies on the system side.

[probonopd]

Script based options can never be really reliable, as they have even more dependencies on the system side.

...and since text editors would try to open AppImages, which we had in the long gone past. It was simply not practical.

However, if someone has a good idea how to do multiarch, please let us know. We are interested in it.

[TheAssassin]

They would not if MIME type integration would be done properly. That's solvable, but requires stuff to be installed/set up properly on the client systems.

I think we should rather push FatELF again...

[probonopd]

I think we should rather push FatELF again...

:+1:

[shoogle]

In the meantime, launcher applications like AppImageLauncher could detect when the user tries to run an AppImage compiled for the wrong architecture and show a human-understandable error message along the lines of:

"This application must be run on computers with x86_64 (64-bit Intel/AMD) processors. Your computer has an i686 (32-bit Intel/AMD) processor. Check the application website to see if they have a version for i686 processor architectures."

(Maybe it does this already - I haven't checked.)

[TheAssassin]

@shoogle that's a good idea. You should open an issue over there!