Supertux: Please provide an AppImage for SuperTux

[fusion809]

Hi,

I'd like SuperTux to be distributed officially as both source code (which is already provided, of course, from this repository's releases page) and an AppImage binary package. AppImages are cross-distribution packages that can run on any Linux distribution (regardless of package manager or its set of pre-installed software), that is reasonably up-to-date (what "reasonably up-to-date" is in this context depends on how one builds the AppImage) and are essentially self-mounting image files (created in a similar way to ISO files). They contain their own internal file system, complete with essential system libraries that the package they provide relies upon, along with executables (like /usr/bin/supertux2 would be in an AppImage for SuperTux 2). I have attempted to build my own SuperTux AppImage, specifically for 0.5.0, using maths22's Debian packages (i.e., the ones he builds here), but this has failed quite miserably and hence I am here to ask if SuperTux developers can help me.

Thanks for your time,
Brenton

[maxteufel]

The solution to this is distributing a tarball that contains all the necessary stuff, similar to the way Mozilla ships Firefox binaries for Linux, or SuperTuxKart ships their binaries.

If we decide to use AppImage, we're just choosing one of the thousand "app image" solutions out there. Be it AppImage, Flatpak, or Snappy Core. If people can't agree on a standard there, the same thing as with packaged stuff will happen - we'll have to maintain thousands of different versions.

[fusion809]

AppImages have clear-cut advantages over Flatpak and Snap. For one, they are not installed. They are just made executable (with chmod +x) and run (with ./<AppImage> where <AppImage>, is of course, replaced with the file name of the AppImage). Secondly, they need no programs to be installed in order to be run. Flatpak needs the flatpak program to be installed in order for flatpaks to be installed and then run. Likewise Snap packages need the snapd program to be installed and set up in order for the Snap packages to be installed and run. As they need no special tools to be run AppImages can run on any Linux distribution, regardless of whether or not flatpak or snapd are in their software repositories. Likewise both flatpak and snapd have a dependency on systemd, while AppImages do not. A few important distributions are still not shipping with systemd, including:

• Gentoo Linux
• Slackware Linux

so while AppImages will run on these distributions without any modifications, Flatpaks and Snap packages will not.

[maxteufel]

That's still not a solution to the actual problem. If the problem of portable binaries on Linux is to be actually solved, then the probably best idea would be that the authors of AppImage, Flatpak, Snappy Core, people from various distros (those that use systemd and those that don't use systemd), people creating desktop environments, and even people from various operating systems (Linux, other UNIX) sit together and work on a common standard that people can agree on to distribute binaries. If that does not happen the initial problem won't be solved, it will just be shifted to some other place.

[fusion809]

That would be wonderful, but unfortunately, I don't see that happening anytime soon. So in the mean time, what do we do? Do we try to provide a cross-distribution package for SuperTux, or refuse to do anything out of the principle you just outlined and let Linux users go without? Not trying to be rude, I'm just wondering what we do until then...

[maxteufel]

That would be wonderful, but unfortunately, I don't see that happening anytime soon.

If you haven't attempted to start such a discussion yet, I'd encourage you to do so. It's worth giving it a try.

So in the mean time, what do we do? Do we try to provide a cross-distribution package for
SuperTux, or refuse to do anything out of the principle you just outlined and let Linux users go
without?

First of all, OBS builds exist for now. Additionally (from https://github.com/SuperTux/supertux/issues/648#issuecomment-257167579):

The solution to this is distributing a tarball that contains all the necessary stuff, similar to the way
Mozilla ships Firefox binaries for Linux, or SuperTuxKart ships their binaries.

[maxteufel]

Just for information, making the binary relocatable is probably still a valid issue and a ticket should be opened separately for that. It might make building AppImages easier, in case you want to create an unofficial one (which you're of course free to do).

[fusion809]

If you haven't attempted to start such a discussion yet, I'd encourage you to do so. It's worth giving it a try.

How would I do that exactly? It's not like as though AppImage, Flatpak and Snap developers aren't aware of each other. They know each other exist, but they believe their solution is the best (which is invariable, as we all have different opinions), and I doubt a conversation is going to change that.

Your binary tarball solution sounds like a less than adequate solution as I doubt the tarball is going to include all the libraries (including common system libraries) required by the program. It's going to have its own set of dependencies, which will vary according to distribution. Hence we will have to add instructions for each distribution on how to install the required libraries. While an AppImage can be downloaded and run, without the need for installing any programs or extra libraries.

[maxteufel]

How would I do that exactly? It's not like as though AppImage, Flatpak and Snap developers aren't
aware of each other. They know each other exist, but they believe their solution is the best (which is
invariable, as we all have different opinions), and I doubt a conversation is going to change that.

Well I doubt it's going to find large adoption if everyone thinks their system is the best. You might or not notice, but you're also trying to convince us that one particular option is the best.

Your binary tarball solution sounds like a less than adequate solution [...]

It's worked well enough for Mozilla and others for a long time. (Though I can agree it's not as ideal as shipping fully static binaries similar to those Go can create.)

[fusion809]

I realize I'm trying to convince ya to use one particular option. There's good reason why, I have tried building Snap packages and Flatpaks and they are far more challenging than AppImages. Plus AppImages as I said need no installation, need no extra tools to be run, and have several other advantages over them. But if you insist on a tarball and are unwilling to consider using an AppImage instead, at least it's better than no cross-distro binary package at all.

[maxteufel]

@fusion809 I personally still think we shouldn't decide on one option in this case. I don't know what the others think about it (cc: @maths22 @tobbi).

What I believe we can do is make it possible to build AppImages (see https://github.com/SuperTux/supertux/issues/648#issuecomment-257169025) so people can provide unofficial AppImage builds.

If that is possible in the future, and people help us understand how AppImages work, and people use AppImages, we might revisit that option in order to provide automated AppImage builds, but for now I don't think we need to spend any specific effort on this.

[probonopd]

@maxteufel AppImage developer here. The different solutions do serve slightly different needs. AppImage is by far the simplest of them all: just a self-mounting filesystem image. You decide what you put inside. When the user executes it, it gets mounted and executes what you have put inside. That's basically it. Needs no further support from the OS besides a working FUSE setup (which more or less all desktops have out-of-the-box these days).

[maxteufel]

I do know what it is. What's more annoying to me is that if we say we support AppImage, we say that option X is better than other options out there. Plus, users who prefer alternate distro-agnostic packaging methods (e.g. Flatpak, Snappy) might want support for that added too (that leaves us back with the original problem, of having to maintain different package manager stuff).

Another point up for discussion here: it would be amazing if a generic Linux package format could also work in a similar way for non-Linux UNIXes. @probonopd Is it possible to get AppImages working on BSDs, Solaris derivatives, etc. too? I'm aware this probably can't go into the same AppImage, but it would be amazing if I could just spin up e.g. a FreeBSD VM, build SuperTux, and then package it into an AppImage for all FreeBSD users.

[probonopd]

What's more annoying to me is that if we say we support AppImage, we say that option X is better than other options out there

Why? If you package as a .deb, does that mean that you say .rpm is worse?

users who prefer alternate distro-agnostic packaging methods (e.g. Flatpak, Snappy) might want support for that added too

Possibly. Isn't it better to at least offer one (e.g., the one that has the least requirements on the target system, i.e., AppImage) than nothing; and have others possibly contribute the rest if you don't want to do all of them yourself?

Is it possible to get AppImages working on BSDs, Solaris derivatives, etc. too?

If FUSE is available on those platforms, I see no reason why it shouldn't work.

[maxteufel]

Why? If you package as a .deb, does that mean that you say .rpm is worse?

Well, these were designed for more specific use cases, not as intentional standards. And we all know about standards.

Possibly. Isn't it better to at least offer one (e.g., the one that has the least requirements on the target
system, i.e., AppImage) than nothing; and have others possibly contribute the rest if you don't want to
do all of them yourself?

True, reason why we wanted tarballs then.

If FUSE is available on those platforms, I see no reason why it shouldn't work.

FUSE seems to be problematic on BSD. Why does AppImage need it? With generic tarballs we don't have this problem. We package one that includes all the libraries, binaries, (<-- obviously for the given platform) and data, add a wrapper script that runs the binary and done.

[probonopd]

And we all know about standards.

Yes... https://plus.google.com/105493415534008524873/posts/NiQmcwWL4ob

FUSE seems to be problematic on BSD. Why does AppImage need it?

So that you don't have to unpack your 6 GB game AppImage before you can run it. Because it is mounted at runtime.

With generic tarballs we don't have this problem.

You could also loop-mount (using the kernel) or extract the AppImage (the newer ones support --appimage-extract).

[maxteufel]

You could also loop-mount (using the kernel) or extract the AppImage (the newer ones support
--appimage-extract).

So if it supports command line options how is it run by the OS? It appears that AppImages aren't so much different from tarballs containing all required libraries, are they?

[fusion809]

No they aren't. They're essentially like complete (i.e., with all the required libraries and executable files) tarballs, they do not even need to be extracted in order to be run. I'm gonna opt out of notifications from this issue, as my email inbox is getting crowded so if yas need me please ping me with @fusion809.

[probonopd]

They are more like .dmg disk images on macOS.

[fusion809]

I tried running the Corebird AppImage you built, @probonopd, on a FreeBSD 11.0 VM and it failed with the error:

ELF binary type "0" not known.
zsh: exec format error: ./Corebird-0.9.glibc2.14-x86_64.AppImage

not sure if that's what you expected given @maxteufel's previous comment about insufficient FUSE support on BSDs.

[probonopd]

I wouldn't have expected a Linux AppImage to run on FreeBSD. But possibly one can do FreeBSD AppImages.

[maxteufel]

It looks like FreeBSD doesn't support FUSE natively anyway, so an AppImage probably won't work.

After having looked at AppImages more closely, it appears that AppDirs look very similar to what we wanted to package in tarballs. AppImages just seem like a different way of packaging them into a single file.

From the AppImageKit README:

The file AppRun inside the ISO9660 file is not the actual executable, but instead a tiny helper binary
that finds and exectues the actual app. Generic AppRun files have been implemented in bash and
as parts of AppImageKit. The C version, AppRun.c, is generally preferred as it is faster and more
portable.

Where can I find this bash AppRun thingy?

Edit: I guess if we wanted to support AppImages we'd distribute Linux packages as AppImages, and for other platforms it seems like archived AppDirs would work, if compiled for that platform.

[qwertychouskie]

@maxteufel When creating an STK AppImage, I just renamed the script run_me.sh to AppRun and it worked.

[probonopd]

AppDirs look very similar to what we wanted to package in tarballs

Cool :-)

Where can I find this bash AppRun thingy?

You can write your own, just a bash script that launches the app after having set up LD_LIBRARY_PATH and whatnot. (Or you could use ours from AppRun.c)

it seems like archived AppDirs would work, if compiled for that platform

Yes.

[maxteufel]

When creating an STK AppImage, I just renamed the scriptrun_me.sh (sic!) to AppRun and it worked.

Can you provide more details about that run_me.sh script and the general build process, @qwertychouskie?

[qwertychouskie]

I pretty much just grabbed the Linux .tar.xz file from https://supertuxkart.net/Download, renamed run_supertuxkart.sh to AppRun and put it in the root dir, and used the AppImage packaging instructions to turn it into an AppImage. The STK devs could probably give more info about how the .tar.xz is made.

[probonopd]

https://github.com/probonopd/AppImages/blob/master/recipes/meta/SuperTux2.yml seems to work. Would the project like to provide an official, upstream-generated AppImage?

[maxteufel]

Would the project like to provide an official, upstream-generated AppImage?

Looks like we missed that when it was posted, but yes. And ideally without taking the pre-built packages from OBS, but creating the AppImages upon building on Travis (this integrates best with our current workflow for deployment of releases, and later on would facilitate providing nightlies).

[probonopd]

@maxteufel see these examples for packaging binaries built on Travis CI as an AppImage upon each commit.

[maxteufel]

Thanks! I'm going to take a look at this soon.

[probonopd]

Thanks for looking into this @maxteufel. Please let me know if you need more information. Let me summarize, providing an AppImage would have, among others, these advantages:

• Works for most Linux distributions (including Ubuntu, Fedora, openSUSE, CentOS, elementaryOS, Linux Mint, and others)
• One app = one file = super simple for users: just download one AppImage file, make it executable, and run
• No unpacking or installation necessary
• No root needed
• No system libraries changed
• Just one format for all major distributions
• Works out of the box, no installation of runtimes needed
• Optional(!) desktop integration with appimaged
• Binary delta updates, e.g., for continuous builds (only download the binary diff) using AppImageUpdate
• Can GPG2-sign your AppImages (inside the file)

[maxteufel]

@probonopd This tool that allows building from Travis CI builds should be documented. The examples might be a good start, but it doesn't say anywhere what the tool expects. It's a bit too much magic. Plus, it doesn't even tell you what's wrong (see https://travis-ci.org/SuperTux/supertux/builds/196894532#L8940-L9140).

[maxteufel]

I think I fixed what was wrong with that (aa6816eed0047e2f0753cb22bcde68797d604afc). This could be avoided by proper documentation.

[probonopd]

So you are creating a type 1 AppImage rather than a type 2 AppImage now. That is the older format. This is not a fix. The error is

supertux.png{.png,.svg,.svgz,.xpm} not present but defined in desktop file

Please make sure that you have Icon=supertux (__without__ a filename extension) in your supertux.desktop file, and a supertux.png icon file in the top level directory (same directory as the supertux.desktop file).

[maxteufel]

@probonopd I fixed that now in another commit on that branch. I can only mention again, proper documentation helps.

However, when trying to use the AppImage now it fails because something links to usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-4.0.so. The problem is that LD_LIBRARY_PATH as defined by AppRun does not include usr/lib/x86_64-linux-gnu/pulseaudio/.

[maxteufel]

@probonopd Ideally, functions.sh should provide a function that will customize the AppRun binary based on what libraries have been copied over to the image.

[maxteufel]

@probonopd Also, the libkrb5.so.26 being in the blacklist will cause breakage on distros that don't provide a libkrb5.so.26 (Arch Linux ships libkrb5.so.3 only), unless you relink to libkrb5.so.

[probonopd]

The problem is that LD_LIBRARY_PATH as defined by AppRun does not include usr/lib/x86_64-linux-gnu/pulseaudio/

Do you know a reliable way to know all possible places where libraries might be placed? I do https://github.com/probonopd/AppImages/blob/8c3bb1286607c9a2f7b119df2112b0eb885e4bf8/recipes/meta/Recipe#L246-L254

if [ -d "./usr/lib/x86_64-linux-gnu/gstreamer-1.0/" ] ; then
  mv ./usr/lib/x86_64-linux-gnu/gstreamer-1.0/* ./usr/lib/x86_64-linux-gnu/
  rm -r ./usr/lib/x86_64-linux-gnu/gstreamer-1.0
fi

if [ -d "./usr/lib/x86_64-linux-gnu/pulseaudio/" ] ; then
  mv ./usr/lib/x86_64-linux-gnu/pulseaudio/* ./usr/lib/x86_64-linux-gnu/
  rm -r ./usr/lib/x86_64-linux-gnu/pulseaudio
fi

Maybe we should put this into functions.sh too.

[probonopd]

@probonopd Ideally, functions.sh should provide a function that will customize the AppRun binary based on what libraries have been copied over to the image.

To achieve what exactly? Are you aware that you can put your own, totally custom bash script as an AppRun file? If that is not sufficient, please elaborate, ideally on a new issue in https://github.com/probonopd/AppImageKit.

[probonopd]

@probonopd Also, the libkrb5.so.26 being in the blacklist will cause breakage on distros that don't provide a libkrb5.so.26 (Arch Linux ships libkrb5.so.3 only), unless you relink to libkrb5.so.

Which solution do you recommend? 99% of the time the Kerberos stuff is not actually needed but only there as a dependency because libcurl was compiled that way. libcurl compiled without Kerberos support would get rid of that dependency. Discussion regarding this here.

[maxteufel]

@probonopd The moving library stuff seems like a good solution that doesn't require changing default AppRun and probably works.

As to the Kerberos problem, I can't really recommend a solution. I don't know if this works, but I'd give it a try: for every library and binary that links to libkrb5 (check that before), use sed to remove the .3/.26 in the file. I know that for macOS, there's a tool that is actually designed to modify dynamic linking references, I'm not sure if there's a solution besides sed to do so on Linux.

[alexlarsson]

Hi, I'm the flatpak developer, and I happened to need an app to test OpenGL stuff, so i made a flatpak package of supertuxkart: https://gist.github.com/alexlarsson/55d2f1ea8307a808aa94280a8a4ca945

If you want to test it, you can build it and install it locally like this:

# Install runtime and sdk
flatpak --user remote-add --if-not-exists gnome https://sdk.gnome.org/gnome.flatpakrepo
flatpak --user install gnome org.freedesktop.Sdk//1.4 org.freedesktop.Platform//1.4
# build the app
flatpak-builder --force-clean --repo=repo app net.supertuxkart.SuperTuxKart.json
# install it
flatpak --user remote-add --no-gpg-verify  local-supertuxcart repo
flatpak --user install local-supertuxcart net.supertuxkart.SuperTuxKart

At this point supertuxkart is installed and available in the regular desktop menus. (You might need to log in/out the first time after you installed flatpak before it appears in the menus.)

You can also launch it manually as flatpak run net.supertuxkart.SuperTuxKart

If you copy the repo directory to a webserver others can also install/update from there. However, in that case i recommend you sign the build with a gpg key (pass --gpg-sign=KEY to flatpak-builder).

This uses the freedesktop sdk/runtime, so you don't need any development packages on the build machine, and all users will be using exactly the same files to run the game, so no cross-distro issues.

[maxteufel]

@alexlarsson You might want to read up on the previous comments about why we're not interested in flatpak. We're not interested in your solution until you fix the issues mentioned. Plus you might want to make sure what project you're even talking about. This is not SuperTuxKart.

[alexlarsson]

Ah. I just randomly packaged supertuxkart today and got reminded of this issue which is in my inbox because someone mentioned me. Sorry for not looking closely enough to see it wasn't actually about supertuxkart.

[alexlarsson]

That said, with flatpak you'll never run into the kind of issues you're having with libkrb5, because what dependencies you use is fully under your control.

[probonopd]

...at the expense of having to install flatpak in the base system, and having to install a runtime using flatpak. I don't think SuperTux really _needs_ Kerberos. Might be wrong though.

[alexlarsson]

Sure, every design decision has a cost. The flatpak design favors robustness and predictability (i.e. the user run exactly the same code you tested on) over reusing dependencies from the base os. I believe this is the right choice, but clearly not everyone agrees.

[probonopd]

Which is why, as always, choice is _good_, but on this also clearly not everyone agrees ;-)

[maxteufel]

A preliminary test build is now available at https://transfer.sh/ajpum/SuperTux_2-v0.5.0-230-g8aa1f1766-glibc2.14.glibc2.14-x86_64.AppImage (SHA256: 74d1b981d87d06362d0df786aa90b198582257f1e583aeb4f5d639603334a4a2). Please note that this link will expire in about two weeks.

Until resolved by upstream AppImage, we will patch the library blacklist so the binaries work on Arch Linux as well.

[probonopd]

Thanks @maxteufel. Works for me on ubuntu-16.04-desktop-amd64.iso but is relatively large; binaries are not stripped. If you strip the binaries before creating the AppImage, then the download size should decrease a bit.

[maxteufel]

@probonopd That should be done indeed, at least for release builds.

When I start the binary on Arch, everything works fine except it prints these crash messages:

/tmp/.mount_eErAMR/usr/bin//supertux2.wrapper: line 121: 17803 Aborted                 (core dumped) LD_LIBRARY_PATH="" zenity --question --title="$TITLE" --text="$TEXT" 2> /dev/null
/tmp/.mount_eErAMR/usr/bin//supertux2.wrapper: line 121: 17809 Aborted                 (core dumped) LD_LIBRARY_PATH="" zenity --question --title="$TITLE" --text="$TEXT" 2> /dev/null

Why does it even try to launch zenity?

[probonopd]

Have a look inside usr/bin/supertux2.wrapper, it is a bash script that asks the user whether they want to install a desktop file into the menu. If you don't want this, then you can delete supertux2.wrapper and change the desktop file to launch supertux2 rather than supertux2.wrapper.

[maxteufel]

If it works that's nice, if it doesn't but also doesn't affect whether the actual thing works, it's fine too. So we'll keep it like that.