Appimagekit: fuse-3?

[2011]

$ /path/to/any.AppImage 
dlopen(): error loading libfuse.so.2

AppImages require FUSE to run. 
You might still be able to extract the contents of this AppImage 
if you run it with the --appimage-extract option. 
See https://github.com/AppImage/AppImageKit/wiki/FUSE 
for more information


$ ls -al /usr/lib64/libfuse3.so*
lrwxrwxrwx 1 root root     13 2020-10-17 05:19:09 /usr/lib64/libfuse3.so -> libfuse3.so.3*
lrwxrwxrwx 1 root root     17 2020-10-17 05:19:09 /usr/lib64/libfuse3.so.3 -> libfuse3.so.3.9.1*
-rwxr-xr-x 1 root root 249552 2020-10-17 05:19:11 /usr/lib64/libfuse3.so.3.9.1*

I found nothing in the documentation mentioning version 3 of fuse, but do note that fuse-3.0.0 arrived in 2016, and the (dead) version 2 has seen no releases in more than two years. I don't understand why you still use version 2.

In searching through issues before making this post, I noticed somebody suggesting that you include (the relevant parts of) fuse in the AppImages. I heartily endorse that idea.

[probonopd]

Please install libfuse2 for now; systems like CentOS 6 and 7 still need it.
Ubuntu xenial, the oldest still-supported Ubuntu LTS release, also uses it: https://packages.ubuntu.com/de/xenial/libfuse2
So we do not want to drop libfuse2 support.

As far as I know, no one has started working on (optional) support for libfuse3 so far.
Would it be feasible to link libfuse statically into the embedded AppImage runtime, and would this solve these sorts of issues? This would increase size for the sake of increasing compatibility.

[TheAssassin]

You could just dlopen() and call the functions from there like it is done with libfuse2 currently. Would be easier. Also a chance to clean up the mess in the runtime, and extract all fuse-related things into some separate module.

[2011]

Would it be feasible to link libfuse statically into the embedded AppImage runtime, and would this solve these sorts of issues? This would increase size for the sake of increasing compatibility.

The entire libfuse3 dynamic library weighs in at less than 250K on my machine (I would expect an even smaller size for libfuse2). The increase in size for each AppImage (assuming you only linked in the relevant function calls) seems absolutely trivial relative to the gains in user experience. Plus it solves your problem of having to switch later. Some distributions have started to get very strict about removing apps that link to unmaintained software (Python 2.7.x or Gtk+ 2.24.x, to name two), and I expect that those distributions won't even package libfuse2 by the end of the year.

[probonopd]

distributions won't even package libfuse2

Then those distributions don't have a very backward/LTS compatible mindset. In my opinion, when a new major version of a library comes along, the last major version should be carried alongside at least until even the oldest still-supported LTS releases come with the new major version, plus some extra grace period.

[2011]

I don't mean to sound too argumenative here, but doesn't most LTS last about two years (people usually don't get that on many phones)? Libfuse2 has (already) died more than two years ago. Dead software doesn't have the number of eyes looking at it for potential security vulnerabilities, and distribution maintainers almost always have limited resources (and do always have demands that far exceed those resources). I can tell you that distribution packagers hate few things more that having to expend the effort required to maintain applications that demand deprecated versions of tools (and the tools often need patching themselves because other things have changed in the meantime).

Some upstream developers will simply never upgrade if they can get away with it (I know software that still requires a version from the last millennium of a common tool). Why doesn't the project upgrade? I have no idea, but can only guess some combination of arrogance, ignorance, and laziness.

[probonopd]

phones

Phones are clearly not our reference point. In fact, most phones become un-usable after just a few years due to them being abandoned shortly after they have been produced. I know people who are still running 10-year old desktop operating systems, ad I am typing this on a workstation that is over 10 years old and still working perfectly fine.

I can tell you that distribution packagers hate few things more that having to expend the effort required to maintain applications that demand deprecated versions of tools

And that is exactly why we need something like AppImage that embraces backward compatibility to older systems. Many distributions simply don't care and that is a problem.

Why doesn't the project upgrade? I have no idea, but can only guess some combination of arrogance, ignorance, and laziness.

Caring about users who do not want, or cannot, upgrade their operating system every couple of years.

In my personal optinion, distributions should continue to ship the -compat ("old") version of a library until it is absolutely clear that everyone (including users of distributions such as RHEL) have the new version. This would promote binary compatibility. Why don't distributions care about binary compatibility? I have no idea, but can only guess some combination of arrogance, ignorance, and laziness.

[2011]

This would promote binary compatibility.

So would not requiring libfuse at all, and placing the function calls needed directly in each AppImage.

Why don't distributions care about binary compatibility? I have no idea, but can only guess some combination of arrogance, ignorance, and laziness.

That seems specious to me. Distributions often have limited resources (and requests from users to add additional packages). Attempting to include deprecated libraries (especially those where the discovery of security flaws makes that more difficult) places a tremendous load on those resources. When the developer of a library moves to a new (and incompatible) version, application developers who continue to use to old library force every distribution that includes that application to do extra work. Eventually that becomes quite selfish (although libfuse2 remains recent enough that I don't mean to include applications that link to it in that description).

[probonopd]

So would not requiring libfuse at all, and placing the function calls needed directly in each AppImage.

True, unless we still need to invoke commands like fusermount that need to be binary compatible (which I have not investigated yet).

places a tremendous load on those resources

Maybe the sweet spot would be to carry (and co-install by default) the current-1 major version, and deprecate it thereafter. Something like that. So that there is an overlap in available major library versions in all distributions (from Enterprise/LTS to bleeding edge) at any point in time. But I suspect that this kind of thinking may be alien to many distributions.

Well, in pragmatic terms, I think we should try what @TheAssassin has suggested:
If libfuse.so.2 cannot be dlopen()ed, then try to dlopen() libfuse.so.3 (and hope that nothing has changed in a breaking way).

If this doesn't work, we can always think about not requiring libfuse (and fusermount, in case we currently do) at all...

[haampie]

@probonopd what do you mean with using fusermount? I grepped it and didn't see it in the sources (including submodules).

I'm kinda interested in a static version of runtime that has libfuse linked into it. For my use case (high-performance computing) I really don't need all the desktop/icon/update/install stuff that come with AppImages, I just need the self-mounting squashfs. Indeed both libfuse3.so and libfuse.so are roughly 250K, so that's a size I can live with. Any obvious downsides to this?

[haampie]

squashfuse is used, not fusermount it seems

[probonopd]

Doesn't squashfuse use fusermount internally? Not 100% sure

[haampie]

I think squashfuse is just calling libfuse functions, and I would presume that ultimately calls fusermount yes, because at least on my distro (ubuntu 20.04) that is a setuid binary

[haampie]

Yeah, one way to figure this out is by running strace -f which dropt the setuid flag for those binaries, and then the program fails:

$ strace -f -o out.txt ./my.AppImage
fusermount: mount failed: Operation not permitted

[haampie]

I quickly looked into this, but the latest tagged version of squashfuse doesn't play well with libfuse 3 (apparently @TheAssassin's patches were upstreamed to Debian for the libsquashfuse0 package -- stuck on the same version then); meaning that most people likely use libfuse 2 anyways. I've created an issue in that repo to ask them to release a new version: https://github.com/vasi/squashfuse/issues/50.

[haampie]

Actually there is a licensing issue when you statically link libfuse, because it is licensed LGPL. dlopen doesn't run into these issues so should be preferred.

[TheAssassin]

Static linking sure isn't the way to go. I wonder why this licensing issue was found just now, not a couple days ago.

I think this is the perfect opportunity for some polymorphy. The strategy pattern comes into my mind, but since this is pure C, well... I guess a struct containing function pointers is going to have to do. We just need 2-3 functions in such an API, wrapping libfuse2 and libfuse3 respectively. A factory can initialize a struct accordingly, preferring libfuse3, but falling back to libfuse2 if needed:

#include <stdio.h>
#include <assert.h>

typedef struct {
    int (*mount)(char*, ...);
    int (*unmount)(char*, ...);
} fuse_handler_t;

fuse_handler_t* make_fuse_handler() {
    return NULL;
}

int main() {
    fuse_handler_t* fuse_handler = make_fuse_handler();

    assert(fuse_handler != NULL);

    fuse_handler->mount(...);

    /* wait until app in AppImage terminates */

    fuse_handler->unmount(...);
}

Such a FUSE handler could even hold some state, if needed.

[haampie]

Pardon my ignorance, but do you really need to call directly into libfuse in AppImageKit at all? Isn't everything handled through SquashFUSE? SquashFUSE 0.1.104 supports both libfuse2 and libfuse3, and the master branch (which is hopefully tagged soon) adds libsquashfuse_ll.a as a separate lib, so you would only have to move the (new) ll_main.c function into AppImageKit's runtime.c if you insist on not invoking squashfuse_ll as an executable.

Maybe squashfuse can be improved a bit to add support for using dlopen to load either libfuse2 or libfuse3 instead of fixing that at compile time.

[probonopd]

squashfuse is compiled in.

[TheAssassin]

We use a specific version of squashfuse, patched to load libfuse with dlopen.

[haampie]

There's currently some active development in squashfuse, I don't think it's a lot of fun to maintain those patches or stick to a fixed vendored version (also for me, cause I want to build an optimized runtime.c in spack without vendored deps (e.g. i want to use the latest zstd for compression)). Wouldn't it make more sense to just open an issue in the squashfuse repo about this use case? The facebook xar people basically have the same issue and are developing squashfuse too; they currently just assume that squashfuse_ll is in the PATH to work around api/abi issues of libfuse, but they also have an open issue where someone wants to link squashfuse_ll.a instead.

[haampie]

https://github.com/vasi/squashfuse/issues/60

[haampie]

They are unfortunately not interested in supporting fuse and fuse3 at the same time