Peek: Provide AppImage

[phw]

An AppImage would make it easy to run Peek without having it in package repositories.

There has been some initial discussion on https://github.com/phw/peek/issues/15#issuecomment-252522566

There is an recipe at https://github.com/probonopd/AppImages/blob/master/recipes/meta/Peek.yml

[phw]

I tend to like the way of building it with Travis CI. This might be interesting:

Maybe interesting: https://github.com/probonopd/AppImageKit/wiki/Docker-Hub-Travis-CI-Workflow

[phw]

One thing that concerns me a bit is the fact that the app image built with the Peek.yml recipe from above (built on Ubuntu 16.04) does not run on my arch linux due to missing symbols in libpangoft2:

./Peek-0.9.1.glibc2.14-x86_64.AppImage: symbol lookup error: /usr/lib/libpangoft2-1.0.so.0: undefined symbol: hb_buffer_set_cluster_level

@probonopd: Are there some best practices on how to build AppImages that are compatible with different Linux distributions as good as possible?

[probonopd]

I have it almost running, but right now I am getting GLib-GIO-ERROR **: Settings schema 'com.uploadedlobster.peek' is not installed.

Can you load the schema from a relative path (relative to the main executable, ../share/glib-2.0/schemas/ relative to the path where peek executable is, which can be found from /proc/self/exe) rather from the default absolute path /usr/share/glib-2.0/schemas/?

I think this can be done around here.

Similarly, could you look for avconv and any other required helper binaries next to the peek executable in addition to the $PATH?

That way it should continue to work when installed traditionally, but would also make the application relocateable.

https://people.gnome.org/~gcampagna/docs/Gio-2.0/Gio.SettingsSchemaSource.new_from_directory.html

A workaround would be to use a wrapper bash script that exports $XDG_DATA_DIRS and $PATH, but I'd like to avoid that and use a cleaner solution if possible.

[probonopd]

Also I am a bit confused how avconv works. It seems like it also needs ffmpeg for converting from avi to gif. Why not use ffmpeg to record then, in the first place?

Can you build a minimal required version of ffmpeg on Travis CI with all the options and dependencies disabled that are not needed for peek in order to reduce the dependency tree and, hence, reduce AppImage size?

[phw]

Loading the schema from relative path is already in the code, but I think disabled in release build. I can take care of that.

Avconv is a fork of ffmpeg, you need either one or another. Ubuntu had for a while switched to avconv, but they have since changed back. There is a compatibility package libav-tools, but it just is a wrapper for ffmpeg calls.

I can look into building a minimal ffmpeg, but building ffmpeg is a bit of a mess and minimizing it often is a bit trial and error with all the dependencies, so this will quickly take some time.

[phw]

For starting the binaries I would suggest just to set PATH, we could have a small wrapper to start the Peek executable if necessary.

[probonopd]

OK, using avconv should be fine, no ffmpeg needed then? Do we need convert from ImageMagick?

[phw]

Better the other way round if possible, ffmpeg and no avconv. convert from Imagemagick is also needed, it is used to generate a optimized GIF afterwards.

[probonopd]

ImageMagick and ffmpeg are huge beasts, the best way to do this would be to compile them with everything disabled at build time that is not absolutely necessary. I would _greatly_ appreciate if you could do this. Let's see how far we can bring down those 55 MB :-)

For ImageMagick, see http://blog.schdbr.de/imagemagic-osx-static-relocatable-build/ under "manual". Ideally we can do with additional environment variables such as $MAGICK_HOME.

[phw]

Peek is already loading the settings schema in ./data/schemas/ relative to current working directory. Does this help?

[probonopd]

No, because this is relative to the directory _where the user is in_ (which could be anywhere) but we need relative to _the application binary_.

[phw]

Ok, got it. Will change :)

[phw]

I played around with ffmpeg and imagemagick while exploring flatpaks, see https://github.com/phw/peek/blob/master/flatpak.json . Both give quite small results and the built time is really ok.

[probonopd]

Very cool @phw - can you build these in https://github.com/phw/peek/blob/master/.travis.yml?

[phw]

I hope so :) Not yet quite sure how you want this to work, where should I place the binaries?

[probonopd]

Please have .travis.yml install them to PREFIX /usr, I will then take it from there.

[phw]

@probonopd Ok, we are probably getting somewhere :) I took your changes to travis.yml, put them into the build-appimnage branch and added the build scripts for ImageMagick and ffmpeg. I think I also understood things good enough to get them running.
I am a bit confused by linuxdeployqt (it looks so Qt specific), but it seems to work.

The build ran at https://travis-ci.org/phw/peek/builds/211031336 and produced an appimage. It is even running for me (I have the settings schema and dependencies installed, so this is not failing for me). But I get GTK theme parsing errors.

Next step will be to fix the relative path issues for executables, gsettings and ImageMagick config. I think I will add a launcher script that sets up a sane environment to run in. That's cleaner then messing around in the code to hack this into somehow for this specific environment, don't want many workarounds there.

[probonopd]

Hi @phw thanks for looking into this. With your AppImage, I get

(peek-x86-64.appimage:11449): GLib-GIO-ERROR **: Settings schema 'com.uploadedlobster.peek' is not installed

Trace/breakpoint trap

The same issue I discussed here.

[probonopd]

Ping @phw :-)

[phw]

@probonopd Sorry, never forgot about this, but had to find the time to tackle this again. The issue you had above was kinda expected, as I had said above the paths were not yet set correctly for everything inside the AppImage. You have to set the proper environment variables for this to work.

I have now setup the build on OpenSUSE's OBS and hopefully setup the environment properly: https://build.opensuse.org/package/show/home:phiwo:peek/peek

Result is looking good, but so far I have only tested it on my Arch system. Need to test it in other distros. Latest AppImages can be downloaded on https://download.opensuse.org/repositories/home:/phiwo:/peek/AppImage/

Could you give the appimage.yml file a quick review?

[phw]

Works well on Ubuntu 16.04 and Fedora 25. Only thing I had to do is to force the theme used to Adwaita, as the system theme often is not usable unless the Gtk version of the host matches the bundled one. I could not bundle Gtk and expect it on the host, but this brings other issues. I'm actually ok with forcing the theme in this situation.

[probonopd]

Works great on Ubuntu 16.04.1 LTS (Xenial Xerus).

[probonopd]

But:

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=14.04
DISTRIB_CODENAME=trusty
DISTRIB_DESCRIPTION="Ubuntu 14.04.1 LTS"
NAME="Ubuntu"
VERSION="14.04.1 LTS, Trusty Tahr"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 14.04.1 LTS"
VERSION_ID="14.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"

No protocol specified
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-dTEtMO0L2O,guid=102a9d938e2276459c7f9ea659dbd15d
DBUS_SESSION_BUS_PID=19872
/mnt/usr/lib/peek/peek-run: symbol lookup error: /mnt/usr/lib/peek/peek-run: undefined symbol: g_application_add_main_option

Tested on ubuntu-14.04.1-desktop-amd64.iso using testappimage.

[probonopd]

Let's debug:

me@host:~$ Downloads/peek-1507392410.d9fbe79-Build74.1.glibc2.14-x86_64.AppImage  --appimage-extract

me@host:~$ LD_DEBUG=libs ./squashfs-root/AppRun 2>&1 | grep " init: /usr/"
     20000: calling init: /usr/lib/x86_64-linux-gnu/libXdmcp.so.6
     20000: calling init: /usr/lib/x86_64-linux-gnu/libxcb.so.1
     20000: calling init: /usr/lib/x86_64-linux-gnu/libX11.so.6
     20000: calling init: /usr/lib/x86_64-linux-gnu/libdrm.so.2
     20000: calling init: /usr/lib/x86_64-linux-gnu/libdatrie.so.1
     20000: calling init: /usr/lib/x86_64-linux-gnu/libffi.so.6
     20000: calling init: /usr/lib/x86_64-linux-gnu/libgobject-2.0.so.0
     20000: calling init: /usr/lib/x86_64-linux-gnu/mesa/libGL.so.1
     20000: calling init: /usr/lib/x86_64-linux-gnu/libthai.so.0
     20000: calling init: /usr/lib/x86_64-linux-gnu/libfontconfig.so.1
     20000: calling init: /usr/lib/x86_64-linux-gnu/libpango-1.0.so.0
     20000: calling init: /usr/lib/x86_64-linux-gnu/libpangoft2-1.0.so.0
     20000: calling init: /usr/lib/x86_64-linux-gnu/libpangocairo-1.0.so.0
     20000: calling init: /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0
     20000: calling init: /usr/lib/x86_64-linux-gnu/libgdk_pixbuf-2.0.so.0
     20000: calling init: /usr/lib/x86_64-linux-gnu/gvfs/libgvfscommon.so
     20000: calling init: /usr/lib/x86_64-linux-gnu/gio/modules/libgvfsdbus.so
     20000: calling init: /usr/lib/x86_64-linux-gnu/gio/modules/libdconfsettings.so
     20000: calling init: /usr/lib/x86_64-linux-gnu/libstdc++.so.6
     20000: calling init: /usr/lib/x86_64-linux-gnu/libicudata.so.55
     20000: calling init: /usr/lib/x86_64-linux-gnu/libicuuc.so.55
     20000: calling init: /usr/lib/x86_64-linux-gnu/libxml2.so.2
     20000: calling init: /usr/lib/x86_64-linux-gnu/libcroco-0.6.so.3
     20000: calling init: /usr/lib/x86_64-linux-gnu/librsvg-2.so.2
     20000: calling init: /usr/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-svg.so
     20000: calling init: /usr/lib/x86_64-linux-gnu/gdk-pixbuf-2.0/2.10.0/loaders/libpixbufloader-png.so

Everything with "gdk" in the name may be suspect. So let's see:

me@host:~$ grep -r g_application_add_main_option squashfs-root/
Binary file squashfs-root/usr/lib/peek/peek-run matches
# So, this symbol is not resolved within the AppImage

# Let's check which libraries reference it on the (Xenial) host system
me@host:~$ grep -r g_application_add_main_option /usr/lib/
grep: /usr/lib/cups/backend/serial: Permission denied
Binary file /usr/lib/x86_64-linux-gnu/girepository-1.0/Gio-2.0.typelib matches
Binary file /usr/lib/x86_64-linux-gnu/libgio-2.0.so.0.4800.1 matches
Binary file /usr/lib/x86_64-linux-gnu/libgiomm-2.4.so.1.3.0 matches

So I assume you'd need to bundle the gio stuff, hoping it doesn't introduce other "too new" dependencies.

[phw]

Thanks for the analysis and all the support. I though I had it nearly running, but copying in libgio ended up copying much more libraries, including libglib and libpango, which failed with missing configuration and failed loading fonts. Also icons are missing all over the place.

In addition using the SUSE libs now required libselinux, which broke things for Arch. At this point I will just give up on Ubuntu Trusty, not worth the trouble. I know those users can run the Snap. I am more than happy that the AppImage is running on newer distributions.

Only sad point for me is that AppImageHub won't work this way. Any plans here for supporting AppImages that run only on newer than Trusty?

[probonopd]

Yes, once Trusty is no longer a supported (by the vendor) version.

But I respect your decision.

[phw]

Yes, once Trusty is no longer a supported (by the vendor) version.

That's still a while :) But Trusty makes in general sense as a target for AppImage.

Btw, now that I have managed to build a working AppImage I am not sure I prefer AppImage or Flatpak, both from a user's and developer's point of view. Both have some advantages.

But I found the start into AppImage hard. It's not that the documentation is missing something, it's more how it is structured. Both Flatpak and Snappy emphasize strongly on the tooling, so you quickly see how you can use snapcraft / flatpak-builder to get things built.

AppImage's documentation focuses more on the structure and the available options. You are told to get a proper AppDir, but than you have to search what an AppDir is, how it is structured and what tools exist to create it. Likewise it tells you to gather binaries somehow, whereas Snappy / Flatpak quickly show you how to package libraries and support the common build tools out of the box. The lack of concrete instructions makes the entry rather hard.

It also does not seem the tooling is lacking much in general. There are the YML recipes used by OBS, there is the OBS support, there are handy tools like linuxdeployqt (which despite its Qt focus is useful as a general purpose library gatherer / bundler, but that's not very clear, I would have never touched this tool if you hadn't done so in your first attempt), there is sandboxing. But if you don't know anything about AppImage you are a bit lost between all those options :)

[probonopd]

Thanks for your valuable feedback. AppImage is a volunteer-based community effort, so pull requests are highly welcome, as are wiki edits.