Hi,
do you have any plan about getting an "official" mimetype for .appimage files? Currently they are detected as executable files by the shared-mime-info database, which is fine. However there are use cases where a dedicated mimetype would be handy. For example, users of archiving tools like KDE's Ark or Gnome's file-roller may be interested in browsing the content of an appimage: https://bugs.kde.org/show_bug.cgi?id=363209
This new mimetype could inherit the application/x-executable mimetype, to express that appdata files are still executable files but they are also something more - e.g. they can be browsed with archiving tools.
Good point. Actually I have tried but never was successful; do you know why the following is not working?
<?xml version="1.0"?>
<mime-info xmlns='http://www.freedesktop.org/standards/shared-mime-info'>
<mime-type type="application/x-appimage; version=1.0">
<sub-class-of type="application/x-executable"/>
<sub-class-of type="application/x-iso9660-image"/>
<comment>AppImage</comment>
<magic priority="50">
<match value="ELF" type="string" offset="1">
<match value="CD001" type="string" offset="32769">
<match value="AppImage" type="string" offset="32809"/>
</magic>
<magic priority="40">
<match value="ELF" type="string" offset="1">
<match value="CD001" type="string" offset="32769">
<match value="APPIMAGE" type="string" offset="32809"/>
</magic>
<glob pattern="*.appimage"/>
<glob pattern="*.AppImage"/>
</mime-type>
</mime-info>
sudo xdg-mime install --mode system appimagekit-appimage.xml
xdg-mime query filetype '/home/me/Downloads/test'
# always returns:
# application/x-executable
@probonopd
Some of the <match> tags are not properly ended. You can check whether the installed XML is ok by running:
sudo update-mime-database /usr/share/mime
I was able to get the expected mimetype after fixing the above issues!
Strange. Changed it to
<?xml version="1.0" encoding="UTF-8"?>
<mime-info xmlns="http://www.freedesktop.org/standards/shared-mime-info">
<mime-type type="application/x-appimage; version=1.0">
<sub-class-of type="application/x-executable" />
<sub-class-of type="application/x-iso9660-image" />
<comment>AppImage</comment>
<magic priority="50">
<match value="ELF" type="string" offset="1" />
<match value="CD001" type="string" offset="32769" />
<match value="AppImage" type="string" offset="32809" />
</magic>
<magic priority="40">
<match value="ELF" type="string" offset="1" />
<match value="CD001" type="string" offset="32769" />
<match value="APPIMAGE" type="string" offset="32809" />
</magic>
<glob pattern="*.appimage" />
<glob pattern="*.AppImage" />
</mime-type>
</mime-info>
I get no more syntax complaints but it doesn't seem to work either. Ubuntu 16.04.
No idea, works for me (even your updated version). I'm on archlinux with shared-mime-info 1.6
Anyway, when you submit a new mime to upstream shared-mime-info, they require you to provide a test file as well, and that is what matters. If the test passes, the mime is gonna be accepted :)
Anyone, we need to iron this out and then submit together with a link to an example AppImage such as https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.17-x86_64.AppImage according to https://cgit.freedesktop.org/xdg/shared-mime-info/tree/HACKING. I'm a bit lost here.
I can do it if you want, I recently submitted a mimetype so I have fresh memory of the burocracy :p
That would be great. Before you do please check that it works with and without the suffixes.
...and that the AppImage can still be executed by double-clicking when chmod a+x is set. Worst case would be that it is no longer recognized as an ELF executable in GNOME Nautilus and the likes.
Would making it a subclass of ISO automatically allow tools like Ark or file-roller to process it?
Would making it a subclass of ISO automatically allow tools like Ark or file-roller to process it?
Not necessarily. I can only speak for Ark, which unfortunately hardcodes the list of supported mimetypes
For whatever reason I had to log out and log in again and now it is working. AppImages still run when they have the executable bit set; however when the executable bit is _not_ set they are tried to be opened with file-roller (which says it cannot handle the file). Is there a way around that?
Yeah, same here with Dolphin+Ark. I guess they are indeed detected as ISO by the file-managers.
What should be the expected behavior instead? Just nothing happening?
Another thing, the 2nd and 3rd magic offset are too big and I think the shared-mime-info maintainers will complain about this. Isn't enough to just match the ELF string?
Yeah, same here with Dolphin+Ark. I guess they are indeed detected as ISO by the file-managers. What should be the expected behavior instead? Just nothing happening?
When we don't subclass it to application/x-iso9660-image, then it doesn't try to open the AppImage with file-roller. GNOME then asks "There is no application installed for “AppImage” files. Do you want to search for an application to open this file?".
I am actually not sure which behavior is better. Just found out that when we subclass it to application/x-iso9660-image, then in GNOME I can right-click to open with gnome-disk-image-mounter which conveniently mounts the AppImage. Not bad... I guess we need to test the behavior under multiple *DEs.
Another thing, the 2nd and 3rd magic offset are too big and I think the shared-mime-info maintainers will complain about this. Isn't enough to just match the ELF string?
No, because then we would match every ELF. I guess every ISO is also matched this way?
Perhaps it would be sufficient to treat every ISO that is also an ELF as an AppImage, but I think the 3rd check doesn't really increase the offset by much and it doesn't hurt.
Ah, now I remember. ISO files don't have any magic (because of too long offset required):
<mime-type type="application/x-cd-image">
<_comment>raw CD image</_comment>
<sub-class-of type="application/x-raw-disk-image"/>
<alias type="application/x-iso9660-image"/>
<!-- No magic, see https://bugs.freedesktop.org/show_bug.cgi?id=10049 -->
<glob pattern="*.iso" weight="80"/>
<glob pattern="*.iso9660"/>
</mime-type>
There is also this other bug which breaks ISO files detection: https://bugs.freedesktop.org/show_bug.cgi?id=80877
I can try to submit the mime as is, and then let's see what the maintainers think...
In the worst case, we might have to go without magic?
Do you know what is the largest acceptable offset?
Nope, but I guess something around 2K bytes (the largest offset in the current database seems to be 2112).
Draft of the patch here: https://gist.github.com/aelog/0db7e74520fe5dbfcb7d7e765e2ef986
Quick experimentation suggests that ELF binaries have "GNU" at offset 608 but they continue to work if I change that to "AppImage". However I am entirely uncertain at this time whether this violates some standard... I guess I need to read up on the ELF format to find a way to sneak in a magic string or number into the ELF header reliably.
Allright, no hurry from my side. Just ping me here when you think the mime is ready for submission :)
Opened https://bugzilla.gnome.org/show_bug.cgi?id=766644 on file-roller: Process ISO files regardless of the filename extension
@aelog I was thinking about doing this at some point but you beat me to it!
I just have a couple of small suggestions:
<glob pattern="*.AppImage" /> will be fine on its own.<magic priority="50">
<match value="ELF" type="string" offset="1" >
<match value="CD001" type="string" offset="32769">
<match value="AppImage" type="string" offset="32809" />
<match value="APPIMAGE" type="string" offset="32809" />
</match>
</match>
</magic>
The nested <match> tags should result in this logic:
Ah, I see that the problem is with the ISO offsets being too large. It should be enough to just do the glob for ".AppImage" in the filename and match for "ELF" in the binary, providing the "ELF" magic is set to an appropriate priority. Looking at the shared-mime-info database I can see that "ELF" is already matched with priority 40 and 50, so I think the appropriate value is either 30 or 60. Somewhat counter-intuitively, I think it needs to be a higher priority to prevent matching against random ELF files, but you'd need to test it for yourself.
@probonopd it won't help detection of existing AppImages, but here are some options for embedding data in an ELF file to create your own magic bits.
@shoogle I don't mind if you want to submit the mime. I was just waiting to hear from @probonopd about how he plans to solve this magic thing.
Unfortunately it is not solved yet. We need to find a way to sneak in a magic further at the top of the ELF. Is there a way to get a certain magic into the ELF header and still be fully compliant?
Maybe on the .data or .text sections of the ELF. Now I'm traveling, later
I'll try to figure. out how.
El mié., 15 jun. 2016 10:54, probonopd [email protected] escribió:
Unfortunately it is not solved yet. We need to find a way to sneak in a
magic further at the top of the ELF. Is there a way to get a certain magic
into the ELF header and still be fully compliant?—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/probonopd/AppImageKit/issues/144#issuecomment-226127783,
or mute the thread
https://github.com/notifications/unsubscribe/AK3C25N8WIawjwh76UJNQqmZQPytijYuks5qL73fgaJpZM4IhYFr
.
Un saludo.
Helio.
Apparently bytes 8-15 of the header are unused, which gives just enough room to write "AppImage"!
printf 'AppImage' | dd bs=1 seek=8 count=8 conv=notrunc of=Program.AppImage
Which makes for some pretty neat Elf magic! :smile:
<magic priority="50">
<match value="ELF" type="string" offset="1" >
<match value="AppImage" type="string" offset="8" />
</match>
</magic>
Very well spotted @shoogle, bytes 8-15 are "Unused/padding". Looks like the way to go.
Let's not waste this precious space with the string "AppImage" though, imho we should:
Hence I propose:
0x414901
0x41= ASCII letter "A" (as in "App")0x49 = ASCII letter "I" (as in "Image")0x01= AppImage subtype 01 = current AppImage format (ISO9660 with optional zisofs compression with a file "AppRun" to be executed)And with that, @shoogle, I think you are spot on that we need to start working on getting a formal spec together...
What do people think?
@probonopd you're right, we shouldn't waste bytes. I think your suggestion of AI<subtype> is a good one. I think this was your intention, but just to make it explicit, I don't think the "subtype" should correspond to the "version" of the AppImage Specification. I think we should store the version a different way, e.g. by setting an environment variable when the AppImage is loaded, or using a Volume Identifier field. The subtype should only be changed when the format changes drastically and it becomes impossible to extract the AppImage using the previous method. In other works:
@shoogle fully agree - I was thinking about AI
Perhaps we could reserve the value 0x00 for the Subtype Byte for "non fully standards-compliant AppImages"? So if I release a portable binary that looks and behaves like an AppImage on a user level, but not on a programming level, then I would use the 0x00 byte value.
Added to the spec @shoogle.
So I think this ticket is no longer blocked (at least as soon as someone has tested 0x414901 in practice.)
Sorry, I'm busy yet, however @shoogle found a very good method. Maybe would
be better to use 0xA1 as signature? Is AI in 1337 and leaves unused a byte
more. Even you could add another byte at the end for checksum of all the
data. Then you would avoid a mismatch when a raw binary blob from something
has the same magic at the same offset (even a /dev/urandom dump).
The checksum could be something like:
uint8_t bytes[10] ={0xA1, 0x13, 0x37, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
0x00}
uint8_t sum = 0x00;
for(int i=0; i<10; i++) {
sum |= bytes[i];
}
El mié., 15 jun. 2016 20:33, Peter Jonas [email protected]
escribió:
@probonopd https://github.com/probonopd yes, I'm on it now.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/probonopd/AppImageKit/issues/144#issuecomment-226278781,
or mute the thread
https://github.com/notifications/unsubscribe/AK3C21ghcbW1bc-AgYBp9u3UsJ9zGPpmks5qMEVvgaJpZM4IhYFr
.
Un saludo.
Helio.
What if someone creates an AppImage variant that uses an interpreted (plain text) runtime script instead an ELF? My suggestion, submitted here, is as follows:
The AppImage must either have the magic bytes:
AIx at offset 8, where A and I are ASCII characters and x is the Type Byte, ORAppImage### in the first 128 bytes of the file, where ### is the 3 digit Decimal TypeWhere Decimal Type (000-255) is the decimal equivalent of the Type Byte (0x00-0xFF)
This would allow a runtime script along the lines of:
#!/bin/bash
# AppImage000
export APPIMAGE="$0"
export APPDIR="/tmp/$RANDOM"
self-extract "$APPIMAGE" "$APPDIR"
cd "$APPDIR"
./.AppRun "$@"
@crushedice2000 your idea for a checksum was interesting too. I'm not sure if it is really necessary though. I don't think the shared-mime-info database can handle checksums.
@shoogle
What if someone creates an AppImage variant that uses an interpreted (plain text) runtime script instead an ELF?
Then it is guaranteed to crash your text editor of choice when you double click the file without having the MIME type installed. That is the reason why I moved away from this approach earlier.
But to your question, then it should be worth another Subtype Byte if we come to the conclusion that this should be a valid AppImage.
I see, that would be an issue. But the reason I suggested including the whole "AppImage000" string was because the #/bin/bash shebang has to be on the first line, so you can't put the magic bytes in at offset 8, 9 and 10.
@probonopd, BTW please join me on Gitter or IRC if you have a few minutes spare.
Fyi, @kbroulik is looking into writing an AppImage thumbnailer for Plasma, will most likely need a MIME Type to work properly. So we should better speed this discussion up ;-)
@shoogle right now the spec doesn't allow for bash instead of ELF headers. Should we want to introduce this at some point in the future after careful consideration, I will be all for adding a "second shebang" like you suggest. But right now I would be happy if our current Subtype 1 ELF+ISO AppImages would be recognized :-)
The following seems to work for me:
cat > /tmp/appimagekit-appimage.xml <<\EOF
<?xml version="1.0" encoding="UTF-8"?>
<mime-info
xmlns="http://www.freedesktop.org/standards/shared-mime-info">
<mime-type type="application/x-iso9660-appimage">
<_comment>AppImage</_comment>
<sub-class-of type="application/x-executable" />
<sub-class-of type="application/x-iso9660-image" />
<generic-icon name="application-x-executable"/>
<magic priority="50">
<match value="ELF" type="string" offset="1" >
<match value="0x41" type="byte" offset="8">
<match value="0x49" type="byte" offset="9">
<match value="0x01" type="byte" offset="10" /></match>
</match>
</match>
</magic>
<glob pattern="*.appimage" />
<glob pattern="*.AppImage" />
</mime-type>
</mime-info>
EOF
sudo xdg-mime install --mode system /tmp/appimagekit-appimage.xml
rm /tmp/appimagekit-appimage.xml
sudo update-mime-database.real /usr/share/mime
sudo update-desktop-database
# To test:
# killall nautilus # GNOME
# killall dolphin # KDE
What do you think - subclass it to application/x-iso9660-image or not?
I'm in favour of subclassing. The share mime database won't be updated in any distributions for a while so there is plenty of time to start including README files with AppImages. AppImageKit can insert a generic README automatically during packaging.
In that case, we should also make sure that applications like file-roller and Ark can handle AppImages because otherwise users will never get to see the README...
In that case, we should also make sure that applications like file-roller and Ark can handle AppImages because otherwise users will never get to see the README...
Ark 16.04.2 (recently released) should be able to open appimage files, as long as the appimage mimetype inherits from the ISO mimetype!
@aelog thanks, that I call progress :-)
@aelog want to submit this? I have put 0x414901 at offset 8 into the spec draft.
@probonopd Sure. I need a test .appimage file conforming to that specification. You know where I can find one?
I will generate a tiny one real quick @aelog
@aelog here is the example file:
https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.17-x86_64.AppImage
@probonopd Thanks for the file. What about the icon? I could add <generic-icon name="application-x-executable"/> to the mimetype (I'm not sure about the ISO mimetype icon)
EDIT
Draft of the mimetype:
<mime-type type="application/x-iso9660-appimage">
<_comment>AppImage</_comment>
<sub-class-of type="application/x-executable"/>
<sub-class-of type="application/x-iso9660-image"/>
<generic-icon name="application-x-executable"/>
<magic priority="50">
<match value="ELF" type="string" offset="1" >
<match value="0x41" type="byte" offset="8">
<match value="0x49" type="byte" offset="9">
<match value="0x01" type="byte" offset="10"/>
</match>
</match>
</match>
</magic>
<glob pattern="*.appimage"/>
<glob pattern="*.AppImage"/>
</mime-type>
If we don't set any generic-icon, this is what the specification says:
generic-icon elements specify the icon to use as a generic icon for this particular mime-type, given by the name attribute. This is used if there is no specific icon (see icon for how these are found). These are used for categories of similar types (like spreadsheets or archives) that can use a common icon. The Icon Naming Specification lists a set of such icon names. If this element is not specified then the mimetype is used to generate the generic icon by using the top-level media type (e.g. "video" in "video/ogg") and appending "-x-generic" (i.e. "video-x-generic" in the previous example). Only one generic-icon element is allowed.
In our case, this would be application-x-generic and I'm not sure what icon that would be...
The "Icon Naming Specification lists" should be this one.
<generic-icon name="application-x-executable"/> makes sense to me, as long as it is still possible to install a specific icon later for application/x-iso9660-appimage.
All right, patch submitted: https://bugs.freedesktop.org/show_bug.cgi?id=96685
Great, thank you @aelog
Patch merged in master: https://cgit.freedesktop.org/xdg/shared-mime-info/commit/?id=01fa61fc002afdcf43f61e7df2d6cc6f6968d8d2
Thanks to everyone who helped working on this!
Congratulations @aelog and thanks for your good preparation which made it possible for this to be merged this quickly.
@aelog
Ark 16.04.2 (recently released) should be able to open appimage files, as long as the appimage mimetype inherits from the ISO mimetype!
Does not seem to work for me. Can Ark #363209 be updated?
Does right-click -> extract work for you? Could you verify?

@probonopd Unfortunately, doesn't work as I would have expected. See my comment in the bug report.
@probonopd Can you generate an example AppImage smaller than the LeafPad one (preferably <100kb)? We want to include it in the Ark git repo for unit-testing.
hello-2.8-x86_64.AppImage.zip (sorry for the AppImage being inside a ZIP file; please extract it; GitHub doesn't allow AppImage files to be uploaded to GitHub issues directly)
I think this is close to the smallest possible:
./apt-appdir/apt-appdir hello
cat > hello.AppDir/hello.desktop <<EOF
[Desktop Entry]
Type=Application
Name=hello
Comment=Hello, world!
Icon=hello
Console=True
Exec=hello
Cateories=Utilities
EOF
cp /usr/share/icons/HighContrast/256x256/apps/utilities-terminal.png hello.AppDir/hello.png
rm -rf hello.AppDir/usr/share/
./AppImageAssistant.AppDir/package hello.AppDir/ hello-2.8-x86_64.AppImage
@probonopd Thanks for the example AppImage file. I now committed support for AppImage to Ark repo and it will be available in version 16.08.0 which will be released on August 18th.
Thanks @rthomsen
Now without GNU hello:
mkdir -p hello.AppDir/usr/bin
cat > hello.AppDir/hello.desktop <<EOF
[Desktop Entry]
Type=Application
Name=hello
Comment=Hello world
Icon=hello
Console=True
Exec=hello
Cateories=Utilities
EOF
cat > hello.AppDir/AppRun <<\EOF
#!/bin/bash
HERE="$(dirname "$(readlink -f "${0}")")"
exec "${HERE}/usr/bin/hello" $@
EOF
chmod a+x hello.AppDir/AppRun
cat > hello.AppDir/usr/bin/hello <<EOF
#!/bin/bash
echo "Hello world"
EOF
chmod a+x hello.AppDir/usr/bin/hello
# A black circle cannot have any copyright
wget -c http://www.iconsdb.com/icons/preview/black/circle-xxl.png -O hello.AppDir/hello.png
./AppImageAssistant hello.AppDir/ hello-1.0-x86_64.AppImage
hello-1.0-x86_64.AppImage.zip (sorry for the AppImage being inside a ZIP file; please extract it; GitHub doesn't allow AppImage files to be uploaded to GitHub issues directly)
We should
Currently, the handing in KDE seems to be not optimal. When downloading an AppImage in Firefox while the optional appimaged daemon is running (which does chmod +x automatically) and opening it from within the downloads menu in Firefox, we get this

and then __the AppImage is opened in Ark rather than being executed__ (as you would expect from reading the message).
@probonopd If I double click an executable appimage file, the app is executed as expected here. Are you trying in Neon from a live ISO? Maybe Neon is missing some package?
OK, good to hear @elvisangelaccio. I will continue to investigate. I also have the optional appimaged daemon running on this system.
Ah wait, I don't think I have appimaged installed, could this make a difference?
Possibly. It sets the +x bit.
On a related note, do you think you could help getting the magic for the type 2 (ELF+squashfs) AppImage format magic in?
Oh yes, I can give it a try when I have time, shouldn't be harder than the other one :)
@probonopd I need a type 2 test file (e.g. leafpad, like we did for type 1)
Also, do you like application/vnd.appimage for the mime name? This is for consistency with snap (application/vnd.snap) which also inherits from the squashfs mimetype (application/vnd.squashfs).
Sure, here you go @elvisangelaccio
https://bintray.com/probono/AppImages/download_file?file_path=Leafpad-0.8.18.1.glibc2.4-x86_64.AppImage
@probonopd Since both mimetypes have the same glob pattern (*.appimage or *.AppImage), one of them necessarily will fail the "lookup by filename", and applications will need to check the magic value to resolve the ambiguity. Which one of the two appimage formats would you rather be the "default"? (i.e. the output mimetype when looking up an appimage _only by filename_).
@elvisangelaccio good catch. Magic bytes should always override the filename extension, and the filename extension should point to type 1 (ISO9660) AppImage, since we expect all type 2 (squashfs) AppImages to have the correct magic bytes set.
Still, as the magic bytes should always override the filename extension, a type 2 (squashfs) AppImage with the AppImage filename extension would correctly be identified as type 2.
@elvisangelaccio any news on this? Should we also submit it for the file magic number database (and where is the upstream project for that)?
We should also submit our magic numbers here: https://en.wikipedia.org/wiki/List_of_file_signatures
@probonopd Not yet, I just pinged the freedesktop guy on the bugzilla ticket...
@probonopd Oh, that was quick. Please let me know what do you think about https://bugs.freedesktop.org/show_bug.cgi?id=100608#c4
Looks like you're suggesting vnd.appimage as the MIME type for v2. That doesn't sound like a great idea, since there are going to be future versions of the AppImage specification You should rather introduce some versioned type, e.g. appimage.type2.
The benefit of being able to distinguish between different types of AppImages for end user applications is so they can perform integrations like displaying the icon etc. which is a significantly different operation for different types of AppImages.
Could we not remove the type specific sub-class-of, add application/x-iso9660-appimage as an alias of the more generic application/vnd.appimage, and add the magic to the first type as well?
Type 1 uses ISO9660 whereas type 2 is an entirely different format. They are described in the AppImageSpec.
What about future versions of the spec (as pointed out by @TheAssassin)? Are we going to need a new mimetype every time?
Yes, but we are not foreseeing changes to the spec that would require a type 3 for a very long time, since the type 2 spec is quite generic.
Thank you @elvisangelaccio.
Thanks @elvisangelaccio. I think I can close it safely now.
What does "vnd" mean?
Can we explain what is going on here?
@probonopd vnd seems to be used by the XDG folks for all "containerized" applications.
ZOMG. AppImages are not "containerized", are they?
Did you even bother looking this up on the Internet using a search engine?
https://www.startpage.com/do/dsearch?query=mime+vnd&cat=web&pl=opensearch
https://stackoverflow.com/questions/5351093/what-is-the-meaning-of-vnd-in-mime-types
vnd -> vendor specific
vnd indicates vendor-specific MIME types, which means they are MIME types that were introduced by corporate bodies rather than e.g. an Internet consortium.
Heh, so we are a "corporate body" now. Well, we are not an "Internet consortium" I guess.