Sparkle: Add `xip` support for signed, compressed, application updates

[grahammiln]

Apple offer a signed archive format called xip to be used in-place of zip files:

Usage: xip [options] --sign <identity> <input-file> [ <input-file> ... ] <output-xip-file>

Given many application updates are delivered as plain zip files, Sparkle would benefit from supporting xip. This would provide an additional layer of security.

Sparkle would need to:

• Detect the xip suffix and decompress the contents; see Unarchiver's implementation
• Check for and display certificate errors returned from failing xip extractions

[kornelski]

Comments in the Unarchiver aren't encouraging:

Check for XIP files, the worst file format ever. This is a cpio file inside a gz file inside a xar, plus a metadata file.

What is the benefit of using it over, e.g. tar.xz with a DSA signature?

[grahammiln]

I raise the xip format here as supporting it would add another layer of protection. Notably Apple is using it for the latest Xcode download.

Another related file format change is the quiet introduction of code signed disk images. codesign on 10.11.5 supports signing disk images and macOS 10.12 will treat unsigned disk images differently from signed. Something we might want to be aware of.

[kornelski]

I don't see what attack scenario this would protect from.

For example if an app used xip + DSA, and we had a vulnerability that allows DSA bypass, then the attacker wouldn't be stopped by xip verification, because they could use zip instead.

[zorgiepoo]

Yeah I never saw xip until I downloaded the latest Xcode beta.

Some thoughts though if we decide it's worthwhile:

1. For compatibility reasons, what OS was xip first introduced?
2. How can we extract xip files from the command line? We pipe into utilities for other formats, and this may be useful for automation purposes too. Maybe some way through ditto?
3. How can we inspect or verify the code signature of these archives? I didn't have any luck with codesign --verify - we could want to match the signature with the current installed app for instance, or maybe I'm misunderstanding?

[grahammiln]

The xip tool is present in OS X 10.7 and later. I have not checked OS X 10.6.

I am unaware of any command line tools for extracting and checking xip files. Only the graphical Archive Utility is mentioned in Apple's manual page.

Supporting xip will add an additional layer of defence to Sparkle. Certificates can be revoked, the DSA signature can not.

If either the xip or DSA signature failed, I would expect Sparkle not to apply the update.

I would be content with a valid archive signature without an expectation of matching the host's own certificate. A delegate method could provide the host app with more control over the comparison if desired.

In time, and maybe as a policy choice for the host application, I would consider disabling support for non-signed archive formats.

[kornelski]

Layers of defence are only additive if they're mandatory. Otherwise weakest link in the chain applies.

As I've mentioned before, if DSA private key leaks, revoking xip's certificate will not protect users, because the attacker will be able to use zip+DSA instead.

We would have another layer of security here only if we dropped support for all other formats.

[grahammiln]

We would have another layer of security here only if we dropped support for all other formats.

For my applications using Sparkle, I would likely disable all but the signed formats. Assuming Sparkle supports signed formats.

[depth42]

As I understand it currently, macOS Sierra will translocate all apps coming from unsigned archives which will make it impossible for Sparkle to replace them with updated versions: http://lapcatsoftware.com/articles/app-translocation.html

So for Sparkle to work under Sierra, one needs to use signed DMGs or xip archives.
Xcode 8 beta gets delivered as a xip archive, too.

[kornelski]

I don't think translocation affects Sparkle that much. We check whether the app is writeable, and we remove quarantine bits from unarchived files.

[depth42]

But if an app is translocated, won't sparkle simply replace the translocated app at the temporary location?

[kornelski]

If the app is translocated, then AFAIK it'll be on read-only volume, so Sparkle won't be able to update it at all.

It's not ideal — we might want to show a specific error message when user selects update manually. But I think it's acceptable. When Apple wants to quarantine the app and don't think user has "installed" it, then updating isn't that important. We don't prompt for updates until a second run anyway, so users just trying the app won't be affected.

The important thing is that once the app is moved or otherwise unquarantined it'll update fine.

[zorgiepoo]

Right, App Translocation should be no different than an application running from a DMG trying to update itself, which is impossible. Xip archives (signed or not) have no impact on translocation from my testing.

Sounds like to me if you are distributing an app on your website for a first-time installation as a zip or xip, you may want to consider using a dmg instead because that could encourage the user to copy the app elsewhere. You'd also want to make sure your app is a top level item and not inside another folder (easier to do with dmg).

As for supporting xip, I'm a little reluctant if there's no system way (or way not through 3rd party code) of unarchiving the format. I think a "convenience" argument may be more relevant than "security" regarding that format. DMG is a bit more interesting because codesign actually seems to work with it, but it's also not a great format to deal with and I see very little benefit to signing it for website distribution.

[zorgiepoo]

According to this SO post, a xip can be unarchived by first checking the signature:

pkgutil --check-signature <xip-file>

and unarchiving the xip:

xar -xf <xip-file>

and then extracting the content inside:

tar zxvf Content

As for dmg's, we updated Sparkle's website documentation to recommend developers to ship dmg's, and preferably sign them, on their product's website (just for first time installs). This is due to dmg working best with app translocation and how signing them can avoid translocation.

[grahammiln]

Great to learn it is possible to validate and extract the contents of an xip using existing tools.

As an aside, Technical Note 2206: Code Signing in Depth now mentions the XIP format as a suitable way to ship signed products:

You probably don't need to use a disk image until the final package stage so another less heavy-handed method would be to use ZIP or XIP files.

[markmentovai]

Be aware that the reference to xip has been removed from the prerelease version of TN2206 that goes with 10.12.

[vslavik]

Be aware that the reference to xip has been removed from the prerelease version of TN2206 that goes with 10.12.

And the current text makes it clear XIP is not meant to be used:

Important: Starting with macOS Sierra, only XIP archives signed by Apple will be expanded. Developers who have been using XIP archives will need to move to using signed installer packages or disk images.

[aidin0181]

hey guys, i have an xcode download from outside (not apple website),now my mac high sierra says that my xip hash code is no correct,how can i extract this file?i gave the permission to the mac that could install all file from any source in security panel.but noting changed.

[aidin0181]

@vslavik so i can noway extract it right?what about winrar and repairing?

[kornelski]

@aidin0181 This file may contain malware, or is at least it's damaged and won't work. Don't install it. Get a fresh, valid file from Apple.

[aidin0181]

@kornelski well i downloaded a new version compatible with high sierra and i moved from vbox to vmware for toolbox issues(clipboard,sharing,aspect ratio).and i couldn't extract it from windows any way,not with winrar nor with Zip7.so i downloaded a new version from my country (iran) servers.if this time the mac show me error i think i have to download it from apple.by the way i'm gonna just export an ios app :((((((( what the hell...... right?

[kornelski]

Sorry for your problems. xip is Apple's own non-standard format, and the package includes not just regular files, but also symlinks, so most tools would probably damage it anyway. You have to decompress xip using macOS.

[michelf]

@aidin0181 If you already have a copy of Xcode at your disposal in a VM, you can create a zip archive yourself by right-clicking the app icon in the Finder in the first VM. Then transfer it to another VM, open and unzip there. That is much simpler than trying to fix a corrupted XIP file.

[aidin0181]

@michelf no i don't have any one.by the way if i had one i didn't really make myself nervous for downloading :)

[aidin0181]

well finally it worked.tnx for all of you guys.