Appcenter: Support Sparkle feeds for macOS/Windows in-app updates

[tonyarnold]

When using HockeyApp, we used to be able to distribute application updates for macOS (using Sparkle).

AppCenter doesn't seem to have any replacement for that, which means I can't use it as a full replacement of Hockey yet.

Please consider adding support for this as soon as it's feasible.

[shpakovski]

One more vote for this. Sparkle support is crucial for most Mac apps. So this feature is very important for the App Center. Thanks.

[lumaxis]

@tonyarnold @shpakovski The good news is: Sparkle support is already on our public roadmap! We have also mentioned it a couple of times as part of our HockeyApp transition plans, for example.
Unfortunately, we don't have a fixed plan yet of when exactly we will work on it, so keep an eye on our iteration plans.

@tonyarnold In your comment, you had asked about beta vs. production distribution.
My knowledge of Sparkle is fairly limited but there is not really a difference in the distribution mechanism for this, right? It's mainly just which build you ship through the feed.
I don't think we're intending to intentionally limit this to beta distribution but as hinted to earlier, we're in a fairly early planning stage for this, so we'd be happy about any input! 🙂

[tonyarnold]

there is not really a difference in the distribution mechanism for this, right?

That's correct, so really it comes down to you folks deciding what's acceptable use of your service. I'd love to be able to host production builds of our app on AppCenter (it just simplifies things) but there shouldn't really be front-end technical work required to support that if the feeds themselves are publicly accessible.

[shpakovski]

I don’t think we’re intending to intentionally limit this to beta distribution.

This one is a very important statement. Please do your best to support downloading and Sparkle listing for _public_ builds. This would make your service a full-stack solution for .dmg and .zip distribution. Thanks in advance!

I’d love to be able to host production builds of our app on AppCenter.

Exactly, this would be fantastic.

[vsanthanam]

Any updates on this at all? No status changes in the roadmap -- I really cannot migrate off of HockeyApp without this... I would imagine this is not uncommon for a lot of macOS devs.

[ghost]

If/when you support distribution via AppCenter, will the SUFeedURL that is already out there in our existing macOS releases be mirrored or redirected (https://rink.hockeyapp.net/api/2/apps/APP_ID) for any duration of time? If that link goes dead at some point, or if it won't serve any new versions/releases/updates if we transition to AppCenter, then our users will be stuck on whatever last version they are using.

The later this distribution feature ships in AppCenter, the greater the issue becomes as we won't have time to push out app updates that could redirect our SUFeedURL. I'd love to avoid having to build my own custom replacement beta / production Sparkle distribution workflow if something in AppCenter is already in progress and shipping soonish, but I can't risk leaving my user base behind if it isn't.

[WCByrne]

@rafifyalda In anticipation of this a while back, to be safe I went ahead and setup a url on my own service that redirects the existing Hockey url. But +1 for MS to keep those Hockey urls alive and redirect them to the new AC feeds.

Also, @lumaxis / AC team, for those that are/were using AC to build and distribute, the recent change that caused AC releases to not sync back to HA is a real drag without feeds in AC. I now have to download the build from AC, re-upload to Hockey and release there. I was really enjoying the ease of releasing directly in AC and automatically getting the updated feed from Hockey. Please consider this regression if there is a chance to get these feeds higher on the priority list 🙏.

[ghost]

@WCByrne: Thanks for the tip/idea, just went ahead and did the same 👍

[WCByrne]

And it get's worse if using AC builds😞. Now that app center releases don't sync back to Hockey App, not only do you have to upload the app to HA to get the AppCast feed, the symbols from the AC build aren't made available to the build uploaded to HA so crashes that come in later are unsymbolicated 😭.

[lumaxis]

Hey all! This is of course still firmly in our plans, we just haven't gotten to it yet 😓It will definitely be available before the shutdown though 🙏🏼

@rafifyalda We do have plans to have some APIs from HockeyApp continue working after the shutdown, @derpixeldan can likely comment with more details.

[lumaxis]

@rafifyalda @WCByrne As a temporary workaround, you can use a post-build script in App Center to automatically upload both the build and symbols to HockeyApp.

I had built one for my own purposes a while ago, you can likely use a slightly adjusted version for your app:

#!/usr/bin/env bash

upload_to_hockeyapp () {
  zip_symbols || return 1

  curl -v \
  -F "status=2" \
  -F "ipa=@$APPCENTER_OUTPUT_DIRECTORY/MyApp.ipa" \
  -F "[email protected]" \
  -H "X-HockeyAppToken: $2" \
  https://rink.hockeyapp.net/api/2/apps/$1/app_versions/upload
}

zip_symbols () {
  ZIP_FILE_NAME="symbols.dsym.zip"
  SYMBOLS_DIRECTORY="$APPCENTER_OUTPUT_DIRECTORY/../symbols"

  if [ -d $SYMBOLS_DIRECTORY ]; then
    echo "Zipping symbols..."
    zip -rq $ZIP_FILE_NAME $SYMBOLS_DIRECTORY
  else
    echo "Symbols not found."
    return 1
  fi
}

if [ $AGENT_JOBSTATUS != "Succeeded" ]; then
  echo "Build not successful, cancelling upload."
  exit 1
fi

if [[ -n "$HOCKEYAPP_API_TOKEN" && -n "$HOCKEYAPP_APP_ID" ]]; then
  echo "Uploading to HockeyApp"
  upload_to_hockeyapp $HOCKEYAPP_APP_ID $HOCKEYAPP_API_TOKEN
fi

You would need to add HOCKEYAPP_APP_ID and HOCKEYAPP_API_TOKEN as custom environment variables and adjust the curl call to your liking: https://support.hockeyapp.net/kb/api/api-versions#upload-version

[dipree]

@invariant sorry to hear that, we're currently working on it and it should be done soon.

[rishimodha]

@derpixeldan Hi, thanks for the update. Are you considering extending the transition date as it now seems like Sparkle support will be added very close to the planned end date for Hockey. We have a lot of Mac apps that use the HockeySDK and we need to thoroughly test everything once the AppCenter SDK is switched in. I think it's wise to extend the deadline by a few months. Or at least maintain some kind of legacy support until everyone is confident the Sparkle support is working correctly on App Center.

[dipree]

App Center now supports Sparkle feeds. Please read more in the documentation.

[WCByrne]

Fantastic!

Question on this, what versions make it into the feed? From what I'm seeing it looks like anything released in a public distribution group shows in the feed. What if you have more than one public group?

[dipree]

@WCByrne the five latest public releases, regardless of the group. I’ll clarify that in the docs.

[shpakovski]

@derpixeldan It works, thanks a lot! If releases are provided by public distribution groups, could you please insert sparkle:releaseNotesLink with a direct URL, so that customers could open a webpage in their browser? Should I create another Issue with this request? Thanks in advance!

[dipree]

@shpakovski yes, please.

[rishimodha]

@derpixeldan please confirm the hockeyapp sparkle feed urls will continue to work

[WCByrne]

Is it expected that releases made through Hockeyapp won't be include in these feeds?

Because of the regression I mentioned in my comment above, I had to start uploading direct to hockey app a while back. It seems like the feed starts with uploads made through AppCenter prior to that point.

Not a big problem as it appears to pick up the next release made in AppCenter.

General comment, I'm finding the fact that these feeds aren't tied to a distribution group a little strange. Since you can have multiple public distribution groups, why wouldn't you be able to have multiple feeds?

[lukaskubanek]

please confirm the hockeyapp sparkle feed urls will continue to work

I've just tested a migration from HockeyApp to App Center with a sample app and as soon as I moved the app over, the HockeyApp Sparkle feed stopped working.

@derpixeldan: Is this the desired behavior? Do we have to force our users to re-download the app manually or do you consider adding support for keeping the HockeyApp Sparkle feed temporarily? I can live with both of the solutions but it would be helpful to know which route to go.

EDIT: To be precise, it didn't stop working as the skeleton is there but all versions are gone.

[lukaskubanek]

~The more worrying thing is that I didn't get any versions in the App Center Sparkle feed neither. I've added more app versions in App Center, integrated the App Center SDK (v2.4.0) and triggered the endpoint for uploading the DSA signature as mentioned in the docs. Neither action had any effect on the feed contents. Maybe I'm holding it wrong but do you have any idea what I'm missing? Does it work for you?~

EDIT: Thanks to @shpakovski for reminding me of what @derpixeldan said above. The Sparkle feed contains 5 newest public releases regardless of the distribution group which is different that it used to be in HockeyApp. In order to achieve a setup of a private group with public Sparkle feed, the solution is to create two groups: first private for testers and second public for Sparkle. It'd be great if this information could be stated in the docs to prevent eventual confusion.

[ajaykn]

Hi @lukaskubanek , @shpakovski
Note that the AppCenter sparkle feed url returns only the releases that are distributed via AppCenter.

To have a seamless migration from HA feed to AC feed, please follow the below steps.
1) Update SUFeedUrl from HA feed to AC feed and distribute the new release to Hockey app.
So that the current users gets the update from HockeyApp and once updated, app starts listening to AC feeds.
2) Now proceed with further updates via AppCenter.

Thanks and let us know if you see any issues. cc @derpixeldan

[rishimodha]

@Ajaykn21 What is the AC feed URL and where do we find this?

Edit: found it: https://api.appcenter.ms/v0.1/public/sparkle/apps/{app_secret}

https://docs.microsoft.com/en-us/appcenter/distribution/sparkleupdates

[invariant]

Hi @lukaskubanek , @shpakovski
Note that the AppCenter sparkle feed url returns only the releases that are distributed via AppCenter.

To have a seamless migration from HA feed to AC feed, please follow the below steps.

1. Update SUFeedUrl from HA feed to AC feed and distribute the new release to Hockey app.

This is impossible for those of us that have already migrated from HA to AppCenter. (Unless there is a way to go back?)

Will you consider a simple HTTP 301 redirect from the HA sparkle feed to the AC one? I have an app with several hundred users in the wild now marooned and will never update. It is very unsatisfactory situation. If I had known that this situation would occur I would never have done the migration.

[rishimodha]

Hi @lukaskubanek , @shpakovski
Note that the AppCenter sparkle feed url returns only the releases that are distributed via AppCenter.
To have a seamless migration from HA feed to AC feed, please follow the below steps.

1. Update SUFeedUrl from HA feed to AC feed and distribute the new release to Hockey app.

This is impossible for those of us that have already migrated from HA to AppCenter. (Unless there is a way to go back?)

Will you consider a simple HTTP 301 redirect from the HA sparkle feed to the AC one? I have an app with several hundred users in the wild now marooned and will never update. It is very unsatisfactory situation. If I had known that this situation would occur I would never have done the migration.

This is hugely concerning. We have 100k+ users on hockey and we need a solution. We can't be in a situation where the app is unable to update.

[dipree]

Sorry for the late reply.

For apps that haven't been moved yet:

• Releases uploaded to HockeyApp don't show up in the App Center Sparkle feed
• Releases uploaded to App Center don't show up in the HockeyApp Sparkle feed

For apps that have been moved, all versions that are released to a public distribution group show up in the App Center Sparkle feed (regardless where they were formerly uploaded).

@rishimodha @invariant we're working on forwarding the HockeyApp Sparkle feed to the App Center Sparkle feed of an app when it was moved. I'll update this thread once it's working.

[rishimodha]

@derpixeldan thanks for the clarification. Does this mean that apps still using the hockeyapp sparkle feed URL will be able to grab the next update distributed via app center? I understand the appsecret key is in a slightly different format (app center uses dashes between the key), so I assume you will put in handling for this?

[WCByrne]

@rishimodha From what I have experienced releases made via AppCenter are no longer added to the Hockeyapp feed. This used to be the case but the AC releases stopped syncing back a little while ago. I commented about this above

I feel like cutting that backwards sync really threw a wrench in this transition 😞

[arrizer]

@derpixeldan The feed works fine for us, but the CHANGELOG.md is printed raw to the description instead of the HTML version. This will look ugly in the Sprakle update prompt. Do we need to format it locally and upload as HTML or will that be done server-side in the future?

HTTP/1.1 200 OK
Connection: keep-alive
Content-Length: 1682
Content-Type: text/xml; charset=utf-8
Date: Thu, 10 Oct 2019 15:18:07 GMT
Server: nginx/1.15.8
access-control-allow-origin: *
strict-transport-security: max-age=15724800; includeSubDomains

<rss version="2.0" xmlns:sparkle="http://www.andymatuschak.org/xml-namespaces/sparkle" xmlns:dc="http://purl.org/dc/elements/1.1/"><channel><title>XXXXXXXX</title><link>/v0.1/public/sparkle/apps/XXXXXXXXXX</link><item><title>Version 1.2.2 (75.191010164422)</title><description># 1.2.2 - 2019-10-10
### Changed
- Switched from HockeyApp to Microsoft AppCenter
- Building with Xcode 11 SDK

...

</description><pubDate>Thu, 10 Oct 2019 14:44:49 GMT</pubDate><enclosure sparkle:version="75.191010164422" sparkle:shortVersionString="1.2.2" url="https://rink.hockeyapp.net/api/2/apps/XXXXXX/app_versions/2?format=zip&amp;avtoken=XXXXXXXXXX&amp;download_origin=hockeyapp&amp;mctoken=XXXXXXX" length="16300711" type="application/octet-stream"></enclosure><sparkle:minimumSystemVersion>10.14</sparkle:minimumSystemVersion></item></channel></rss>

[lukaskubanek]

Do we need to format it locally and upload as HTML or will that be done server-side in the future?

If you decide to build it server-side in App Center, please make this optional and configurable in the administration, as I'm relying on the Markdown format to do my own processing. Changing the format would break things.

[ajaykn]

Do we need to format it locally and upload as HTML or will that be done server-side in the future?

Thanks for pointing this, @arrizer .
We have fixed this and you should be able to see the release notes in sparkle feed in html format.

@lukaskubanek, This formats only for sparkle feed. I hope this works for you.

[lukaskubanek]

This formats only for sparkle feed. I hope this works for you.

@Ajaykn21 Actually, I was relying on the Markdown _within the Sparkle_ feed. So it broke my preprocessing pipeline. Do you have any other plans on changing the Sparkle feed format in the near future (e.g. #1150)? It's hard to keep pace with these changes if they are not announced in advance and the HockeyApp termination is moving closer...

[ajaykn]

@lukaskubanek currently in HockeyApp also, we are converting markdown to html and so we are supporting the same in AppCenter feed.
Please let us know if you see any change in the behavior.
Also we are open for suggestions and #1150 is being tracked separately.

[lukaskubanek]

@Ajaykn21 It actually makes sense to align the output in the Sparkle feed with HockeyApp. I’ve now updated my processing, so everything is fine again. My point was just about not knowing what might change under our hands. So now I’ll assume it won’t break again and I’ll proceed with the migration. 🤞

[dipree]

Update: Once an app was moved from HockeyApp to App Center, we forward the HockeyApp Sparkle feed to the App Center Sparkle feed. Please note that we don't move the dsa_signature and it's required to add them to the releases in App Center again.

[rishimodha]

@derpixeldan Hi Dan, can you please clarify if the forwarding of the HockeyApp Sparkle feed to the App Center Sparkle feed includes handling for the differences in the app secret between the two platforms. I believe App Center has added dashes '-' in between the app secret at certain intervals.

[dipree]

@rishimodha we're forwarding to the correct Sparkle URL. So yes, of course we're converting the app secret. This way existing installations aren't locked out, make sure to update to the App Center Sparkle feed with any newer version of your app.

[rishimodha]

@derpixeldan - thanks for confirming, just wanted to make sure!

[invariant]

Has anyone got Sparkle forwarding to work?

My feed URLs are:
HA: https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089
AC: https://api.appcenter.ms/v0.1/public/sparkle/apps/58adb6c5-bea2-8ffe-22f3-04f2f383d089

The AC feed is working correctly, but the HA is not forwarding to AC feed. Instead, it's delivering a feed of really old versions for some reason.

Is there anything I have to do to enable to forwarding, @derpixeldan?

[ghost]

@invariant Did you already move your app to AC? If not, check which distribution group you have set the version to in HA.

Releases uploaded to HockeyApp only show up in the App Center Sparkle feed when added to a public distribution group.

Source

[invariant]

@derpixeldan Thank you. Yes, the app has been moved to App Center already and is set to a public distribution group. The Sparkle feed on App Center works correctly. The problem seems to be with the HockeyApp feed forwarding. Basically, I'm not getting any forwarding.

When I "curl" the AC sparkle feed, I get the expected feed containing the latest public release. So the AC feed is working correctly.

But when I curl the HA sparkle feed, I get a different feed. (Weirdly it contains only releases from two years ago.)

I was expecting the feed accessed with the HA sparkle URL to be the same as the AC one (or possible a redirect? I'm not sure which way you implemented it).

The actual URLs in question are in my comment above.

[rishimodha]

@invariant your HA feed is correctly forwarding to the AC feed for me. It is being redirected.

[invariant]

@invariant your HA feed is correctly forwarding to the AC feed for me. It is being redirected.

Thank you Rishi. This is really weird, because when I curl the feed I get a 200 OK response, not a redirect.

nick@kvyat:scratch$ curl -v https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089
*   Trying 54.86.96.152...
* TCP_NODELAY set
* Connected to rink.hockeyapp.net (54.86.96.152) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; OU=Microsoft Corporation; CN=*.hockeyapp.net
*  start date: Nov 15 22:51:17 2017 GMT
*  expire date: Nov 15 22:51:17 2019 GMT
*  subjectAltName: host "rink.hockeyapp.net" matched cert's "*.hockeyapp.net"
*  issuer: C=US; ST=Washington; L=Redmond; O=Microsoft Corporation; OU=Microsoft IT; CN=Microsoft IT TLS CA 2
*  SSL certificate verify ok.
> GET /api/2/apps/58adb6c5bea28ffe22f304f2f383d089 HTTP/1.1
> Host: rink.hockeyapp.net
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Date: Mon, 21 Oct 2019 09:02:27 GMT
< Content-Type: application/xml; charset=utf-8
< Content-Length: 7231
< Connection: keep-alive

[invariant]

Now this gets even weirder. If I paste the feed URL into Firefox I get a 301 redirect. But when I curl in Terminal I get a 200 OK. I feel like I'm being gaslit 😅

[invariant]

After some experimenting: The redirect is happening only if the HTTP request has an Accept: header. It can even be a blank header, so long as it is present. The following examples with curl illustrate it.

Running curl https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089 results in 200 OK and old data served.

Whereas curl -H 'Accept: ' https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089 results in the correct 301 redirect as expected.

[rishimodha]

@invariant we just tested moving over to AC with an app that's no longer in production. The hockey sparkle feed url is being redirected, but the page is missing all releases: https://api.appcenter.ms/v0.1/public/sparkle/apps/9a440195-a60b-447e-90d4-5b01c63bbdcc

Did you have to make any changes to get the previous releases to show up? I believe we had the app as a public distribution, but where explicitly is this set?

[invariant]

@invariant we just tested moving over to AC with an app that's no longer in production. The hockey sparkle feed url is being redirected, but the page is missing all releases:

I'm getting releases in the App Center sparkle feed including my public releases from recent weeks, done on HockeyApp. But the most recent release which I did post-migration in AppCenter, only about 10 minutes ago, is not showing up. (Yet - I was assuming for now it was some kind of cache update delay, or maybe I didn't select the right options.)

[invariant]

tested moving over to AC with an app that's no longer in production.

if only I had been so wise .... 🤦‍♂️!

[rishimodha]

@invariant we just tested moving over to AC with an app that's no longer in production. The hockey sparkle feed url is being redirected, but the page is missing all releases:

I'm getting releases in the App Center sparkle feed including my public releases from recent weeks, done on HockeyApp. But the most recent release which I did post-migration in AppCenter, only about 10 minutes ago, is not showing up. (Yet - I was assuming for now it was some kind of cache update delay, or maybe I didn't select the right options.)

Did those HA releases appear instantly on the AC sparkle feed or did you have to wait a while? We're not seeing anything!

[invariant]

@invariant we just tested moving over to AC with an app that's no longer in production. The hockey sparkle feed url is being redirected, but the page is missing all releases:

I'm getting releases in the App Center sparkle feed including my public releases from recent weeks, done on HockeyApp. But the most recent release which I did post-migration in AppCenter, only about 10 minutes ago, is not showing up. (Yet - I was assuming for now it was some kind of cache update delay, or maybe I didn't select the right options.)

Did those HA releases appear instantly on the AC sparkle feed or did you have to wait a while? We're not seeing anything!

On HA it was always instant (or near-instant).
Edit: I misinterpreted the question. There was no Sparkle feed available on AC at the time I migrated from HA to AC. And since then, I have not added any new releases (until this morning).

[rishimodha]

I figure out what I had done wrong. After moving an app from HA to AC, (even if it was public on HA), I had to go into the app's settings on AC and edit the distribution group to enable "App can be downloaded by anyone with link access".

[invariant]

Mini update. When my app was only in the auto-created "All-users-of-app" group, it did not show up in the feed, even though the group was set to public.

I added a brand new Distribution group set to public and released it to that group too, but still the new release did not show up in the feed.

Finally I deleted the "All-users-of ...." group, and at last it appears in the feed.

[invariant]

Thought I was nearly there but ..... Now I discover that my apps still won't update. The version of Sparkle my apps have in the wild has Accept: */* in the HTTP header (not sure if more recent version have a different value? I'm using a fairly old version.) Unfortunately, HA is replying 200 OK to this request, instead of redirecting.

curl -H 'Accept: */* ' https://rink.hockeyapp.net/api/2/apps/58adb6c5bea28ffe22f304f2f383d089

Give a 200 OK with very old data.

[invariant]

I've now created a new issue #1238 to address this specific redirect bug.

[rishimodha]

@invariant have you noticed that even releases uploaded via AC have a download URL that points to hockeyapp.net - is this intentional?

[invariant]

@invariant have you noticed that even releases uploaded via AC have a download URL that points to hockeyapp.net - is this intentional?

Yes the download link of my latest release uploaded to AC points to a URL on the hockeyapp.net domain. But it is functional and serves the correct file.