Mopidy: Move Mopidy infrastructure away from Rackspace

[jodal]

Rackspace has been sponsoring Mopidy with a VPS for the last several years. They are now discontinuing their Open Source Discount program, ref. this email I received from them yesterday:

Thank you for participating in the Open Source Discount program! Over the past 3 years, Rackspace has provided over 100 startups with discounted cloud infrastructure through this program. We are proud of our contribution in this area, and hope these credits have helped contribute to the success of your organization as it has for so many others.

With that said, we have decided that now is the right time to close out this program. On December 31, 2017, the Open Source discount program will end, and credits will no longer be applied to your account. You will be responsible for your full invoice amount moving forward.

A big thanks to Rackspace for the support!

---

Now, let's move on...

Let's keep discussion around alternative hosting in this issue and track what is left to do before the end of the year.

Currently we have the following Nginx vhosts on our Rackspace server:

apt.mopidy.com

Static APT archive, includes scripts, software and GPG key to update the archive with new packages.

Hosting alternatives:

• PackageCloud. I think this alternative includes changing to signing with a GPG key issued by PackageCloud, which would require all existing users to import a new GPG key to get updates. That's a deal breaker for me.
• A new Mopidy VM on the machine that @adamcik and I have our private VMs.
• Other alternatives?

auth.mopidy.com

Spotify OAuth bridge so we can use OAuth from non-web applications.

Hosting alternatives:

• Heroku would be nice, but I guess we would need the Hobby level ($7/month) to avoid the app sleeping.
• A new Mopidy VM on the machine that @adamcik and I have our private VMs.
• Other alternatives?

discuss.mopidy.com

Our Discourse forum.

I think we should apply to Discourse for free hosting. I can do this.

dl.mopidy.com

Pi Musicbox release image hosting.

I think we should upload all of these to GitHub Releases ASAP and drop the vhost at the end of the year. I can do this if @kingosticks agrees.

[kingosticks]

apt.mopidy.com

I have a new (reduced, more manageable) Pi Musicbox v1.0 which only has a couple of bits packaged. I want to get this out there at some point. Would I be able to get sufficient access if it's a VM on your private machine? And I assume you guys pay for this yourselves, which isn't fair.

I think we should apply to Discourse for free hosting. I can do this.

I agree, I think we fit the bill.

Pi Musicbox release image hosting.

I think we should upload all of these to GitHub Releases ASAP and drop the vhost at the end of the year. I can do this if @kingosticks agrees.

I agree and I will do this, probably at the weekend.

[kingosticks]

Sorry, I missed that @jodal said he'd upload the old releases. If you want to do that then go right ahead!

[adamcik]

The only expense for the VM machine is buying it initially and repairs if something goes wrong. We've not used it before as we don't want to rely on "private" infrastructure and we can't give out as much access to it.

For the OAuth bridge I was looking at if AppEngine or something like that could have a tier that would work. It really doesn't need much to function, and could easily be ported to an other storage system than sqlite if needed.

[kingosticks]

Netlify do a free tier that's suitable for OSS. Could we use them to host apt.mopidy.com?

[adamcik]

Some combination of https://cloud.google.com/free/docs/always-free-usage-limits might work for the OAuth bridge.

[allquixotic]

I wonder if you could get a company that doesn't normally do free hosting to sponsor the project? OVH, Hetzner, online.net... there are plenty of options which are extremely high volume hosting companies that own their own DCs and would hardly notice giving out a physical machine with 128GB of RAM and 2 x Xeon E5 to an OSS project :) Then you could use a secure container solution like LXD, OpenShift or SmartOS to create areas with different levels of sensitivity/access (e.g. to give certain people ability to update / publish packages in one area without giving them dangerous user-level SSH to the whole infrastructure).

[jodal]

I've now applied to Discourse at https://blog.discourse.org/2016/03/free-discourse-forum-hosting-for-community-friendly-github-projects/ for a hosted Discourse instance, with the following motivational letter:

Greetings!

The Mopidy project already have a Discourse site at https://discuss.mopidy.com which serves the community around the Mopidy project, the 50+ Mopidy extensions (https://pypi.org/search/?q=mopidy), and the Pi Musicbox (http://www.pimusicbox.com) distribution for Raspberry Pi.

We've been self-hosted on a sponsored Rackspace server for the last few years, but Rackspace are now ending their Open Source Discount program (see https://github.com/mopidy/mopidy/issues/1643 for details), and we're looking for a new host for our Discourse site.

Can you help us out?

[jodal]

All Pi Musicbox releases from https://dl.mopidy.com/pimusicbox have now been uploaded to https://github.com/pimusicbox/pimusicbox/releases.

Disclaimers:

• To be able to create GitHub Releases, I needed a git tag for each release. As old Pi Musicbox releases was quite handmade there is probably no commit that matches perfectly. As a workaround I created an orphan commit that I pointed all the new tags to.
• I tried setting the commit and tag date to 1970-01-01T00:00:00Z to get them sorted after the currently latest release, 0.7.0rc4, but that didn't work.

[kingosticks]

Thanks @jodal, sounds like that would have taken me a long time to figure out and do.

There are some suggestions for Rackspace alternatives at https://news.ycombinator.com/item?id=15502356, Linode sounds interesting.

[jodal]

Three updates...

dl.mopidy.com

Thanks to @trygveaa we figured out that only the author date and not the commiter date for the release placeholder commit was set to 1970. With that fixed, the old Pi Musicbox releases are now sorted after the latest releases.

The dl.mopidy.com vhost has been updated with the following redirect rules to keep all URLs working:

rewrite ^/pimusicbox/$ https://github.com/pimusicbox/pimusicbox/releases permanent;
rewrite ^/pimusicbox/pimusicbox-(.*).zip https://github.com/pimusicbox/pimusicbox/releases/download/v$1/pimusicbox-$1.zip permanent;

discuss.mopidy.com

Discourse has approved our application for free hosting with the only requirement being that we change the hostname from discuss.mopidy.com to discourse.mopidy.com. I don't have an issue with that and would love to get rid of the regular task of upgrading Discourse and the irregular task of rebuilding the Docker image.

I'll move ahead with this migration unless someone has strong arguments against. I'll of course make discuss.mopidy.com redirect to discourse.mopidy.com.

Rackspace turns around

I received the following email from Rackspace today:

We apologize for any confusion yesterday’s communication may have caused. While we’re no longer accepting new applicants to the Open Source Discount (OSS) program, all existing OSS program customers will keep their credits. Rackspace is committed to open source technology, and we look forward to continuing to support you and your Open Source project.

The use of the word "credits" makes this a bit vague, but that's probably just their internal solution to giving us free hosting. I interpret this as: we're free to stay on Rackspace for as long as we like.

With that message in mind I suggest that we:

• continue the deprecation of dl.mopidy.com,
• continue the move to Discourse for hosting of the forum, and
• leave apt.mopidy.com and auth.mopidy.com running on the Rackspace server for now.

[jodal]

Two updates...

dl.mopidy.com

All references I know of in the Pi MusicBox docs and web site have been updated to refer to GitHub Releases instead of dl.mopidy.com.

I think the only thing that remains now is to let a couple of years pass before dl.mopidy.com silently disappears.

discuss.mopidy.com est mort, vive discourse.mopidy.com

I've just completed the migration of the Discourse forum from our own Rackspace server to sponsored hosting by Discourse over at https://discourse.mopidy.com. Thanks to Discourse for great support so far! :-)

Migration process

1. Put discuss.m.c in read only mode.
2. Take a full site backup.
3. Share the backup with Discourse support.
4. Discourse restored the backup on discourse.m.c and adjusted upload paths, etc, to match their setup.
5. Update the settings for GitHub/Google/Facebook/Twitter OAuth login. Yahoo seems to not require any settings. Verification of Yahoo login still working would be nice.
6. Test, test, test.
7. Redirect discuss.m.c to discourse.m.c.
8. Tear down old email reply setup which included a Mailgun forwarding route, a GMail account, and POP3 polling.
9. Update all references to discuss.mopidy.com across the repos in https://github.com/mopidy and https://github.com/pimusicbox.

Known regressions

Replying by email to emails sent out from before the migration, with reply address on the form "[email protected]", does not work as Discourse has their own email reply setup. I've removed the Mailgun route to make these replies bounce instead of disappearing in silence.

@kingosticks will probably be affected by this, so a mention by name here to make him aware of this.

[kingosticks]

Brilliant thanks @jodal. I tend not to reply by email much the days since it was quite slow.

[jodal]

Two years later, I've now removed all traces of dl.mopidy.com and discuss.mopidy.com.