Go: proposal: Vanity URLs via DNS

[benbjohnson]

Problem

Using the hosted URL of a package (e.g. github.com/benbjohnson/foo) for the import URL causes portability issues since all dependent code requires changes if the hosting is changed. This occurs in a few notable situations:

1. A hosting provider shuts down (e.g. Google Code).
2. A project moves to a new GitHub organization or owner.
3. A project moves to a new hosting provider (e.g. migrating from GitHub to GitLab).

This currently can be solved using vanity URLs but this approach requires that the project owner host her or his own vanity URL server which requires more effort and it is likely less reliable than GitHub.

Solution

Instead of using a vanity server, I propose that the Go toolchain supports vanity URLs via DNS TXT or SRV records. The toolchain can lookup the record at import time and fetch the project from its host (e.g. GitHub). Lookups can be cached based on the TTL in the DNS record.

This would provide the following benefits:

1. DNS is highly reliable.
2. Easy to change hosting provider.
3. Could support backup mirrors by using SRV priority.

In a discussion with @rakyll on Twitter, I provided the example that rakyll.pizza could point to her github.com/rakyll account. Another option would be to point to a specific project such as using hey.ninja to point at github.com/rakyll/hey. (Ridiculous TLD names are optional, but fun)

I don't have specifics on the implementation right now but I am proposing this initially to see if it is of interest to the Go community.

[rakyll]

We can easily differentiate whether the record is an org or a repo and give the user to flexibility to use either of their URLs.

[titanous]

The major downside to using DNS for this is that there is no authenticity guarantee, whereas the current requirement to serve a meta tag over HTTPS ensures that an attacker cannot maliciously redirect a package download just by spoofing a DNS response. For this reason, I'd strongly suggest sticking to serving vanity import URLs over HTTPS.

[stp-ip]

There is a small project on my list to make it easy to do redirects with a minimal go server (or use a hosted one) and a standardized DNS TXT record to store the actual redirect. We might just provide one of the redirect servers for convenience.

[rakyll]

I have a minimal server at https://github.com/googlecloudplatform/govanityurls, and I can move it to x/tools -- a more suitable place now without any dependencies to App Engine -- and we can collaborate. In my personal experience, downtime of this server has never been an issue (at least not as much as downtime of GitHub), but I host the server on a very stable provider with very stable networking.

As far as I understand from the linked Twitter thread, the remote idea of owning/maintaining servers scare people and hence we were considering options that doesn't require a server. Though we should never sacrifice security.

/cc @zombiezen

[benbjohnson]

@titanous That's a good point. It's certainly not worth giving up security. Is that something that could be fixed using DNSSEC?

[titanous]

@benbjohnson No, DNSSEC is not widely deployed and there's no evidence that it will ever be deployed to the ubiquitous level that would be necessary for this. See these articles for details:

• https://sockpuppet.org/blog/2015/01/15/against-dnssec/
• https://sockpuppet.org/stuff/dnssec-qa.html
• https://sockpuppet.org/blog/2016/10/27/14-dns-nerds-dont-control-the-internet/

[benbjohnson]

Yeah, looks like sockpuppet.org _really_ doesn't like DNSSEC. :)

I'll close this proposal for now. If anyone has improvements around it then I can reopen.

[daenney]

Why not allow for both? If your domain does support DNSSEC, and a lot of g/cTLDs do, it's a question of deploying it yourself or having a DNS provider that does. Just b/c the owner of sockpuppet.org doesn't like DNSSEC and happens to have a few blog posts about it doesn't make this shouldn't be pursued. About 89% of TLDs are signed and cTLDs current sit at about 47%, adoption wise. Aside from that, the major deployed DNS servers and client libraries support DNSSEC and so do most recursive resolvers.

.com, .net, .org and .io are all signed (so is .us and aside from Albania and Belarus so are all the European ccTLDs, quite a few in Asia, South America, Australia and New Zealand) which are rather prevalent TLDs when it comes to where packages are hosted on. DNSSEC has become significantly easier to deploy since it was conceived and more and more registrars offer support for it in their control panel, it's just a box you need to tick. Though it does require some testing before enabling it, it's technically not complicated to do so. If you also host your own authoritative servers for your domain then yes, it'll be a bit more work.

If the response does pass DNSSEC validation it should be entirely feasible to follow along with this proposal. If it doesn't, fall back to the current way of doing it. That would allow to gradually upgrade to this proposal without breaking anything current.

[titanous]

@daenney The problem is that DNSSEC as currently deployed doesn't do anything for clients. The vast majority of machines use stub resolvers that don't do DNSSEC validation, so this whole thing is moot.

[daenney]

Oh right, the validation part. Most seem to fallback when it fails 😞. Afaik though you can see if the request did pass DNSSEC validation, and systemd, macOS and Windows 7+ do support it in their client code, which covers a lot of people.

[titanous]

@daenney The Authenticated Data flag that you are talking about is just a bit in a cleartext DNS packet from the resolver, it can be tampered with/spoofed by any attacker, so it doesn't help security at all.

[daenney]

But how does that create a problem? If AD is set your system's stub resolver will validate the response and afaik it's not feasible for an attacker to tamper with the signature. If an attacker removes AD then you simply go for the fallback behaviour. I'm not aware of any (stub) resolver that just blindly trusts the AD flag and doesn't verify the signatures.

[titanous]

@daenney There are no signatures for the stub resolver to verify, so it does just blindly trust the AD flag in responses from the resolver. This Microsoft documentation has a good explanation.

[daenney]

@titanous Damn. I've always understood it the other way around :|. That sucks. Never mind indeed then.