Mastodon: Support SRV records in Mastodon

[sardemff7]

With XMPP, the well-defined DNS SRV record is used to specify which server will serve the XMPP protocol (one record for server-to-server and one record for client-to-server).
Mastodon should do the same.

This is not a way to point a user to change its name. Let’s see a simple configuration:

I run an instance at example.com (and this is the domain configured in my .env.production). My client/other server asks for _mastodon-client._tcp.example.com/_mastodon-server._tcp.example.com and get SRV 3000 mammoth.example.com (or even SRV 443 mammoth.example.com, nothing prevents that). That means the client/other server must connect (with HTTPS/WebSocket, nothing changes) to the machine mammoth.example.com on port 3000.
If some one adds _mastodon-client._tcp.example.org SRV 3000 mammoth.example.com it won’t work (at least it shouldn’t, the server would check if the domain matches its configured domain).
A good thing would be to have a specific header identifying the request as being done this way (either generic like DNS-SRV: mammoth.example.com or specific Mastodon-SRV: mammoth.example.com).

This changes only one part of Mastodon (and clients wanting to support that, I bet there will be many): the TCP address and port when connecting to a server, all the rest (except the would-be-neat extra header) will stay the same.
I couldn’t easily spot where this connection is done in Mastodon or its dependencies, but someone with better knowledge of the codebase would probably find it in a blink. (Maybe are there multiple places but introducing an helper should work then.)
I’m willing to provide a patch in one could point me to these entry points.

I read several issues (#1032, #1211 and others I cannot find again) throwing SRV records in the conversation with (apparently) little knowledge of how they work. It would solve at least a few use cases, while being (hopefully) not too invasive in the codebase and providing ground work for future multiple-domains or on-the-fly migrations work.

---

• [x] I searched or browsed the repo’s other issues to ensure this is not a duplicate (mostly).

[bortzmeyer]

I fully agree and this is an excellent idea but it alas goes against everything HTTP does since the beginning, and the very unfortunate mistake by Tim Berners-Lee who decided not to use indirection in domain names :-(

For instance, today, https://mastodon.gougere.fr/@bortzmeyer/394973 works in every Web browser. If Mastodon were to use SRV records, how would it work with ordinary browsers, which unfortunately do not have SRV support?

[sardemff7]

The web interface has nothing to do here, since it would be for the API only.
API consumers being Mastodon servers (server-to-server, fully controlled code) or dedicated clients (hopefully not limited to JS APIs).

OTOH, there is an already working solution based on WebFinger, using LOCAL_DOMAIN and WEB_DOMAIN. Since it seems to be the first and single entry point, and we can redirect to another domain from that, maybe can we call it the HTTP SRV. :-)

[bortzmeyer]

@sardemff7 First, if SRV were to work only for the API and the Ostatus updates, but not for ordinary Web browsers, it would be quite disconcerting for the users. "Your friend is [email protected] but if you want to see his toots in a browser, go to https://mammoth.example.com:3000/@michu"

Second, there are already several clients that would need to be upgraded. We risk a chicken-and-egg problem (people not deploying SRV because not all clients are up to date, and clients not being upgraded because there are not SRV in the wild), problem which is very common on the Internet.

[sardemff7]

The Mastodon web interface itself would be aware of that, because the backend would be. So all links in the interface would work just as expected. (For the streaming API, at the cost of a SRV request done in the backend, since JS cannot do that itself.)
However, nothing can tell that a browser request is supposed to show the Mastodon web interface, so poking https://example.com/@michu directly will just never work (in the “not the same domain case”, whatever underlying technology we use to decouple them).

Anyway, let’s see how far the WebFinger solution can get us, at least there already is code for it, and it works for several instances.

[bortzmeyer]

@sardemff7 The following discussion about SRV in WebFinger is relevant, I think https://groups.google.com/forum/#!msg/webfinger/famyJhnvKPU/pK4hAGQDS0wJ

[Gargron]

OStatus is entirely in HTTP. So I don't think SRV has any use here.

[plinss]

@gargron OStatus being on top of HTTP does not preclude use of a SRV record, other HTTP based protocols use them, see Matrix for an example: https://github.com/matrix-org/synapse/#setting-up-federation

A SRV record simply provides a layer of indirection between the domain name used as a federation address and the domain name (and port) of the server ultimately providing the service. Please reconsider.

[Gargron]

The entrypoint to Mastodon from a federation perspective is Webfinger, which is standartized to live under /.well-known/webfinger - from there, you can link whatever URLs you want (just like SRV). But the Webfinger URL is not negotiable. On the other hand, it is entirely possible to reverse-proxy just that path from a domain used by some other software. (https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Serving_a_different_domain.md)

[plinss]

Right, but some people, like myself, run multiple services under different subdomains. Several of these have Webfinger entry points and they can't all be proxied from the base domain without stepping on eachother. SRV records would allow each service to have its Webfinger URL provided at the subdomain and be discoverable by service type while using identifiers based solely on the base domain.

[nightpool]

@plinss

SRV records would allow each service to have its Webfinger URL provided at the subdomain and be discoverable by service type while using identifiers based solely on the base domain

How would this work? Are you saying that if I have an identifier [email protected] I should query two different domains to resolve the web finger template? That seems ridiculously complex. How are you ensuring a that the identifier is unique?

[plinss]

The premise is that you can have an identifier [email protected] and run your Mastodon instance and related Webfinger service at social.example.org. When another server wants to find your server it does a DNS query for a SRV record like: _social._tcp.example.org, getting a result of 0 0 443 social.example.org. Then the server performs a Webfinger lookup at https://social.example.org/.well-known/webfinger and proceeds from there. If there's no _social._tcp SRV record at example.org, then it does the Webfinger lookup at example.org as normal.

There's still only one Webfinger lookup, just an additional DNS query before, allowing indirection to another domain, possibly a subdomain, possibly another domain entirely (it also allows load balancing with multiple SRV records and/or moving the service to a different port). XMPP works this way, as does Matrix, as does email (though that uses MX records instead of SRV). This allows the user to use the same identifier for multiple services and aids in discoverability (eg. if you know my email address, you also know my XMPP address, and should be able to find me on Mastodon without having to guess which subdomain I'm running it on).

[nightpool]

you said "Several of these have Webfinger entry points and they can't all be proxied from the base domain without stepping on eachother". So how do you ensure that the webfinger records are all in sync if they're on separate subdomains? the solution you're proposing would basically defeat the point of using webfinger.

[plinss]

My point is that the Webfinger records of the multiple services are already not in sync, and I have to distinguish the services by different host names. For example, I'm running a Mastodon instance at social.example.org and a Nextcloud instance (providing WebDAV, CardDAV, CalDAV, and Webfinger) at cloud.example.org. Currently I have to use the identifier [email protected] for Mastodon and [email protected] for my DAV services. The solution proposed earlier is to proxy https://social.example.org/.well-known/webfinger from https://example.org/.well-known/webfinger, while this allows me to use [email protected] for Mastodon, it leaves me out of luck for my DAV services.

Ideally I suppose I'd have a Webfinger aggregator running at https://example.org/.well-known/webfinger and it would have knowledge of the other services provided at the other domains (and all the accounts at each) and be able to synthesize a proper combined response, but I don't know of any such service existing at the moment. Having a SRV record would allow clients and federation servers to get the Webfinger data they need for Mastodon without needing anything other than a Mastodon server and a single DNS record.

[nightpool]

If you want to run multiple web finger services on the same domain, you
NEED a webfinger aggregator. To propose otherwise would defeat the entire
point of the webfinger protocol.

Webfinger is designed to be the authoritative source on user information
for that domain. The concept of having multiple authoritative sources
depending on what TYPE of record you want to look up is just madness.

Mastodon provides a webfinger server to simplify the situation for the 90%
of servers that only run one service. If you want to set up your domain in
a more complicated way, then you're going to need to manage your own
webfinger configuration.
On Fri, Jun 30, 2017 at 2:43 PM Peter Linss [e-mail]
wrote:

My point is that the Webfinger records of the multiple services are
already not in sync, and I have to distinguish the services by different
host names. For example, I'm running a Mastodon instance at
social.example.org and a Nextcloud instance (providing WebDAV, CardDAV,
CalDAV, and Webfinger) at cloud.example.org. Currently I have to use the
identifier [email protected] for Mastodon and
[email protected] for my DAV services. The solution proposed
earlier is to proxy https://social.example.org/.well-known/webfinger from
https://example.org/.well-known/webfinger, while this allows me to use
[email protected] for Mastodon, it leaves me out of luck for my DAV
services.

Ideally I suppose I'd have a Webfinger aggregator running at
https://example.org/.well-known/webfinger and it would have knowledge of
the other services provided at the other domains (and all the accounts at
each) and be able to synthesize a proper combined response, but I don't
know of any such service existing at the moment. Having a SRV record would
allow clients and federation servers to get the Webfinger data they need
for Mastodon without needing anything other than a Mastodon server and a
single DNS record.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tootsuite/mastodon/issues/1931#issuecomment-312344596,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAORV3zYj8msgtnd2BSxulJQcD2Yo32pks5sJUHhgaJpZM4M-o8r
.

[cpacejo]

As an outsider researching Mastodon for the first time, I want to say that I find it strange that a federated, decentralized service would require me to run my own web server if I want to maintain ownership of my digital identity. Neither e-mail nor XMPP require this.

That Mastodon happens to utilize HTTP, that the HTTP community happens to have standardized on WebFinger for locating services within an HTTP server, and that vanilla HTTP doesn't require SRV lookups, seems like a weak argument against support of SRV records.

Counterarguments: WebFinger is not authoritative for a "domain"; it is authoritative for an HTTP server. Consider that multiple HTTP servers may live at a single domain, but on different ports. Consider also that it is arguably a bad idea to run an HTTP server on certain domains, such as apex domains. For these reasons WebFinger does not replace SRV as an authoritative means to locate service providers for a given domain.

Further, just because vanilla HTTP does not support SRV, does not mean that a protocol derived from HTTP cannot support SRV. See for example WebDAV, specifically RFC 6764.

I do hope Mastodon development will reconsider their stance on this issue. It is key to making the benefits of federation accessible.

[nightpool]

@cpacejo as an outsider researching mastodon for the first time, I would expect that you would take time to understand the decisions people have already made and the constraints they were made under before assuming you know better then them.

WebFinger is not authoritative for a "domain"; it is authoritative for an HTTP server

this is untrue. Have you read the webfinger spec? webfinger is authoritative for a URI. If you give mastodon the URI acct:[email protected] it will make the request https://example.com/.well_known/webfinger?uri=acct:[email protected]. this is an unambiguous mapping from URI -> webfinger requests.

run my own web server if I want to maintain ownership of my digital identity. Neither e-mail nor XMPP require this

not sure what you mean by this. You can make DNS records, but not serve a static HTTP page? that's not a use-case i've ever heard before. There are hundreds of free static hosting providers out there.

Mastodon does not support users on other domains but it is fully compatible with other software that does, such as bridgyfed, software that all uses the same activitypub + webfinger protocols. Before saying that a usecase isn't addressed by the protocol, look to see if other implementations of the protocol prove you wrong.

It is key to making the benefits of federation accessible

normal users, in general, do not own domains. if requiring someone to own a domain is what you consider "accessible", then you're using a very different meaning of the word then we are.

[cpacejo]

I would expect that you would take time to understand the decisions people have already made and the constraints they were made under before assuming you know better then them.

Snarky response. I have read this thread; the only constraints I've seen put forward is that existing clients would need to be patched (this seems… tolerable?), and that web interface URIs would no longer conveniently map directly to usernames. Maybe the latter is a bigger issue than it's made out to be in this thread. Certainly no other constraints were cited when the issue was closed. Are there other reasons I'm missing that SRV support is burdensome for servers?

Yes, I read the WebFinger spec in its entirety. You're right, it's authoritative for a URI. That doesn't conflict with anything I said: https://example.com/ and https://example.com:1234/ are different URIs and have different authoritative WebFinger servers, despite being the same domain. Ergo, WebFinger is not authoritative for a domain.

You can make DNS records, but not serve a static HTTP page? that's not a use-case i've ever heard before.

Free DNS servers are similarly easy to come by (they are often provided by registrars), and DNS (unlike J. Random Free Webhost) is highly reliable. And, anyone who cares about their digital identity (like I do) has access to one, by virtue of owning their own domain. Not everyone with a personal domain wants to manage their own infrastructure, even if it's a static web host.

Please don't make the assumption that "has control of DNS records for example.com" implies "has control of https://example.com/.well_known/webfinger". In particular there are many reasons that one may need to outsource the web service of their domain, and therefore cede control of that URL. Even if one controls the web space of a given domain now, that may not be true in the future.

normal users, in general, do not own domains.

Nor do they run web servers, or use Mastodon for that matter. But for users wishing for digital identity independence, the burden of "buying a domain name and entering a couple lines in the free DNS server it comes with" is significantly less than doing that and obtaining web space and setting up WebFinger, and it would be good to encourage that.

[nightpool]

it was a snarky response because the fact of the matter is that Mastodon
isn't a standards making body and this issue has no purpose being on this
repo. lots of people who are far smarter then me have written extensively
about why the existing social web uses web-based protocols. the mastodon
issue tracker is not the place to relitigate that discussion

On Sun, Aug 5, 2018, 11:34 PM Chris Pacejo [e-mail] wrote:

I would expect that you would take time to understand the decisions people
have already made and the constraints they were made under before assuming
you know better then them.

Snarky response. I have read this thread; the only constraints I've seen
put forward is that existing clients would need to be patched (this seems…
tolerable?), and that web interface URIs would no longer conveniently map
directly to usernames. Maybe the latter is a bigger issue than it's made
out to be in this thread. Certainly no other constraints were cited when
the issue was closed. Are there other reasons I'm missing that SRV support
is burdensome for servers?

Yes, I read the WebFinger spec in its entirety. You're right, it's
authoritative for a URI. That doesn't conflict with anything I said:
https://example.com/ and https://example.com:1234/ are different URIs and
have different authoritative WebFinger servers, despite being the same
domain. Ergo, WebFinger is not authoritative for a domain.

You can make DNS records, but not serve a static HTTP page? that's not a
use-case i've ever heard before.

Free DNS servers are similarly easy to come by (they are often provided by
registrars), and DNS (unlike J. Random Free Webhost) is highly reliable.
And, anyone who cares about their digital identity (like I do) has access
to one, by virtue of owning their own domain. Not everyone with a personal
domain wants to manage their own infrastructure, even if it's a static web
host.

Please don't make the assumption that "has control of DNS records for
example.com" implies "has control of
https://example.com/.well_known/webfinger". In particular there are many
reasons that one may need to outsource the web service of their domain, and
therefore cede control of that URL. Even if one controls the web space of a
given domain now, that may not be true in the future.

normal users, in general, do not own domains.

Nor do they run web servers, or use Mastodon for that matter. But for
users wishing for digital identity independence, the burden of "buying a
domain name and entering a couple lines in the free DNS server it comes
with" is significantly less than doing that and obtaining web space
and setting up WebFinger, and it would be good to encourage that.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/tootsuite/mastodon/issues/1931#issuecomment-410579533,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAORVyRtglY7Gfd55tKP-yBX7Zv5G0FNks5uN7kogaJpZM4M-o8r
.

[nightpool]

anyway, if you want to set up your mastodon server at example.com:1234 that
literally works today 100%, that's why I said authoritative for a uri. I
don't know who you're thinking you're arguing against by splitting hairs
between the authority section and the domain. mastodon uses the authority
section (host + port)

>

[nightpool]

also, wait, the thing you're talking about doesn't even need a srv record at all! you can do everything you want to today with a normal cname, the same way custom domains work on any other website.

[cpacejo]

Fair enough (re: standards bodies). Though ActivityPub, in contrast to OStatus, seems to have ceded the choice of ID→URI mapping to the implementations themselves, and WebFinger allows some wiggle room in host discovery.

CNAME is a good idea but unfortunately, unlike SRV, the redirection is opaque to the client, so it will run into issues with name-based virtual hosting. It also precludes other records (such as MX) from existing on the same domain (though of course, one could relegate Mastodon identities to their own subdomain).

Anyway thanks for your patience and for taking the time to understand my issue.

[nightpool]

so it will run into issues with name-based virtual hosting

because of the way webfinger lookup works, you'd need to configure the target server anyway. it's not like you can set up a MX record pointing at gmail.com and expect everything to work haha.

It also precludes other records (such as MX) from existing on the same domain

yes, although you could use ANAMEs or other similar hacks to get around this. many services provide the opportunity to use either CNAMES or A records to host a custom domain. (tumblr, squarespace, github pages)

[r3pek]

can we have another shot at this?

I really tried to understand WebFinger to see if it could solve the problem of having "same ID for multiple services", but right now, I really can't see how WebFinger could solve it. The only thing I see that could solve it is SRV records on the DNS Zone.

[sneak]

Please reopen this. I use the domain sneak.berlin for my identity, and [email protected] is my identity, printed on my business cards. sneak.berlin is static, CDN hosting, and is not and never will run dynamic web content, but I would like to use [email protected] as my identity, via SRV records that can point at sneak12345.hosted-mastodon.com or whatever.

The fact that https://sneak.berlin/@sneak isn't the canonical URL for my posts is irrelevant.

[Perflyst]

Nitrokey somehow did it: https://social.nitrokey.com/@nitrokey
They have it hosted on social. but the id is @[email protected]

[nightpool]

@Perflyst yes, that's using the technique i mentioned. https:​//nitrokey.com/.well-known/host-meta serves a document referencing social.nitrokey.com

[nightpool]

I'm locking this issue, because the original topic was specifically about using SRV records, which do not work for a variety of reasons. If you find this topic while trying to host mastodon on a different server from the @[email protected] webfinger domain, please see this doc, which explains in detail how to set that up:

https://github.com/tootsuite/documentation/blob/master/Running-Mastodon/Serving_a_different_domain.md