Syncthing: Announce internal and external address to the global discovery service

[vision57]

There are two PCs in my office, they are both in the same internal office network of my company, but they are not in the same network segment.

I find that they can't connect to each other directly without relay server and UPnP(not enabled by gateway). My current solution is setting internal address rather than default on one of the PCs. This works fine, but I don't think it's a nice solution.

Here I suggest a default behavior of announcing both internal and external address to the global discovery service, so that instances can try to connect internal address of peers if it's possible.

Thanks.

[AudriusButkevicius]

Thats exactly the reason we have local discovery for.

[vision57]

It seems that local discovery can't find peers in different network segments(connected via routers)?

[scienmind]

@vision57 publishing internal addresses to a global discovery is not a good idea security-wise.

[vision57]

@scienmind Could you please explain a little more about security problem? I used to think published internal address would not risk more than external address, because internal address can't be widely accessed.

[AudriusButkevicius]

It's just an address, I don't think it reveals much, yet it's a valid concern. Connections failing under the conditions OP described is also a valid concern however.

[AudriusButkevicius]

It would be easier if we supported zero knowledge discovery, as in discovery messages were encrypted and only readable by the destination device (that the message is intended to).

[scienmind]

@vision57 it is indeed "just an address", and right, your public IP is already published.
However, internal addresses are the layout of your network, and such extra bits of metadata one by one make attacker's life easier (the better you know where to look, the easier the hunt is).

In a nutshell, the more separation you have from the "outside world" the better. Just as having your ports "stealthy" is better than having them simply "closed".

In the current state, syncthing is a very decent solution for people who seek minimal-knowledge secure syncing solution. And it should continue to move towards zero-knowledge direction, not backwards.

As per Audrius's comment, if someday ST will support encrypted zero-knowledge discovery, then it will may be appropriate to consider adding local addresses.

[AudriusButkevicius]

So I think an advanced option that users can opt into to advertise local addresses is not a tragedy until we have zero knowledge discovery.

[calmh]

In the meantime, "office network" often means "AD" which means functioning dynamic DNS.

Zero knowledge discovery is "easy" but means that you cannot find a device until they have added you, making initial contact even less user friendly than it is today - no "this device wants to connect" prompts.

[calmh]

I propose the following:

• Announce global addresses like we do now. There is not much point in hiding these, as the discovery server will see them anyway as part of where the announcement comes from.
• Announce all known addresses (including private ones), encrypted for the known recipients only. These are thus unavailable to unknown devices, and to the discovery server operators.

Optionally, also encrypt the entire announcement (including device ID), thus making it "zero knowledge" from the discovery server point of view and making it impossible to discover the device if you are not already added. The announced "device" ID should be something like the hash of the device ID and the current eight hour time period, or something like that - so that if I know the device ID, I can compute the hash and look it up, but I can't deduce the device ID from the hash only.

[h1z1]

Rather surprised at this. It was my understanding the comunication between nodes was entirely encrypted already? As pointed out above by @scienmind above, leaking internal addresses is bad security practice. Unless there are safeguards in place already to prevent a host from advertising IPs it doesn't actually have, there is nothing to stop them from doing so. Skype was notorious for finding internal (VPN) paths between offices for example.

Why couldn't the nodes exchange their IPs after a session is established and avoid this entirely? From a relay perspective the only thing I care about is what address you come from. Even then I have firewall policies to not trust that unless you establish a connection. At the very least it certainly limited amplification attacks.

[calmh]

@h1z1 Your probing of any and all issues you deem privacy related, yet where you currently lack the technical background to understand the context, is tiring. Please stop. There are many different moving parts (local discovery, global discovery, sync connections, relay connections) and they differ in purpose and capabilities.

[h1z1]

I don't believe you're aware of my technical background. You're asking for feedback and the privacy issues were raised well before I replied. It's interesting you continue to dismiss them however.

[calmh]

I refer to your technical understanding of Syncthing protocols and internals specifically. Your questions/concerns above don't make sense to me in the context, and attempting to answer or counter them is therefore difficult.

[h1z1]

I don't know why you're again focusing this on me guy, the privacy issue was raise before me. This thread is archived none the less should you contine to be belligerent.

[calmh]

This is not archived. It is an open feature ticket. I have a proposal above that you replied to with surprise. I don't understand your surprise so I can't respond reasonably to it, I think because you are not clear on what is proposed, perhaps. In the previous ticket you expressed concerns I also do not understand what you're saying so I can't respond it. This is the pattern that I reacted belligerently to.

[h1z1]

If you wish to continue this discussion here I can. I assumed you did not.

This is archived in that these responses are sent in email as notifications.

[calmh]

If you have some relevant concern that related to this ticket I'm more than happy to discuss it. As it appears we have been talking at cross purposes, let me explain this once what I saw as the problems with your questions above.

Rather surprised at this. It was my understanding the comunication between nodes was entirely encrypted already?

It is. This is not about device to device communication, but a different thing - the global discovery protocol.

As pointed out above by @scienmind above, leaking internal addresses is bad security practice.

The proposal is to encrypt them specifically for the recipient, so I'm not sure "leaking" is the term here. In any case it would clearly be optional.

Unless there are safeguards in place already to prevent a host from advertising IPs it doesn't actually have, there is nothing to stop them from doing so. Skype was notorious for finding internal (VPN) paths between offices for example.

OK.

Why couldn't the nodes exchange their IPs after a session is established and avoid this entirely?

Once they have a session established they could do this. Relay sessions are a last ditch effort, are unreliable, and take a long time to establish. We currently don't rely on relay sessions to be able to do discovery and I don't see any reason to start doing it now. (I'm talking about relays here because that's the only way the question makes sense; the purpose of the discovery protocol is to find the addresses to establish the session over.)

From a relay perspective the only thing I care about is what address you come from. Even then I have firewall policies to not trust that unless you establish a connection. At the very least it certainly limited amplification attacks.

I have no idea what this means. :)

[imsodin]

My opinion, more on the discussion than the matter at hand: I propose to move over to the forum and explain what your understanding of Syncthing's communication is and where you see the privacy concern. Thus there is a chance to both address possible misconceptions as well as legitimate concerns. Once there is some actionable point, this point can be summarized on an appropriate github issue. Thus there won't be a lot of notifications going out to contributors, because that's a waste of time - questions should be sorted out there.

[AudriusButkevicius]

Not archived as in not closed, as this is not solved.

This has nothing todo with node communication (that is all TLS encrypted), but node discovery, as in finding the internet routable IP address of some device.

We can't encrypt addresses for particular recipients as if someone new wants to discover our device there will be no set of readable encrypted records for them, hence device addresses are currently provided in plain text so that new devices.

Furthermore, some of the addresses are not provided by the device itself but computed by discovery server (as discovery server sees the advertisers public ip), hence the discovery server could not encrypt on the devices behalf.

Also, it's not that you can get a listing of all devices and their addresses from the discovery server, you have to know who you are looking for.

[h1z1]

If you have some relevant concern that related to this ticket I'm more than happy to discuss it. As it appears we have been talking at cross purposes, let me explain this once what I saw as the problems with your questions above.

Rather surprised at this. It was my understanding the comunication between nodes was entirely encrypted already?

It is. This is not about device to device communication, but a different thing - the global discovery protocol.

Discovery is part of that communication. I understand you view them as different things but from a security perspective they are the same risk.

As pointed out above by @scienmind above, leaking internal addresses is bad security practice.

The proposal is to encrypt them specifically for the recipient, so I'm not sure "leaking" is the term here. In any case it would clearly be optional/

That is part of the proposal indeed, it is not what appears to be happening now, thus why I'm surprised.

A part your missing is :

Announce global addresses like we do now. There is not much point in hiding these, as the discovery server will see them anyway as part of where the announcement comes from.

There is absolutely a point in hiding anything but the public IP (or primary). The issue is less about the discovery or relay servers and more the intermediate networks between them especialyl with public servers.

Unless there are safeguards in place already to prevent a host from advertising IPs it doesn't actually have, there is nothing to stop them from doing so. Skype was notorious for finding internal (VPN) paths between offices for example.

OK.

Not sure you're agreeing or fulling understanding here? It's directly related to the point above it.

Why couldn't the nodes exchange their IPs after a session is established and avoid this entirely?

Once they have a session established they could do this. Relay sessions are a last ditch effort, are unreliable, and take a long time to establish. We currently don't rely on relay sessions to be able to do discovery and I don't see any reason to start doing it now.

Agreed, I wasn't refering specifically to relays here though.

From a relay perspective the only thing I care about is what address you come from. Even then I have firewall policies to not trust that unless you establish a connection. At the very least it certainly limited amplification attacks.

I have no idea what this means. :)

It means there are a handful of abusive hosts that are attacking relay nodes. One of the ways they were doing this is a glorified syn flood, the other was icmp. In the ICMP case, a connection comes up and someothing upstream starts sending icmp unreachables. The primary difference between that and a normal connection is the connection state is fully established which would pass most statefull firewalls. At one point the host was receiving thousands mostly from nodes in China. The ampllification comes from one connection can cause 10s of such icmp garbage.

I could find no tweak or setting in syncthing* to deal with abusive or otherwise broken hosts, thus the firewall policy.

[calmh]

Discovery is part of that communication. I understand you view them as different things but from a security perspective they are the same risk.

They are not. Compare, for example, DNS and HTTPS which are analogous to our discovery and sync protocols. In our case, discovery is also over HTTPS so it is encrypted in transit. Yet, it is necessary to be able to look up the address of a device, else it cannot be connected to. Hence, addresses in general cannot be encrypted so that noone can read them. That would render the discovery protocol useless.

The rest of your comment is interesting, but not related to this issue. Mind creating an account on the forum so we can discuss it there? I'm particularly curious about the ICMP amplification you're seeing (I suspect the TCP SYNs are just client probes).

[h1z1]

Discovery is part of that communication. I understand you view them as different things but from a security perspective they are the same risk.

They are not. Compare, for example, DNS and HTTPS which are analogous to our discovery and sync protocols.

EDNS aside, it does not disclose any other IP nor host details. The DNS server knows simply the external address.

Mind creating an account on the forum so we can discuss it there?

Perhaps later. Suffice to say there was more to it then just the syn.

[psyhomb]

Anything new on this topic?

[AudriusButkevicius]

It's still not closed, so no.

[whalehub]

@AudriusButkevicius This has been implemented in #6928 now, right?

[AudriusButkevicius]

Indeed

[bminer]

One question, is this feature turned on when "local discovery" is disabled? Perhaps, it shouldn't be?

Digression: When I saw this feature in the changelog, I thought I'd investigate the implementation details (I was also curious to see if there were any additional security risks). I'm so glad to see brilliant folks working hard to ensure Syncthing continues to be such a well-engineered piece of technology. You guys rock!

Keep up the great work everyone! Special thanks to Jakob @calmh! What a great success story for the Go programming language, as well.

[AudriusButkevicius]

One of the things this tries to combat is non-functioning local discovery (due to network configuration), hence local discovery enablement has no effect on what addresses are advertised.

[bminer]

@AudriusButkevicius - I think I understand a little better now.

As an aside, why would one ever disable local discovery while keeping global discovery enabled?

I originally thought that by disabling local discovery, you might want also want to avoid announcing local addresses to the global discovery service. After all, "Local discovery" is a more prominant option that appears in the settings modal in the UI; whereas, AnnounceLANAddresses is hidden in the advanced options modal.

However, the more I think about it... local discovery is about sending UDP discovery packets; whereas, AnnounceLANAddresses is more closely related to global discovery (allowing peers to try announced LAN addresses, too).

So, in short, never mind. :)