Beaker: Add a proxy configuration to the dat protocol

[felix91gr]

Operation System: Linux
Beaker Version: 0.7.5

Hi all,

In the CS faculty, we want to use Beaker from campus. But outside of wired LAN, the uni’s network seems to be blocking some of the ports used by Beaker.

We can ask the IT department to open the ports, but they are usually closed to faculties’ requests. Therefore, we’re thinking of using a proxy: wired and therefore free of blockage, through which we would connect to the p2p network.

We have permission to do so. Is it possible?

Thanks,
Félix

[pfrazee]

We haven't tackled this yet, and I haven't researched what it will involve, but I think it'd be a good idea. I'm going to turn this into a todo. I'll try to updated it with some research soon.

[joehand]

👍 been thinking about how to do this on a per-archive basis. (basically listen to incoming requests and download those files before responding)

Would be interesting to add a layer on top that would do a similar thing when requesting archives that don't exist yet in the proxy.

[ScriptTiger]

I think @joehand is basically describing a type of caching proxy setup, but I think there should be options for both caching and non-caching to work with different setups, if that's possible with Dat. Perhaps it could even work similar to other current Dark Web implementations where a simple service runs in the background to provide a SOCKS5 or HTTP/HTTPS proxy, and then they can in turn be easily chained into something like a Privoxy/Squid setup.

Since Dat is fundamentally a different protocol from HTTP/HTTPS, a bit more conversion might have to happen, but maybe it could also tag on a .dat top level domain to Dat addresses so clients of such proxy servers would essentially not know the difference between Dat and HTTP/HTTPS, conceptually similar but fundamentally different from the way .onion, .i2p, etc., addresses work. I don't know if there might be some security concerns about that or not though. Right now there is already the ability to re-serve Dat websites as HTTPS, so essentially a proxy would be doing the same thing for any requested address. It doesn't seem like too much of a leap anyway.

[xek]

I have the same issue as the original reporter - basically I can't try out Beaker on a corporate laptop and can't really recommend it to anyone else, or show it to them, because it doesn't work here. The only way out to connect to the wider Internet is through a proxy, so the best solution would be to make it autodetect proxy settings or, at the very least, have the option to provide this configuration manually.

I suspect there will still be connection issues, even after this specific issue is resolved. This is because Beaker wouldn't be able to accept connections behind a proxy by listening on a socket. The socks protocol has an option to bind ports, but I don't think it is widely supported. So, in that scenario, a different and more proactive connection strategy may have to be adopted.

[ScriptTiger]

Tor is open source and has plenty of methods around situations like that which maybe could be borrowed, all of their pluggable transports or forcing data over common ports like HTTP/80, etc. I2P, another open-source P2P technology, gets around this using the concept of "introducers." In the meantime there's also just simple ssh reverse port forwarding, but I don't know the specifics of how the P2P pieces listen for the Beaker Browser or how many ports are needed, if they are known ports, etc. You could also try VPN software, depending on what exactly you're trying to do.

And just as a note, when talking about self-organizing peer-to-peer technology not working within an internal network, such as a campus or corporate setting, it's more than likely going to be a client isolation configuration on the internal network distribution device and not actually related to blocked ports at all. Client isolation is a fairly common security measure for larger networks with many fluid clients. It's a setting that prevents clients from communicating with other clients on the same network. This is even commonly employed by well-respected ISPs for non-business clients, so two neighbors sharing the same ISP/ASN with client isolation set would also not be able to directly communicate with each other. To get around this, there must be an active peer external to the neighbors' ISP/ASN in which they can communicate with, similar to how signals are bounced in satellite communications.

[xek]

I did a little research. So, apart from being more proactive in initializing new connections to Dat peers, this would require implementing a DHT proxy over the established TCP/IP connection. This is because the BitTorrent Kademlia DHT, which is used by Dat, only works on UDP.

@miguelfreitas implemented something like this in Twister https://github.com/miguelfreitas/twister-core/commit/e52f2c25275119bf43054cdf711b91fe16db3238

He wrote a blog entry about it: http://twister.net.co/?p=363

@pfrazee I'm curious how straightforward it would be to extend the Dat protocol that way. I see there is an extensions string in the first handshake, but I didn't find examples on how to use it.

Implementing something like BitTorrent peer exchange (PEX) will also be beneficial.

[leethax666]

I'm pretty sure I2P has a UDP transport and they are more open to p2p networks in general

[pfrazee]

BTW I'm not ignoring this issue, just don't have good responses yet. We're starting to think about how we want the next iteration of discovery to work. This thread will be helpful when we get into it.

If anybody's interested, we've established a proposals process called DEPs for speccing and modifying the Dat protocol, so that might be a good place to discuss this https://github.com/datprotocol/DEPs