Libtorrent: trackers do not work in 1.2.4 with qbittorrent, 1.2.3 was ok

[Safari77]

libtorrent version (or branch):
1.2.4

platform/architecture:
x86_64 Linux Fedora 30

compiler and compiler version:
gcc 9.2.1

When using qbittorrent 4.3.0-git with libtorrent 1.2.4, qbittorrent reports that none of the trackers work.
Switching back to libtorrent 1.2.3 makes everything work just fine.
Unfortunately qbittorrent does not produce any logs.

[mhertz]

I've read this issue reported by 3 others using RC_1_2 branch after 1.2.3 released, and 1.2.3 fixes it like this report states. Deluge and windows btw, and tracker error reported in deluge as: "Error: The system cannot open the device or file specified".

[skuizy]

Looks like the subject discussed in https://github.com/arvidn/libtorrent/pull/4320

[mhertz]

Thanks, but I don't believe so. I saw it reported first on deluge-forum on the 3'rd of february(this month), and was stated by reporter that it worked a few weeks ago with libtorrent 1.2.3, and now after switching to RC_1_2 for first time, then it didn't anymore.

[FranciscoPombal]

Related: https://github.com/qbittorrent/qBittorrent/pull/11999

[Safari77]

FYI, I use wireguard tunnel interface with qbittorrent.

[ghost]

Do you mean they're listed in the Error section? Do the trackers say "working" in the trackers tab?

[gubatron]

We have these same connectivity issues with our jlibtorrent SWIG/Java wrapper as well. Clients won't get connected with or without VPN tunnel interfaces.

We're still testing to see if it has anything to do with the new get_gateway (after removing the get_default_gateway, if this has anything to do when setting ip_interfaces to listen to) or if it has something to do with the removal of upnp_ignore_nonrouters or the removal of lsd_broadcast setting as it's not connecting to local peers either (did multicast ever work for any of you?).

We fire up our session, we get positive alerts binding the interfaces but it's dead, we don't get any DHT peer connections, magnets cannot be fetched, looking at your reports clearly something is broken.

It won't work for us either on Mac (clang), Linux (gcc) nor Android (clang on both arm/x86/x86_64).

Will report more as I know more.

[mhertz]

An0n66, on deluge then besides listed under error heading, then tracker status is: Error: The system cannot open the device or file specified", so seemingly not working.

I don't believe the 3 reporting it was using VPN, as never mentioned that, and is a older issue than the VPN recent change.

Still could be the qbittorrent issue of one tracker or more in same tier, making all fail, I dunno, but as said 1.2.3 worked, but not 1.2.4 and some of the RC branch before that.

It doesn't happen always though, as I don't see it in quick testing of Linux ISO downloads.

[arvidn]

could anyone provide any more details than "trackers don't work" please?
What is the error message for instance?
Could anyone provide any (minimal) steps to reproduce this?

[linuxkernelhack]

magnet fetching doesn't work anymore, I now see 0 DHT peers.
My code hasn't changed, except from going from 1.2.3 to 1.2.4.

No error messages, I do get "INFO: Listen succeeded on endpoint: fe80::4..." alerts on all interfaces, but it doesn't seem to be working anymore. Had to revert back to 1.2.3.

Something has broken.

[arvidn]

Are you using a VPN or a proxy?

[gubatron]

for us (frostwire) the problem happens regardless.

Will be testing more tonight

[gubatron]

we had excluded the new get_getway function, managed to map a version of it our swig wrapper could pick up and it seems to be working now without issues (on and off VPN)

(perhaps excluding it was making some internal use of it break if it couldn't find the symbol)

[arvidn]

Is it just DHT that doesn’t work, or are UDP and HTTP trackers also affected?

When trackers fail, there’s an alert with an error message, could anyone post that message?

If it’s just DHT, the ticket should be renamed.

I have not managed to reproduce this yet, but if you enable dht notification alerts, you can tell a lot about what the dht node be a doing.

[arvidn]

I have a suspicion of what’s going on. It’s probably the last patch to get_gateway(). I think that part of the condition should be removed. I’ll post a patch in a day or so.

[arvidn]

If anyone that has this problem could post the output from the enum_if test tool, it would be really helpful. Alternatively, the output from ifconfig and netstat -r -n

[arvidn]

Here's my theory of what's going on: https://github.com/arvidn/libtorrent/pull/4331
(I'm still interested in the output from those commands if anyone is able to post that)

[Safari77]

enum_if lists the wireguard device's addresses (10.5.5.5 and fc00:bbbb::3) as local.
It also doesn't find out the correct gateway for route 0.0.0.0 (lists emp42s0 instead of wg).

But gateways are ok.

# ip -4 r g 2.2.2.2
2.2.2.2 dev wg table 51820 src 10.5.5.5 uid 0 
    cache 

# ip -6 r g 2020::1
2020::1 from :: dev wg table 51820 src fc00:bbbb::3 metric 1024 pref medium

[arvidn]

I suspected there's an issue in get_gateway(). The patch above addresses my suspicion. The get_gateway() function depends on both the routing table and the interfaces.
Specifically which masks they have and which devices they are attached to. Without the full output it's not possible for me to understand what's causing it to fail.

[Safari77]

enum_if does not always add whitespace after ipv6 address in the output..

=========== Routes ===========
destination       network           gateway                            mtu    source-hint       interface
0.0.0.0           0.0.0.0           192.168.178.1                      1500   -                 enp42s0
192.168.178.0     255.255.255.0     -                                  1500   192.168.178.57    enp42s0
192.168.178.1     255.255.255.255   -                                  1500   -                 enp42s0
10.5.5.5          255.255.255.255   -                                  1420   10.5.5.5          wg
127.0.0.0         255.255.255.255   -                                  65536  127.0.0.1         lo
127.0.0.0         255.0.0.0         -                                  65536  127.0.0.1         lo
127.0.0.1         255.255.255.255   -                                  65536  127.0.0.1         lo
127.255.255.255   255.255.255.255   -                                  65536  127.0.0.1         lo
192.168.178.0     255.255.255.255   -                                  1500   192.168.178.57    enp42s0
192.168.178.57    255.255.255.255   -                                  1500   192.168.178.57    enp42s0
192.168.178.255   255.255.255.255   -                                  1500   192.168.178.57    enp42s0
::1               ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff-                                  65536  -                 lo
::                ffff:ffff:ffff:ffff:ffff:ffff::-                                  65536  -                 lo
::ffff:0.0.0.0    ffff:ffff:ffff:ffff:ffff:ffff::-                                  65536  -                 lo
2002:a00::        ffff:ff00::       -                                  65536  -                 lo
2002:7f00::       ffff:ff00::       -                                  65536  -                 lo
2002:a9fe::       ffff:ffff::       -                                  65536  -                 lo
2002:ac10::       ffff:fff0::       -                                  65536  -                 lo
2002:c0a8::       ffff:ffff::       -                                  65536  -                 lo
2002:e000::       ffff:e000::       -                                  65536  -                 lo
2003:df:3a99:4040::ffff:ffff:ffff:ffff::-                                  1500   -                 enp42s0
2003:df:3a99:4040::ffff:ffff:ffff:ff00::fe80::c225:6ff:9411:e451%enp42s0   1500   -                 enp42s0
3ffe:ffff::       ffff:ffff::       -                                  65536  -                 lo
fc00:bbbb:bbbb:bb01::3:4ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff-                                  1420   -                 wg
fe80::            ffff:ffff:ffff:ffff::-                                  1500   -                 enp42s0
::                ::                fe80::c225:6ff:9411:e451%enp42s0   1500   -                 enp42s0
::1               ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff-                                  65536  -                 lo
2003:df:3a99:4040::ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff-                                  1500   -                 enp42s0
2003:df:3a99:4040:1111:2222:3333:4444ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff-                                  1500   -                 enp42s0
fc00:bbbb:bbbb:bb01::3:4ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff-                                  1420   -                 wg
fe80::            ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff-                                  1500   -                 enp42s0
fe80::4444:3333:2222:1111ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff-                                  1500   -                 enp42s0
ff00::            ff00::            -                                  1500   -                 enp42s0
ff00::            ff00::            -                                  1420   -                 wg
========= Interfaces =========
address                           netmask                                      name                flags               gateway                           description
127.0.0.1                         255.0.0.0                                    lo                  local loopback            -                                  
192.168.178.57                    255.255.255.0                                enp42s0             local                     192.168.178.1                      
10.5.5.5                          255.255.255.255                              wg         local                     -                                  
::1                               ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff      lo                  local loopback            -                                  
2003:df:3a99:4040:1111:2222:3333:4444ffff:ffff:ffff:ffff::                        enp42s0                                 fe80::c225:6ff:9411:e451%enp42s0   
fe80::4444:3333:2222:1111%enp42s0 ffff:ffff:ffff:ffff::                        enp42s0             local                     -                                  
fc00:bbbb:bbbb:bb01::3:4          ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff      wg         local                     -                                  

[mhertz]

Is it just DHT that doesn’t work, or are UDP and HTTP trackers also affected?

When trackers fail, there’s an alert with an error message, could anyone post that message?

If it’s just DHT, the ticket should be renamed.

I have not managed to reproduce this yet, but if you enable dht notification alerts, you can tell a lot about what the dht node be a doing.

I cannot reproduce neither, but as I stated, it's been reported by three people recently on deluge forum in a thread about new deluge(which uses lt-RC-1_2 branch), and not just DHT(the reports where solely about trackers, not DHT).

I asked on the forum for the affected people please running debug-logs so I could foreward to you, but nothing yet. I also just built and posted the get_gateway branch on the forum and asked for feedback from affected, also to foreward to you, so we'll see.

Anyway, the error message of failing trackers in deluge(which i'd presume comes from libtorrent, though not fully sure, could be from deluge), was: Tracker status: Error: The system cannot open the device or file specified". Reports of failing trackers posted firstly 3'rd february, so bad commit should be older than that. 1.2.3 and RC-1_1 not affected.

Sorry for repeating myself, but I reposted because of your questions.

Thank you.

[arvidn]

@Safari77 thanks! as far as I can tell things should work. However, if you bind to 10.5.5.5 (or wg) there is no gateway for this network. I may not fully understand how the routing algorithm works, but as far as I understand, the only IP you can send a message to from 10.5.5.5 is 10.5.5.5. That's the only route over the wg device (on IPv4).

I'll see if I can test wireguard myself to see if I can figure anything out that can be done. I'm open to suggestions, but right now I think you just have to leave the listen_interfaces as the default.

[Safari77]

wg does:

wg set wgdevice fwmark 51820
ip -4 route add 0.0.0.0/0 dev wgdevice table 51820
ip -4 rule add not fwmark 51820 table 51820
ip -4 rule add table main suppress_prefixlength 0
...
iptables -I OUTPUT ! -o wgdevice -m mark ! --mark $(wg show wgdevice fwmark) -m addrtype ! --dst-type LOCAL -j REJECT

(same for ipv6)
With rule add table main suppress_prefixlength 0 the /0 route from main routing table is ignored.

[rwasef1830]

I'm seeing this issue as well with a self-compiled qBittorrent 4.2.1 with RC_1_2 branch, running on a SeedHost shared slot. qBittorrent doesn't produce any logs. No VPN in use. But the host server seems to be using Docker and also is running an OpenVPN server and has multiple IPv4 and IPv6 addresses configured.

[arvidn]

@Safari77 unfortunately I don't know what those commands do. But in the routing table you posted I don't see a default route for the wg device. Do you see it if you use netstat -r?

@Safari77 since this is your ticket, could you clarify whether this affects all trackers (HTTP and UDP) or if it only affects DHT. If my suspicion is correct, it would only affect UDP trackers and DHT.

Anyone else posting to this ticket, please make sure to only post if you have new information, to keep the noise down. I would be interested in knowing if anyone has tried the PR I posted earlier:

https://github.com/arvidn/libtorrent/pull/4331

If someone could enable error messages to be logged, in some way, those would be great as well.

[arvidn]

@rwasef1830 could you try rebuilding libtorrent with #4331 applied, and report back?

[Safari77]

netstat is mostly obsolete, and it doesn't show the correct routing table.
I didn't test if http trackers work.

4331 applied to 1.2.4 didn't fix the issue, and with the patch applied qbittorrent strace shows:

15:15:36.283506 bind(36, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("10.5.5.5")}, 16) = 0 <0.000006>
15:15:36.283532 bind(36, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("10.5.5.5")}, 16) = -1 EINVAL (Invalid argument) <0.000005>
15:15:36.283561 close(36) = 0 <0.000007>

[rwasef1830]

@arvidn The patch doesn't fix the issue. Same exact symptoms.

[arvidn]

@rwasef1830 Thanks for testing! and libtorrent-1.2.3 worked for you? this is a regression?

[rwasef1830]

@arvidn Yes libtorrent-1.2.3 works correctly in regards to tracker announcements. Yes this is a regression. Note: The trackers are all regular http / https trackers in all my tests.

[arvidn]

ok. thanks! I could really do with more precise error messages. So far I've seen:

Tracker status: Error: The system cannot open the device or file specified

Is that the error everyone experiencing this issue see?

[arvidn]

@rwasef1830 is there any chance you could post the error messages you see when contacting the trackers?
I suspect the failure is related to the exotic network setup. I would like to understand more about it. Could you run enum_if on the machine you run the client on?
or netstat -r and ifconfig.

[rwasef1830]

@arvidn qBittorrent does not produce any relevant error messages, even when compiled with --enable-debug. In qBittorrent trackers status, it just says Not Working and doing Force Reannounce does nothing. There 0 peers and 0 seeds, though I have DHT disabled since this is a seedbox. And no traffic at all.

I don't have root access on this box. However, to avoid violating my seedbox provider terms of use, can you provide a means to contact you privately with the interface config information ?

How do I get an enum_if binary ?

[arvidn]

[email protected]

[rwasef1830]

@arvidn Please check your mail for a message from me.

[ghost]

netstat is mostly obsolete, and it doesn't show the correct routing table.
I didn't test if http trackers work.

4331 applied to 1.2.4 didn't fix the issue, and with the patch applied qbittorrent strace shows:

15:15:36.283506 bind(36, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("10.5.5.5")}, 16) = 0 <0.000006>
15:15:36.283532 bind(36, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("10.5.5.5")}, 16) = -1 EINVAL (Invalid argument) <0.000005>
15:15:36.283561 close(36) = 0 <0.000007>

Are you using port randomization on startup?

[Safari77]

Are you using port randomization on startup?

No port randomization, but I specify one port to listen on.

[ghost]

But from your log it looks like its trying to bind to port 0(random)?

[Safari77]

After bind one can do connect also...
But if I specify to listen on port 5555 and use interface wg, qbittorrent manages to listen only on 127.0.0.1:5555 .

[arvidn]

@an0n666 yeah, I think those calls to bind are for outgoing connections. I haven't been able to figure out under what circumstance bind() is called twice though. I imagine that's the problem causing the EINVAL error.

[arvidn]

I've received logs from @rwasef1830 which strongly suggests the problem being that libtorrent fail to enumerate the routing table. It's empty.

I don't think I will land https://github.com/arvidn/libtorrent/pull/4331 I think that was a red herring.

I intend a two-pronged approach:

1. add a special case when listening to a single specific IP, and just make that avoid all complicated routing table logic. This should fix the issue for most people, or make it possible to work-around by specifying the IP to listen on.

2. include more debugging in enum_if to diagnose exactly what the system calls return and try to understand how the current code enumerating the routing table is broken. This will support more future edge cases when you're actually multi-homed.

[Safari77]

I'll see if I can test wireguard myself to see if I can figure anything out that can be done. I'm open to suggestions, but right now I think you just have to leave the listen_interfaces as the default.

If I don't specify listen interface (1.2.4), qbittorrent listens on all the IP addresses but doesn't manage to send to trackers the /announce udp packet. It doesn't need to figure out any routing tables, kernel does it for you.

[arvidn]

If I don't specify listen interface (1.2.4), qbittorrent listens on all the IP addresses but doesn't manage to send to trackers the /announce udp packet.

Is it saying "no route to host"?

It doesn't need to figure out any routing tables, kernel does it for you.

If you bind a socket to "127.0.0.1", the kernel will still allow that socket to send a packet to an address on the internet. That's fundamentally the problem that requires reproducing the routing logic at the application level.

[Safari77]

If I don't specify listen interface (1.2.4), qbittorrent listens on all the IP addresses but doesn't manage to send to trackers the /announce udp packet.

Is it saying "no route to host"?

Didn't say anything... but now that I think about it, it could be sending using the wrong source IP address (192.168.x.x instead of 10.x.x.x ), so kernel or VPN provider might drop them. Anyways tcpdump capture showed it did send udp packets to tracker but they didn't have the /announce part.

It doesn't need to figure out any routing tables, kernel does it for you.

If you bind a socket to "127.0.0.1", the kernel will still allow that socket to send a packet to an address on the internet. That's fundamentally the problem that requires reproducing the routing logic at the application level.

I thought it gets error EADDRNOTAVAIL on bind() if source IP does not exist, and EINVAL on connect() if it's otherwire invalid (e.g. using 127.0.0.1 to connect to 24.24.24.24).

[arvidn]

I thought it gets error EADDRNOTAVAIL on bind() if source IP does not exist, and EINVAL on connect() if it's otherwire invalid (e.g. using 127.0.0.1 to connect to 24.24.24.24).

127.0.0.1 always exists though, as loopback. To expand a bit, this is all to support multi-homed hosts, where you may have multiple IP addresses that can reach the internet, as well as IP addresses that cannot reach the internet (such as 127.0.0.1, ::1 or any other internal network).

The point of looking at the routing table is to know which listen addresses should be announced to the tracker. If the tracker runs on your internal network for instance, only that one IP should be announced. A tracker may also resolve to multiple IP addresses, of different address families. If you have both an IPv6 and IPv4 address, all local IP addresses that can reach its IPv6 should announce as well as the ones that can reach its IPv4 should announce.

[Safari77]

In the more common use case the tracker has a public IP address and clients can send it 0.0.0.0; tracker knows from which IP address the connection came from. I have been trying to open a bug report about qbittorrent sending 10.x.x.x and 192.168.x.x addresses (when anonymous_mode is enabled) to these trackers but github is having some trouble now.

[arvidn]

that is exactly the problem 1.2.4 was trying to address, by figuring out which one of those addresses were connected to the internet.

[ghost]

Lets say I'm connected via both IPv4 & v6. If I'm announcing through both IPs, won't the tracker treat those as a different peer? And in such a case, my statistics will get recorded twice due to announcing the same information from two different addresses?
Most private trackers will not accept such behavior.
Maybe I understood something wrong..please clarify.

The announces should be made from one working endpoint only. If let's say IPv4 succeeds, no need to announce same info from v6. The change should be made in tracker end, where it can accept multiple IP parameters in a single announce request so any client can pass multiple IPs in the GET request.

[Safari77]

Can't the code be borrowed from iproute2 iproute_get() or something? There is also matching of uid etc if you want to get fancy...

[arvidn]

I think the problem is that I filter all routing tables except RT_TABLE_LOCAL and RT_TABLE_MAIN.

[arvidn]

Here's a patch I would appreciate if people would test: https://github.com/arvidn/libtorrent/pull/4339

[Safari77]

I tried the patch with 1.2.4 release. UDP trackers work, thanks a lot! I configured qbittorrent to listen on the VPN device and bind to All IPv4 addresses. However, for 1.2.5 I would very much like to have the anonymous_mode working—now it sends 10.x.x.x and 192.168.x.x addresses to UDP tracker in the /announce packets. Or is it already fixed in RC_1_2 branch?

[arvidn]

@Safari77 thanks for testing! Would you mind filing a separate issue for the anonymous_mode issue? I don't quite understand what the problem is, but I think it's separate from this ticket. Specifically, if you listen to a single IP address, a socket bound to that address will be used to connect to the tracker. It sounds like you see two announces to the tracker, is that right? Are both of those networks (192.168.x.x and 10.x.x.x) connected to the internet? (please answer in a new ticket)

[Safari77]

Even though the trackers are marked as working in qbittorrent (when using the patch at #4339),
qbittorrent is slow to connect to new peers and I see the double bind() error as noted in the earlier comment of mine.

[xnoreq]

I have the same issue.

Last version I've built and worked, before upgrading to 1.2.4, was af12f5d6b8dc47add564d7ce544bbaf26cd1de1e.

With 1.2.4 qBittorrent appears as if it was offline. No DHT nodes are discovered, no trackers are contacted ...

Here's the log with af12f5d6b8dc47add564d7ce544bbaf26cd1de1e:

qBittorrent v4.3.0alpha1 started
...
Trying to listen on: 0.0.0.0:8382,[::]:8382
...
Successfully listening on IP: 0.0.0.0, port: TCP/8382
Successfully listening on IP: 0.0.0.0, port: UDP/8382

However with 1.2.4 I get see this instead:

Successfully listening on IP: 127.0.0.1, port: TCP/8382
Successfully listening on IP: 127.0.0.1, port: UDP/8382
Successfully listening on IP: 192.168.101.2, port: TCP/8382
Successfully listening on IP: 192.168.101.2, port: UDP/8382

The process runs in a container with 3 network interfaces:

• loopback (127.0.0.1)
• eth0 (192.168.101.2)
• tun0 vpn tunnel (10.8.8.x)

There are a bunch of network routes:

• default via 10.8.8.1 dev tun0 (the vpn gateway)
• 10.8.8.0/24 dev tun0
• 192.168.101.0/30 dev eth0
• other static routes to private networks via 192.168.101.1

[xnoreq]

With 1.2.4, configuring and limiting the libtorrent network interface to tun0 makes it bind to 10.8.8.x and it can communicate through the VPN again.

However, and I'm not sure why this has happened, after setting it back to "any interface" in qBittorrent it now binds to all three IPs with 1.2.4: lo, eth0 and tun0.

I now have in my qBittorrent.conf:

[Preferences]
Connection\Interface=
Connection\InterfaceName=

Before it listened on 0.0.0.0, now on all interface IPs... which is fine.

[xnoreq]

As for doing routing in the application: this is doomed to fail.

On Linux you can do very sophisticated routing. For example, you could have multiple default routes in different routing tables and make decisions dynamically for each packet.
I know of VPN providers that have "features" which are implemented that way.

I also have to revise my previous comments:
It only worked because I simplified the routing table.
Again, some VPN provides "overrule" the default route by adding more specific routes that still cover the whole IPv4 range:

• 0.0.0.0/1 via 10.8.8.1 dev tun0
• 128.0.0.0/1 via 10.8.8.1 dev tun0

Given those routes, libtorrent 1.2.4 breaks connectivity again.

[arvidn]

I'm open to suggestions of how to determine which IP addresses to bind to and perform outgoing announces from

[rwasef1830]

I saw a technique done in some code before, where you create an unbound UDP socket, and send a single packet to any random IP (to make sure you enter the route chosen by the OS for the torrent client) and then checking the local IP of the socket assigned by the OS. I've done this before, it works.
(Intentionally using UDP to avoid establishing a TCP 3-way handshake and requiring a response before the code unblocks from the connect).

This way it will always work. It will pick a random one and stick with it in case of having multiple default routes, and when the user explicitly chooses one, you use that.

@arvidn

In case of internet connectivity change or loss event from OS, the process should be repeated unless an explicit IP was chosen by the user.

[arvidn]

That would tell you one network that can route to the up you picked. You would have to do the same thing for IPv6, but if you have two separate paths to the internet, you would only find one of them, right?

Or if you run a swarm and tracker on a local network, you might announce your external address instead of the internal one.

[rwasef1830]

The desired behavior for a bittorrent client and a bittorrent tracker is different.

In case of client, you want only 1 address and that's the external one (there's actually a default route for IPv4 and another on IPv6) so you actually need to do the UDP thing twice, once to a IPv4 and another to a IPv6 and use the correct one for each connection type.

In case of tracker, if no explicit bind IP is specified, the correct behavior in my opinion is to query all interface IPs and announce / bind on all of them. Or actually just bind 0.0.0.0 and ::

[arvidn]

I'm just talking about client behavior. And I would like to support hosts with multiple external addresses (i.e. multi-homed). In those cases I believe you may have multiple default routes, to different gateways, one for each path to the internet.

Probably the most common case is having an IPv4 and one IPv6 connection, but you can also have two IPv4 connections. e.g. one wired and one over wifi.

[rwasef1830]

Then in this case, you can try the following, loop over interfaces, for each address on each one bind a socket, and send a udp packet to a random ip. If you get a route error, then this interface is not a path to the internet, if you don't get any error, then read the local IP and add it to the list.

In the end you will end up with a list of local addresses for which there is a path to the internet.

[xnoreq]

I'm open to suggestions of how to determine which IP addresses to bind to and perform outgoing announces from

Why bind in the first place if the user hasn't configured any interface or IP to bind to?
That way you'd let the OS do it's routing job.

When binding you have to make sure to end up with a single, unambiguous socket.
With multiple bound sockets, theoretically you'd have to send the same packet over each of them.

[rwasef1830]

I'm assuming he needs to send to the tracker the client's IP in some header of some sort, there are legit use cases for needing to know the external IP.

[xnoreq]

I'm assuming he needs to send to the tracker the client's IP in some header of some sort, there are legit use cases for needing to know the external IP.

The interface IP is not necessarily the external IP. In fact, on the client side this is standard because of NAT.

Also, why would you need to send your own (external) IP as data to someone who can see the actual external IP the packet came from? Sounds like a design defect in the protocol.

[rwasef1830]

It's useful in some cases when in Bittorrent the connection to the tracker is blocked, so you use a proxy for the tracker communication, and keep the P2P traffic outside the proxy, the tracker needs to know the IP being used for the P2P communication which might not be the same as the source IP of the client-to-tracker connection.

Yes, in such case, you should check the external IP with an external service.

[arvidn]

this is not related to knowing the external IP. It's not sent as part of the payload of announces. The issue is knowing how many paths to the internet exist from the computer. Each of those paths should be announced to the tracker.

or rather, the tracker should be announced to via each of those paths.

[xnoreq]

@arvidn As I mentioned before, as soon as you have more than one socket you'd have to send the same packet through each of one of them.

But again, this defeats the purpose of the OS doing the work which e.g. if connected through both ethernet and wifi to the same network would not duplicate packets but choose the "better" interface (based on some metric like connection speed, so it would prefer ethernet).
The OS or network management software would also automatically switch to wifi if ethernet was disconnected.

I still don't get why you would want to re-implement routing inside libtorrent.

[arvidn]

I still don't get why you would want to re-implement routing inside libtorrent.

It's not really reimplementing routing. It's looking at the routing table to determine which local IP addresses ought to be used to announce to the trackers (and the DHT). To enable peers to reach you via all of them.

[xnoreq]

It's looking at the routing table to determine which local IP addresses ought to be used to announce to the trackers (and the DHT).

That's literally routing and exactly what happens when you send a packet from an "unbound" socket: the network stack looks through the routing tables (yes, plural) based on the destination address (or not, see below) and picks the most specific route, and therefore interface.
The main difference is that it selects only a single source interface/IP.

Besides reimplementing routing and at the risk of repeating myself, this is doomed to failed anyway because you'd also have to reimplement e.g. policy-based routing (which could make routing decisions on something other than the destination address alone). And afaik all of this is highly OS-specific.

[arvidn]

That's literally routing

Well, that's not what "literally" means :)

his is doomed to failed anyway

I'm open to suggestions. I'm not convinced your earlier suggestion would work. If you bind the socket, you can't use getsockname() to learn whether it found a route or not. And I believe even if you bind to an address whose network doesn't have a gateway, the kernel may still route via another network.

One way to mitigate the issues of enumerating the routes would be to always assume that manually entered IP addresses can reach the internet. This logic would then only apply to "implicit" IP addresses found by expanding 0.0.0.0/:: or specifying a device name.

[xnoreq]

That is actually, without exaggeration, routing. That's literally what I said and meant. :)

If you bind the socket, you can't use getsockname() to learn whether it found a route or not.

That statement makes no sense to me. Did you mean if you do _not_ bind, you can't use getsockname()?

Also, getsockname() just gives you the address the socket is bound to. It tells you nothing about how to route packets save for a few exceptions. So based on this you cannot say whether the kernel has found a route or not anyway.

And I believe even if you bind to an address whose network doesn't have a gateway, the kernel may still route via another network.

Whether a route has a gateway or not is typically irrelevant for the routing decision. In the simplest case of static routing the kernel looks through the routing tables and picks the most specific rule for the destination address.
Whether the rule then specifies a gateway the packet needs to be sent to or not is something that has an effect later on, after having made the routing decision. The gateway is used for physical addressing when packaging your IP packet in e.g. an ethernet frame.

Afaik, by binding a socket to a specific IP you're taking away routing options to begin with. So even if a rule with the matching destination network for another device exists then the packet will still be sent through the device that has the socket's IP assigned to it.

[xnoreq]

A correction regarding the last paragraph:
There are two models, strong and weak. Different OS's use different models.

Basically, in the strong model the source IP is used as a constraint in the routing tables lookup and in the weak model it's not.

Apparently Windows switched to a strong model after XP, similar to what some BSDs and Unixes are doing while Linux is using the weak model by default ... but there are sysctl knobs that can change some behaviors. And there's policy-based routing as mentioned before.

---

So you could have an eth0 with IP address 192.168.0.1 and a socket bound to 192.168.0.1 and when sending a packet with some Internet IP as destination it could still get routed through tun0. How? See https://github.com/arvidn/libtorrent/issues/4325#issuecomment-589698813. These routes for tun0 take precedence over the default route for eth0.

So even if your tun0 has some 10.x.x.x IP address and you're sending through a socket bound to 192.168.0.1 (which is assigned to eth0) you can end up sending that packet through tun0 with a source address of 192.168.0.1.
On the other side of the VPN tunnel such (invalid) packets should get dropped.

Sending one packet from eth0's address and one from tun0's address would end up sending both through tun0.

---

With policy based routing one could create a multi-homed setup where the local source IP address is used to pick the outgoing interface accordingly. If all these local addresses are actually external IP addresses and you want to announce all of them to a tracker then you'd need to send packets from each of these IP addresses.

However, if the source IP addresses were behind a NAT then they could resolve to a common external IP address.

---

Users should only bind to IP addresses if they know exactly what they're doing.

libtorrent: I would not bind to anything by default. The best route will win. This is what most users would expect anyway.

If you want to use the binding feature to "filter" out interfaces from which to ignore torrent traffic then I'd still not bind to anything and just drop received packets based on destination addresses inside libtorrent. (This is similar to what dnsmasq does if you don't explicitly tell it to bind to interfaces.)

Finally, if you really want multiple announcements from multiple IP addresses from the same host then you have to bind and send multiple announcements, one for each IP.
This behavior and the pitfalls have to be documented however.

[xnoreq]

@arvidn Will you release a hotfix for 1.2.4 and revert the socket binding? Because in the release it's broken.

[arvidn]

the changes in 1.2.4 that caused these problems were there to fix (in my mind) worse issues in 1.2.3.
I think I'm close to feel comfortable with releasing 1.2.5 though.

I would appreciate anyone able to test this: https://github.com/arvidn/libtorrent/pull/4339

I've tested it on linux with both privateinternetaccess' OpenVPN and NordVPN's OpenVPN and wireguard, as well as no VPN of course. Both with explicitly binding to a device and leaving the default listen interfaces.

That pull request also contains a description of the new logic. I invite anyone to review it!

[LAP87]

I hope this finally solves it and qbittorrent implements the fix asap

[xnoreq]

@LAP87 qbittorrent just uses libtorrent. Except for the Windows build where libtorrent is included, whether libtorrent is updated or not is up to you or your distro.

[xnoreq]

@arvidn #4339 works for me. Without any interface/IP specified it now binds to all IPs (lo, eth, tun).

There's also three additional udp sockets all bound to 0.0.0.0:6771. There's traffic sent from those sockets to 239.192.152.143:6771. Why?

[FranciscoPombal]

@xnoreq

There's also three additional udp sockets all bound to 0.0.0.0:6771. There's traffic sent from those sockets to 239.192.152.143:6771. Why?

This has been like this for a very long time. It's Bittorrent LSD. See http://www.bittorrent.org/beps/bep_0014.html

TL;DR:

LSD uses the following multicast groups: A) 239.192.152.143:6771 (org-local) and B) [ff15::efc0:988f]:6771 (site-local)

[xnoreq]

Yeah but why are there 3 sockets now bound to 0.0.0.0? It looks like they should have been bound to the same IPs as my interfaces or there should be just one interface that can receive traffic from any interface/IP address, no?

[arvidn]

@xnoreq interesting. Those sockets were supposed to get bound to each individual interface too, by this patch which was part of the 1.2.4 release.

well, technically, they are still bound to the unspecified address, but they are tied to their respective interface, which is a multicast option.

Do you see packets sent over the same interface?

[xnoreq]

Yeah they're just bound to 0.0.0.0:6667.
netstat -gn shows

Interface       RefCnt Group
tun0            3      239.192.152.143

But if I run tcpdump then I can see udp packets being sent on all three interfaces from the respective IP:6667 to to multicast group:6667.

[xnoreq]

@arvidn when joining the multicast group you should also specify the interface to join with.
https://github.com/arvidn/libtorrent/blob/6d5bdbdb1a879910e610b24bc4bc405d2844a994/src/lsd.cpp#L121

For example from linux:

struct ip_mreq
{
        struct in_addr imr_multiaddr;   /* IP multicast address of group */
        struct in_addr imr_interface;   /* local IP address of interface */
};

setsockopt (socket, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq));

In boost you should be able to additionally provide an interface (address) to join_group.

[arvidn]

@xnoreq thanks! I had totally missed this. Fixing for libtorrent-1.2.5

[arvidn]

this ought to do it: https://github.com/arvidn/libtorrent/pull/4378

[xnoreq]

@arvidn Thanks. Haven't tested with multiple clients in the same multicast domain but the network traffic I see seems right.
It's doing the multicast group join on each interface and sends the BT-SEARCH packets on each interface for each torrent with this feature enabled.

[xnoreq]

@arvidn There's another issue that wasn't present with the old versions that didn't bind sockets to IPs:
While qbittorrent is running reconnecting the VPN connection leads to qbitorrent losing connectivity and stalling all torrents. Trackers also turn into "not working".

[arvidn]

Is the expected behaviour that it stops working until the vpn is reconnected, or that it keeps working by using the “plain” network interface?

[xnoreq]

I would expect it to continue using all available interfaces. That's not the issue though.

The issue is that even after the VPN has reconnected, qbittorrent/libtorrent will say that trackers are "not working". Stopping/starting the torrent doesn't change that. Adding a new torrent.. trackers are not woking. Number of DHT nodes also drops to 0 and stays at 0.

qbittorrent does seem to notice what's going on and logs after the reconnection:

(I) 2020-03-04T21:12:42 - Successfully listening on IP: 10.8.8.x, port: TCP/xxxx
(I) 2020-03-04T21:12:42 - Successfully listening on IP: 10.8.8.x, port: UDP/xxxx

And socket stats also show a UDP and TCP socket bound to that tun0 IP address/port after the reconnection.
There does seem to be some connectivity however, as it did reconnect to some peers after the VPN connection was up again.

To fix this I have to completely restart qbittorrent.

[arvidn]

@xnoreq thanks! would you mind giving this patch a try? https://github.com/arvidn/libtorrent/pull/4384

[xnoreq]

@arvidn Doesn't work for me. DHT nodes also stay at 0.

[arvidn]

it works for me, turning the VPN off and back on again. It doesn't recover right away though.
Although, you're right about the DHT. I don't see it recovering

[narvalotech]

I had the same issue using qbittorrent 4.2.1-1 on Manjaro linux and the packaged libtorrent 1.2.4-1.
I am using wireguard setup with networkmanager.
When importing via magnet, the torrents would stay at "Downloading metadata" until I disabled the vpn.
Just wanted to report that with the 1.2_RC branch built using the AUR script, it now works again !

[arvidn]

I believe this is fixed in the 1.2.5 release

[ghost]

libtorrent 1.2.5.

Umm..I think it's not fixed yet. I just installed deluge to test the ongoing VPN issue but even without a VPN my tracker status says: "The system cannot open the device or file specified" when I manually update the trackers. It works fine when I start/stop the torrent. Maybe it's a deluge specific issue since I didn't face it in qBt yet.

[Austin76016]

Can also confirm the above with Deluge. Start/stop also working as a temp remedy.

[arvidn]

@Austin76016 what error message do you get in Deluge?

[Austin76016]

@arvidn Error: The system cannot open the device or file specified
LT 1.2.5

[arvidn]

In Deluge, do you manually enter the interface to bind sockets to? That error message is consistent with the interface being misspelled in the outgoing_interfaces setting. i.e. the setting doesn't match the name of the interface as listed by ifconfig for instance

[Safari77]

I tried 1.2.5 with 9ac4e6eed88c1ea1e6cdb6865889320164ad0b85 (fix double bind()), binding to one specific network device works OK in qbittorrent.