libtorrent 🚀 - libtorrent 1.1.{7,8,9} + Deluge 1.3.15, connecting only to small fraction of peers over dozens and only on certain ...

is it possible that the main reason pausing and resuming solves the problem is because of re-announcing with the tracker?

could you try just re-announcing to the tracker instead of a full pause?

Do you have access to the connect_candidates counter for the torrent? I would expect it to either be 0 or libtorrent to attempt more connections.

arvidn on 7 Aug 2018

is it possible that the main reason pausing and resuming solves the problem is because of re-announcing with the tracker?

Pause/resume fixes the issue, update tracker does not. Unless you want me to try something else?

Do you have access to the connect_candidates counter for the torrent?

How do I access the counter? I have root access and deluge-console is installed, if it helps.

Happy to try anything you want! It is also weird that the issue is not here with 1.0.11.0.

quarreldazzle on 7 Aug 2018

I don't know how Deluge prints torrent states. it's part of the torrent_status type, as connection_candidates. It's the number of peers in the peer list that are eligible to be connected to.

What about the max peer connection limit? is it set low?
libtorrent won't ever exceed the connection count, which means the margin between the connection count limit, and the number of peers you're already connected will limit the rate at which you can try new peers.

arvidn on 7 Aug 2018

try to bump the connection_speed setting. I believe the default value was lowered for libtorrent 1.1.

If you set it to 50 or maybe even 100, I would expect peers to be connected to more quickly. One problem some people have is that too high connect speed causes (low quality) routers to crash.

In Deluge you may need the ltconfig plugin to access this setting.

arvidn on 7 Aug 2018

could you try this patch to see if it improves things? https://github.com/arvidn/libtorrent/pull/3231

arvidn on 8 Aug 2018

I could unfortunately not find any connection_candidates anywhere. I also tried the info command for deluge-console, but there is nothing related.

Regarding max peer connection, it is set to -1 in the settings.

connection_speed in ltconfig is set to 500.

I will try the patch ASAP and report back to you.

quarreldazzle on 8 Aug 2018

Ok @arvidn, I have tried the patch. No luck.

I confirm the following:

1) Announce OK.
2) 2 over 4 peers connected. Uploader was not among them (or not shown).
No download or upload activity for the next 5 minutes or so. I then notice one of the two peers is progressing.
3) update tracker a couple times
Nothing happens
4) pause/resume
5) 18 peers are shown immediately, most have finished downloading. Download starts.

quarreldazzle on 8 Aug 2018

The patch seems to make connecting to the initial swarm faster with the other trackers, however. I would not discharge it :-)

quarreldazzle on 8 Aug 2018

I updated the patch to also affect peers received from peer-exchange. would you mind testing it again? (I made a force-push, so you may have to reset your branch)

arvidn on 8 Aug 2018

Done. Still nothing, unfortunately. Only pause/resume seems to work. Is there anything that comes to your mind that could have introduced it in the last few minor releases in 1.1.x? I cannot tell if the bug was there in <1.1.6, but it is not there in 1.0.x/

I really appreciate how you have been reactive to this issue!

quarreldazzle on 8 Aug 2018

it's been several years since 1.0.x, it would be quite a lot of code to go through

arvidn on 8 Aug 2018

would it be possible to get a log from when this happens? I would be interested in seeing the tracker announce, and the log about how many peers it receives. PEX messages would also be interesting to look at, as I imagine most peers will arrive via PEX if you're really early onto a swarm.

Ideally a log would also include the pause-resume step to demonstrate the case where it works.

arvidn on 9 Aug 2018

Tracker announces and PEX messages may not tell the whole story.
In a public torrent swarm, a LOT of the peers+seeds may be firewalled so until they connect to you (their outgoing connection/an incoming connection from your point-of-view) ...you may never connect to them.

Any problems with qBitTorrent/libtorrent accepting incoming connections will make this worse.

I am seeding a large torrent shared on 2 computers on 2 different internet connections -- 1 using qBT4.1.1 32bit, the other uTorrent.
The uTorrent one connects to 20 peers despite not making outgoing uTP connections, using a large ip blocklist, running in initial seeding (super seeding) mode, and even blocking peers that use certain listening ports.
The qBT one connects to 14 "peers" -- it claims 1 or 2 of those are "seeds" despite not showing any at 100%. It's allowing uTP connections both ways, is port forwarded correctly...but is almost always connected to considerably fewer peers. Most of the peers are incoming connections. I am not using super seeding on qBT. If anything, qBT should be connected to equal or more peers than uTorrent!

Seeker2 on 9 Aug 2018

@arvidn I will attempt to log as much as possible, but I would rather send it privately. Are you ok with you @libtorrent.org e-mail address?

quarreldazzle on 12 Aug 2018

yes, that works great

arvidn on 12 Aug 2018

Done and sent. Object of the e-mail is the title of this issue. Let me know!

quarreldazzle on 12 Aug 2018

this log isn't quite as detailed as I was hoping. Perhaps Deluge doesn't support enabling full logging. However, I made one interesting observation. Grepping for the tracker announces (I redacted the actual URLs):

[DEBUG   ] 15:37:33 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-1>) (200) tracker sent a failure message "unregistered torrent" (0) [unregistered torrent]
[DEBUG   ] 15:37:33 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-2>) (200) tracker sent a failure message "unregistered torrent" (0) [unregistered torrent]
[DEBUG   ] 15:37:33 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-3>) (200) tracker sent a failure message "unregistered torrent" (1) [unregistered torrent]
[DEBUG   ] 15:38:06 torrentmanager:1073 Tracker Error Alert:  -  (<tracker-1>) (200) tracker sent a failure message "unregistered torrent" (1) [unregistered torrent]
[DEBUG   ] 15:38:06 torrentmanager:1073 Tracker Error Alert:  -  (<tracker-3>) (200) tracker sent a failure message "unregistered torrent" (2) [unregistered torrent]
[DEBUG   ] 15:38:06 torrentmanager:1073 Tracker Error Alert:  -  (<tracker-2>) (200) tracker sent a failure message "unregistered torrent" (2) [unregistered torrent]
[DEBUG   ] 15:38:11 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-2>) (200) tracker sent a failure message "unregistered torrent" (1) [unregistered torrent]
[DEBUG   ] 15:38:11 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-1>) (200) tracker sent a failure message "unregistered torrent" (1) [unregistered torrent]
[DEBUG   ] 15:38:11 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-3>) (200) tracker sent a failure message "unregistered torrent" (2) [unregistered torrent]
[DEBUG   ] 15:38:18 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-1>) (200) tracker sent a failure message "unregistered torrent" (2) [unregistered torrent]
[DEBUG   ] 15:38:18 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-2>) (200) tracker sent a failure message "unregistered torrent" (2) [unregistered torrent]
[DEBUG   ] 15:38:18 torrentmanager:1073 Tracker Error Alert: TORRENT_FOR_TESTING (<tracker-3>) (200) tracker sent a failure message "unregistered torrent" (3) [unregistered torrent]

[DEBUG   ] 15:38:34 torrentmanager:1030 on_alert_tracker_reply: TORRENT_FOR_TESTING (<tracker-1>) received peers: 0
[DEBUG   ] 15:38:34 torrentmanager:1030 on_alert_tracker_reply: TORRENT_FOR_TESTING (<tracker-2>) received peers: 7
[DEBUG   ] 15:38:34 torrentmanager:1030 on_alert_tracker_reply: TORRENT_FOR_TESTING (<tracker-3>) received peers: 0

[DEBUG   ] 15:43:14 core:403 Pausing
[DEBUG   ] 15:43:14 torrentmanager:1030 on_alert_tracker_reply: TORRENT_FOR_TESTING (<tracker-3>) received peers: 0

[DEBUG   ] 15:43:15 core:439 Resuming
[DEBUG   ] 15:43:15 torrentmanager:1030 on_alert_tracker_reply: TORRENT_FOR_TESTING (<tracker-3>) received peers: 90

[DEBUG   ] 15:52:47 torrentmanager:952 on_alert_torrent_finished
[DEBUG   ] 15:52:47 torrentmanager:1030 on_alert_tracker_reply: TORRENT_FOR_TESTING (<tracker-3>) received peers: 68

There are 3 trackers, we see that all 3 keep failing with "unregistered torrent" the first few announces (as we expect). Then at 15:38:34, tracker-2 returns 7 peers, the two other ones return 0. Presumably this causes the re-announce script to stop, since we got some peers. However, 7 peers aren't that many. A few minutes later, at 15:43:14, the torrent is stopped. This time we announce to the trackers with event=stopped and (as we would expect) we don't get any peers back.

A second later the torrent is resumed, at 15:43:15. This time we get lots of peers.

At 15:52:47 the torrent finishes and we announce with event=finished and, again, we get a bunch of peers.

From what I gather, this is still fundamentally the same problem as https://github.com/arvidn/libtorrent/issues/315 just with the twist that there are multiple trackers, and the one with very few peers happens to become responsive first.

I believe this also explains why it doesn't always happen. If would be the first one to return peers, presumably everything would be fine.

arvidn on 13 Aug 2018

however, it does not explain why simply re-announcing to the trackers wouldn't help. You didn't do that in this session, right?

arvidn on 13 Aug 2018

You are right, I didn't trigger any re-announce after the first "announce: OK.". I will repeat the test, perhaps with the other tracker I suspect is also having the same issue. That one has only one domain, though.

quarreldazzle on 13 Aug 2018

As promised, I sent you another Deluge log with the same tracker as before.

This time I made sure I updated the tracker several times before using the pause/resume thing. Please note I am not using any script for the announce issue here. I am doing everything manually to keep things polished.

Once again, there were not enough peers up to the pause/resume, and the uploader was not connected.

11:02 Torrent snatched, various update tracker up to announce OK
11:03 Announce OK, 0(0) Seeders, 1(2) Peers, no download, no upload
11:04 update tracker, nothing happens
11:05 update tracker, nothing happens
11:06 update tracker, nothing happens
11:07 update tracker, nothing happens
11:07 pause/resume, 0(1) Seeders, 28(35) Peers, download starts
11:09 seeding

With this tracker, it is almost never possible to get connected to the uploader, besides of the issue of not enough peers connected from the start. Both these issues are not present with the 1.0 branch.

quarreldazzle on 14 Aug 2018

Are these 28(35) some/mostly/entirely incoming peers? (meaning -- is qBT acting like it's firewalled?)
Also are they all uTP or regular TCP peer connections?

The reason I ask is after all those tracker updates, the tracker should be giving out your ip:port address to other peers so they can connect to you (well, your qBT ip:port) even if the tracker doesn't give you a peer/seed ip:port list.

But I guess you'd need to wait 15-60 minutes to rule out peers just being slow to respond, since they'd have to do a tracker update after your first or the tracker would not register your ip as on the torrent to hand out to other peers yet.

Seeker2 on 14 Aug 2018

@Seeker2, I am sorry, I do not know how qBT work. This issue I am having is with Deluge. Anyway, 28(35) means that Deluge is connecting to 28 over 35 available peers.

15-60 minutes is usually too late in the racing situations I am describing. Also, this issue is not present in the 1.0 branch of libtorrent. I have recently sent another log to @arvidn where the issue is present despite repeated announce commands I am sending during the race.

quarreldazzle on 16 Aug 2018

The issues I brought up should be identical with Deluge, as these are common features of any BitTorrent client.

Private trackers (which block PEX, LSD, and DHT) can make these problems a lot worse.

Seeker2 on 18 Aug 2018

I will look into what the distinction may be between the first announce and subsequent ones. would it be possible to enable more detailed logging in deluge? I'm not familiar with the details of how Deluge logs, but you may need ltconfig plugin to set the alert_mask to include all kinds of alerts, but you may have to do something else to have deluge actually print all libtorrent alerts to the deluge log as well. @cas-- is there a way to get more logging?

Alternatively, you could capture the tracker traffic in wireshark, but if the tracker is running over HTTPS, that might be tricky as well.

It may be that the tracker penalises early re-announces by not returning any more peers (and event=started is what makes it return peers). There could also be an issue in libtorrent where it accidentally sends num_want=0 or something.

I will experiment locally, but I don't have very high hopes of being able to reproduce this

arvidn on 18 Aug 2018

actually, it looks like all the force-reannounce you issue are scheduled, but never executed. This is most likely because the tracker responded with a min-interval that's longer than the time from the first response to you pausing and stopping the torrent.

Is it possible that perhaps libtorrent-1.0 did not correctly honor min-interval?

arvidn on 18 Aug 2018

@quarreldazzle would you mind testing this patch? https://github.com/arvidn/libtorrent/pull/3250
it adds an option to ignore min-interval from trackers, and enables it by default (in the python binding, to make it easier to test).

arvidn on 19 Aug 2018

I feel pretty confident that patch at least solves one problem causing this issue (if not resolving the whole issue). @quarreldazzle when you have a moment, please test it! I would like to have some confirmation before I land it and declare victory.

arvidn on 22 Aug 2018

@arvidn I will try it by the end of the week, thanks!

quarreldazzle on 22 Aug 2018

@arvidn Why would you add on purpose an option to ignore something that the tracker told the client? If the tracker gives a min interval property in the response then that is there for a reason. If that interval is set to e.g. 5 minutes, then clicking like crazy on "force re-announce" in the client a thousand times in a couple of seconds and the client not sending all those requests to the tracker is the expected/proper behavior. Having the option to ignore on purpose the min interval seems like the opposite of an improvement to me. If there was a proper reason for this I'd understand, but it seems to me that the only reason for this is ratio whoring on some private trackers because they get only a fraction of the peers on the first announce (which is actually worth looking into why does that happen if there are obviously a lot more peers there). Not honoring the min interval (instead of looking into that issue) doesn't feel like a proper fix at all, it just seems to conveniently avoid the problem at hand.

X-Coder264 on 22 Aug 2018

Tracker updates only hand out 50-200 (or fewer!) "random" ip addresses.
If private trackers aren't set up properly, they'll be handing out seed ips to seeds.
If there's 500 seeds and 5 peers, the odds of any peer ips even being in the list handed out isn't very good.
This is also much of the reason people use high half-open connection limits because their seeds have to try 100's of seed ips to find 1 peer ip.

Seeker2 on 23 Aug 2018

@X-Coder264

If the tracker gives a min interval property in the response then that is there for a reason

I don't think that's necessarily a safe assumption. For instance, in this case, where the tracker only hands back yourself, clearly it makes sense to announce again sooner, once there are more peers. The tracker could know this simply by the age of the torrent (if it was created a few seconds ago, and it's handing out few peers, it could have a lower announce-interval).

All this requires code, testing and tweaking. I don't have a lot of insight into tracker development (unfortunately), but I don't get the impression performance optimisations are very high on their lists. As @Seeker2 points out, ideally trackers would not hand out seeds to other seeds either, but they do.

If that interval is set to e.g. 5 minutes, then clicking like crazy on "force re-announce" in the client a thousand times in a couple of seconds and the client not sending all those requests to the tracker is the expected/proper behavior.

One could argue that if the tracker would exhibit "proper" behaviour, client's wouldn't have to violate the min-interval. For example, the tracker could set the re-announce interval to 5 seconds if the torrent is < 1 minute old and there are less than 5 peers. Or something along those lines. Perhaps the announce interval could be proportional to the number of peers and the age. But now trackers seem to expect all "smarts" to be in the clients, which isn't necessarily bad.

they get only a fraction of the peers on the first announce (which is actually worth looking into why does that happen if there are obviously a lot more peers there)

The first peer to announce will always get 0 peers in the response (or possibly itself). The peer that polls the tracker with the highest frequency will almost always be the first to announce, but even if you're the second or third to announce, that may be just as bad. If the initial seed is the 20th to announce, all the first 19 peers will just sit there idle (at least as long as they can't accept incoming connections).

With the current logic, of not being allowed to re-announce for 5 minutes, you get penalised for announcing early. The optimal strategy then is to predict when the initial seed may announce, and get in right after that. Clearly that's very difficult, and there isn't really a good technical reason to design the system like that.

With this patch (but I'll undo the part where the flag is set by default in the python binding). I'm hoping clients can be smarter about when to honour the min interval and when not to. And maybe even when to ignore the regular interval and re-announce automatically. iirc, uTorrent has had logic to do this for a very long time, if it doesn't get enough peers, it will try to re-announce again 5 or 10 seconds later (but not indefinitely).

arvidn on 23 Aug 2018

Not sure how much of an input this is, but I have mentioned it before - There's certainly something "odd" going on with how the 1.1.X branch connects to peers in general..

From my testings, it connects to about half of what 1.0.11 would connect to, just tested it now with 1.1.5 branch directly from ubuntu to avoid me messing something up with a compile.

Start the torrent on deluge 1.3.15 with 1.1.5, the torrent has 150 peers with 1.1.5 it will connect to 75-85 of those - Close deluge and install 1.0.11 and it will connect to 130-140 peers, all of that within seconds.

Im not entirely sure where this comes from, I have tried disabling UTP entirely, and I have tried basically all options from ltconfig that might have an influence on this, so im totally blank on why this is happening, I can see though that on 1.1.X it makes no difference if you right click on press "Update tracker", in deluge it will show the peers in ( ) next to it, which shows how many peers the tracker sends back, once a torrent it started, it will never "update" that before you stop/start the torrent, after that it shows the correct peer amount but it then takes ages to actually connect to those peers which is totally different from 1.0.11..

I have not tested the patch, as for some odd reason I cannot compile libtorrent on my system, it just keeps failing even without the patch applied.

khnielsen on 23 Aug 2018

Yup @khnielsen, same issue. And the worst part of it is that libtorrent 1.1.X often misses connecting to the uploader, making you a "satellite" peer. Will test this out ASAP.

quarreldazzle on 23 Aug 2018

@khnielsen is there a reason to believe that this is an issue of connecting and handshaking with peers (or accepting incoming connections/handshakes), rather than an issue of actually getting the peer from the tracker?

so far I've only seen logs indicating the latter. It doesn't preclude that there are two problems though. I would really appreciate it if you would want to capture logs from 1.1.x and 1.0.x (i.e. a failure and a success) for me to look at! (the more verbose logging the better, please set all bits in the alert_mask)

arvidn on 23 Aug 2018

I cannot say if it is either one, but there's definately a difference between what it collects and what it ends up being connected to, the more peers in the swarm the larger the difference between the two.

I will try and reproduce this on my test bench and see if I cannot find something for you.

khnielsen on 23 Aug 2018

Further testings done here, I believe that what im seeing might be related to what @quarreldazzle is seeing aswell.
Stopping a torrent once it is running will make 1.1.5 connect to more peers after a short period of time, it as if it simply stalls with tracker communication after the first announce, and only changing the status of the torrent will result it changing.
I spotted a torrent, that were running with 35 peers, all of them about the same way into the torrent as me, deluge showed 35(36) so only one peer not connected to, as soon as it shifted into seeding status it ramped up and connected to 78 peers and the status changed to 78(102), right clicking and pressing "force update" makes no difference at all.
I still cannot really compile this, if someone can make a deb file with 1.1.9 and the pull request I would be happy to test it out on my end aswell just to get more data on this.

khnielsen on 23 Aug 2018

the numbers you see in the torrent list, 35(36), are reported by the tracker in its response. it's only updated on tracker responses, which happens every 30 minutes (normally) AND when the torrent completes. When you complete the torrent there's an additional announce to say that you're a seed. That's why it updated at that point, and that's also why you got more peers. Presumably all other sources of peers (local peers, DHT and peer exchange) have been disabled, so announcing to the tracker is the only way to get more peers.

When you force announce to the tracker, do you see the tracker timers actually indicate that it announced? (perhaps deluge doesn't display these). But chances are that the announce is deferred to after the min-interval expires.

arvidn on 23 Aug 2018

PEX etc are all enabled except DHT as private trackers tend to not like that - The thing is that even forcing an update will not change this, and it also does not reset the "Time to next announce", there's no action on that at all.
I see that the min interval is changeable via ltconfig, maybe that might be worth taking a look at?
I ran a really quick test on version 1.1.1 which is the stable in debian 9 from the repos, this one does not appear to have this "issue", more on that later :)

khnielsen on 23 Aug 2018

@arvidn

For instance, in this case, where the tracker only hands back yourself, clearly it makes sense to announce again sooner, once there are more peers.

Why? As soon as the next peer announces he will get your IP and they will connect.

As @Seeker2 points out, ideally trackers would not hand out seeds to other seeds either, but they do.

That happens only on really old tracker implementations so I don't see how that's relevant here.

For example, the tracker could set the re-announce interval to 5 seconds if the torrent is < 1 minute old and there are less than 5 peers. Or something along those lines. Perhaps the announce interval could be proportional to the number of peers and the age. But now trackers seem to expect all "smarts" to be in the clients, which isn't necessarily bad.

It's true that the trackers that set the min interval do so by hardcoding the value directly in the code, but I don't think that there would be any kind of an improvement for the clients even if the tracker was "smarter" about that since like I said if there are only a few peers in the swarm (like in your examples) then all following peer announces (e.g. the seeder starts to seed or a new leecher starts to leech) would automatically get back all other peers in the swarm. Most of the trackers send back 30 or 50 peers if the client didn't send the numwant parameter so if there is really so few peers in the swarm each following peer which would announce would get all previous peers.

With the current logic, of not being allowed to re-announce for 5 minutes, you get penalised for announcing early.

I don't see how they get penalised as I explained above how that scenario works.

I still believe that the min interval change doesn't have anything to do with the issue at hand as a lot of people here are complaining specifically how 1.1 doesn't connect to nearly as much peers as 1.0 does.

X-Coder264 on 23 Aug 2018

Why? As soon as the next peer announces he will get your IP and they will connect.

I said: "(at least as long as they can't accept incoming connections).". But it's also not a given that peers will necessarily connect to you.

That happens only on really old tracker implementations so I don't see how that's relevant here.

That's great, I'm glad to hear!

I don't see how they get penalised as I explained above how that scenario works.

It is my experience that a lot of people are not able to accept incoming connections. Given that there's a non-zero risk of being such person, I think you could say it's a penalty. Granted, there are other more severe penalties for not being able to accept incoming connections.

I still believe that the min interval change doesn't have anything to do with the issue at hand as a lot of people here are complaining specifically how 1.1 doesn't connect to nearly as much peers as 1.0 does.

So perhaps it's an issue with libtorrent's NAT-PMP or UPnP implementation. It could also be some case in the peer connection state machine that fails with some probability, making incoming connections more likely to be disconnected.

Could anyone who experience this problem produce a full log?
Does anyone know how to make Deluge log all alerts?

arvidn on 24 Aug 2018

You can enable all alerts in deluge by editing deluge/core/alertmanager.py. There is a call to self.session.set_alert_mask on-or-about line 57 where you can replace the list of alert categories with lt.alert.category_t.all_categories. Unfortunately this is not trivial to do on Windows because the pre-built installer only ships with the bytecode compiled python files.

ssiloti on 24 Aug 2018

First trial of the patch in #3250 is promising, but there is not much activity on the tracker currently. Will do another run in 10 hours or so.

quarreldazzle on 24 Aug 2018

@quarreldazzle please keep an eye at the number incoming connections too. does deluge tell you whether a peer was incoming or outgoing in the peer list?

arvidn on 24 Aug 2018

@quarreldazzle - would you be able to make a .deb file? I can test it then, cant seem to figure out how to get git to actually pull in the patch.
@arvidn just a quick note, there's no way in deluge to see if a peer is in/out-bound which makes it hard to debug these kinds of issues - I have been switching a bit between 1.1.9 and 1.1.1 from the debian repo, and 1.1.1 appears to be working quite okay, im not seeing the same behaviour of not grabbing peers in the start.

khnielsen on 24 Aug 2018

@khnielsen sure, creating the deb is actually part of my building process: https://gofile.io/?c=AZ7DaP

quarreldazzle on 24 Aug 2018

@quarreldazzle absolutely awesome, I have installed it and are running tests now aswell..

Fingers crossed as the performance potential of the 1.1.X branch is quite a bit higher than the 1.0.X branch.

khnielsen on 24 Aug 2018

@arvidn, unfortunately the issue seems to be still there. I sent you the full Deluge log file. I am reporting here the events I manually logged:

19:11 TORRENT_1 added

19:11 announce OK - peers 9(11) seeders 1(1)
19:12 update tracker, nothing new
19:12 pause/resume, peers 32(46)
19:15 seeding, peers 12(45)

19:27 TORRENT_2 added

19:28 announce OK - peers 0(1) seeders 0(0), not downloading
19:29 update tracker, nothing new, peers 0(1) seeders 0(0), not downloading
19:30 pause/resume, download starts, peers 2(7), Seeders 18(21)
19:31 seeding

In both cases the peers appearing after the pause/resume were at advanced download state already (plus many had become seeders already).

@khnielsen how about you?

Edit, just to be sure that I did it right. Here is how I patched libtorrent with #3250:

git clone -b RC_1_1 https://github.com/arvidn/libtorrent.git
cd libtorrent
git fetch origin +refs/pull/3250/merge
git checkout FETCH_HEAD
./autotool.sh
./configure --enable-python-binding --with-lib-geoip --with-libiconv
make -j8
checkinstall -y --pkgversion=RC_1_1 
ldconfig

Please tell me if I made it wrong :-) git log included this as 3rd to last entry:

commit 541955b5e0a83c2f2d674d2a93baa1e4b73afdfb
Author: Arvid Norberg <[email protected]>
Date:   Sun Aug 19 00:16:03 2018 +0200

    TEMP: default ignore min-interval in python binding

quarreldazzle on 24 Aug 2018

@quarreldazzle - That is actually odd because im seeing a massive difference with the deb you sent - The issue seems to have more or less fixed itself here.

I can confirm that updating the tracker is not done even with a manual update, that might be something else or something that cant be fixed due to the tracker refusing that, but from my testings it has performed more or less the same for me over the tests i have made against 1.1.1 and 1.0.11

khnielsen on 24 Aug 2018

@khnielsen great to know :-) I also think I am seeing good performance improvements overall. But the issue is still there. Just to be sure, are you also sending from autodl to Deluge through the console with a 0 seconds delay and using no "force announce" script as I am doing for these tests? I wanted to cause no artificial fixes.

quarreldazzle on 24 Aug 2018

@quarreldazzle No im not actually, that might be the thing for you, im using autodl with 60 seconds delay in my case for the IPT torrents, I have not messed about with the other one as 60 seconds means im on the torrent 10-15 seconds before the initial seeder in almost all cases.

That might be the thing here, seeing as the tracker update, even forcing it is not done, then you might be on the torrent too soon for all the other peers to have actually announced anything, so when you get the announce in, there's actually no other peers before an update is done.

khnielsen on 24 Aug 2018

@khnielsen with 1.0 I was using 0 seconds and the force update script I refer to in my first comment. Worked fine. The action simply started within 20-30 seconds. Also in this case, I force the announce manually and see the initial announce within 20-30 seconds (similarly to setting the delay). I tried avoiding the pause/resume for up to 5 minutes sometimes, it does not change anything. Only the pause/resume thing will help.

What works most of the times as a quick fix is to trigger a pause/resume right after the first Announce OK. There I see the number of peers double or more. There is definitely something else going on.

I think that with your 60 seconds delay you are artificially solving the issue in a similar way :-)

Edit: I also wanted to tell in my issue report that earlier versions of 1.1 did not have this issue as you are also reporting. But it was maybe just a false memory and could not be precise about it.

quarreldazzle on 24 Aug 2018

@quarreldazzle That might be in some way, with 1.1.5+ I was actually seeing what you were seeing, it is fixed on this build, so while it might not be 100% the same issue, im still seeing an improvement over the previous builds.

khnielsen on 25 Aug 2018

if the problem is solved entirely by a pause and resume, it seems unlikely that the underlying cause has anything to do with failing to accept incoming connections, as that would (presumably) be unrelated to announces to the tracker.

arvidn on 25 Aug 2018

@quarreldazzle judging from those logs, it still looks like the announce is being suppressed. I can see the attempt to force-reannounce, but I don't see any actual announce nor response.

I will take a closer look if there's a problem with my patch. Are you running on windows, or would steven's patch to deluge be possible for you? (it would be easier to understand what's going on if I could see all alerts).

arvidn on 25 Aug 2018

@quarreldazzle while the build runs better im seeing a ton of random crashes, are you facing the same thing? Deluge just randomly stops running and I cannot really see why.

khnielsen on 25 Aug 2018

@quarreldazzle I updated the patch with a few fixes, please test again! (and sorry about force-pushing, you may need to reset your branch)

arvidn on 25 Aug 2018

@arvidn Im going to follow the directions that quarrel dumped above and test it aswell, I am seeing that it for some reason do not wish to connect to the same amount of peers as 1.0.11 does, not entirely sure why but even forcing an update to the tracker has no effect, even on a tracker that I know are coded to not reject tracker updates, an update is simply not processed, even in situations where on 1.0.11 connects to 100 peers (Tracker sends 100 max), 1.1.9 will never connect to more than half of that.

Edit; This is what im seeing when it is running, shutting down deluge and installing 1.0.11 will instantly connect to all of the peers, close it and start it up again and you will see about the same peers again, with a ton missing even though the tracker sends more back.
screenshot_14
Im not sure how to debug this for you, would you need a "debug" log from deluge to look into this?

Worth noting here, is that this only appears to effect linux, on windows with the same "setup" it will connect to all peers, which makes it even more tricky imo.

Edit 2: I noticed that even after letting a torrent run for 30 minutes and then clicking "Force tracker update", then the timer inside deluge never resets back to the normal update interval of 30 minutes, this is on 1.1.9 with the patch - Doing the same on 1.0.11 will reset the timer back to 30 minutes indicating that it actually updates the tracker, letting the time run out it will update the tracker and connect to more peers - So it is as if the updates are simply not happening.

khnielsen on 25 Aug 2018

note that I updated the patch around the time you posted, so you probably tested the previous version. I discovered some issues with my first patch that were fixed

arvidn on 25 Aug 2018

Did you update it after your post? I just build it entirely fresh after your latest post, just to be safe imgoing to build it again tonight with a completely fresh one again.

khnielsen on 25 Aug 2018

@arvidn thank you, I will test the updated patch in the next days

quarreldazzle on 25 Aug 2018

@arvidn Alright, so I ran the build again as per the above instructions, and guess what, im seeing the timer go back to the "normal" interval and also update the "peers" available in the () in deluge, so that part at least appears to have resolved things.
Im still seeing it connect to much less peers though than 1.0.11 -
screenshot_15
1.0.11 will instantly connect to all of them, im not sure where to look for this, it might be an entirely different issue.

khnielsen on 25 Aug 2018

@khnielsen when you see less peers than expected (as I am also seeing), can you try a pause/resume to see if you also immediately receive tons of more peers?

quarreldazzle on 25 Aug 2018

@quarreldazzle - Im not seeing any difference on that, judging from what you have stated im actually fairly certain that you are seeing a different issue than me, the patch provided by arvid works perfectly here. What I did discover is that I have half open connection set to -1, when I have that it will not connect properly to all the peers available, if I set it to something fixed like 100 it will connect properly, although quite a bit slower than 1.0.11 but it will eventually get a hold of all peers the tracker send and often more as PEX works its wonders.
The issue you are seeing @quarreldazzle are more likely to be a tracker problem, seeing as you have an "OK" announce but in that announce you are so quick to connect to the tracker that no other peers are actually there, as im loading torrents with a 60 second delay im not seeing that as others have announced - But with the patch I can properly "Force announce" all torrents.
What I am seeing @arvidn is that peers can connect just fine on 1.1.9 - Like upload a torrent and wait for it to get 40-50 peers, I can then start the torrent, I get the announce and it connects to all of them right away, it will then drop them off one by one until about half are connected and then from there starting accepting it again - It might be something im doing here, but I have tried nearly everything I can think of. This does not happen on 1.0.11.

khnielsen on 25 Aug 2018

@arvidn when testing the revised patch in #3250 (is it updated if it shows infos from 1 week ago btw.?), I simply repeat the process I described here. Hope this is ok!

quarreldazzle on 26 Aug 2018

@arvidn yes, this one is a great improvement! I will send you the log file after sending this comment, but the revised patch in #3250 works when hitting update tracker.

Deluge was not able to retrieve andy seeder initially and also just a few peers, but hitting "update tracker" made me receive more peers first, and seeders then. No need to pause/resume anymore.

Now, for both cases I am sending you, Deluge/libtorrent starts to see seeders at a point but do not connect to them and prefer connecting to peers/leechers. This is still a different behavior from 1.0.X. But perhaps this is a tracker issue at this point.

Maybe @khnielsen can test this one, too?

quarreldazzle on 27 Aug 2018

@quarreldazzle - I am seeing the same thing, but I cannot really debug it as there's no real indication as to where it goes wrong.
For now I have been forced to drop back to another build as deluge keeps crashing when using 1.1.9 even without the patch, not entirely sure why as even the highest logging level has no indication as to why it crashes, I prefer stability over the last 10% performance :)

khnielsen on 27 Aug 2018

@khnielsen weird, both this and the previous patch as well as 1.1.9 before I submitted this issue report were stable to me. Sorry to hear!

quarreldazzle on 27 Aug 2018

It is indeed odd, I found some indications of crashes with deluge on the qbittorrent part with another bug im looking into for that client.
But I cannot reproduce it, I can leave it running and come back 4 hours later, 50% of the time it has crashed without logging anything which is no good for me.

khnielsen on 27 Aug 2018

Hello,

I'm sorry I don't want to hijack this thread, but I don't get the full impact of the related commit Option to ignore min interval #3250.

So, ok, with this, If I force a re announce, it will do it, even if the tracker sent a higher number, right ?

But, what about the situation, on a lot of private tracker, when you download a torrent via rss or auto-dl, but at first it sends a error like "torrent is not registered", or 0 peers 0 seeds. In that cas, utorrent or rtorrent "hammer" the tracker until it's working (like every seconds).

Will this commit allow this behaviour too ? I've read the code, my gut says "no", but I'm not sure.

Right now, qbittorent (based on 1.1.9) will wait around 1 hour to retry... which is a dead sentence on some private tracker.

Thx.

Rootax on 3 Sep 2018

So, ok, with this, If I force a re announce, it will do it, even if the tracker sent a higher number, right ?

yes, if "a higher number" means the tracker sent a higher min-interval.

But, what about the situation, on a lot of private tracker, when you download a torrent via rss or auto-dl, but at first it sends a error like "torrent is not registered", or 0 peers 0 seeds. In that cas, utorrent or rtorrent "hammer" the tracker until it's working (like every seconds).

Right, that is my understanding as well. libtorrent does not do that. I've suggested a few times that perhaps it should. The counter-arguments are that it violate the specification and that the problem really should be fixed at the tracker (for instance by doing what I suggested here)

Will this commit allow this behaviour too ? I've read the code, my gut says "no", but I'm not sure.

~~It does not.~~ It doesn't implement that behavior, but with this flag, it allows a client to implement this behaviour.

Right now, qbittorent (based on 1.1.9) will wait around 1 hour to retry... which is a dead sentence on some private tracker.

I would have expected 30 minutes (as that's typically the default announce interval). In the case of it taking 60 minutes, is that on trackers that have 60 minute announce intervals?

arvidn on 3 Sep 2018

All right, thx for the answer.

Another question about

It doesn't implement that behavior, but with this flag, it allows a client to implement this behaviour.

What value (and where) do I need to change to havec the utorrent / rtorrent behaviour ?

For the hour delay, I've only this problem with 1 tracker, so I guess that's theirs settings.

Rootax on 3 Sep 2018

What value (and where) do I need to change to havec the utorrent / rtorrent behaviour ?

it's not as simple as just changing a setting. there would have to be some logic on tracker failure alerts, recognising that the torrent hasn't been registered yet. Probably via some heuristic of looking for the string "registered" or something along those lines. And then force re-announce (by calling torrent_handle::force_reannounce()).

Also in the case of a successful announce, but where the number of peers returned is < 5 (or something), it should re-announce.

On top of it all, there should be a limit on the number of times either of those cases trigger.

arvidn on 4 Sep 2018

I have written a script according to the above comments for Deluge.

Link : https://pastebin.com/Hr9eZJB4

darthShadow on 4 Sep 2018

it's not as simple as just changing a setting...

Thx for the explanation. I thought it could be managed at the libtorrent level, but I get why letting each app deciding this behavior is maybe better. Now, If Qbittorent could look at this too... @sledgehammer999

Rootax on 4 Sep 2018

I am using basically the same approach as @darthShadow and it works great with the patch. I have added a couple more update requests after seeing connected peers, too, for getting most of the swarm.

quarreldazzle on 4 Sep 2018

I've landed the patch, but note that the flag to ignore min-interval is not set by default now (that was just to simplify testing).

arvidn on 4 Sep 2018

Little confused here, as per the RC_1_1 branch the flag ignore_min_interval is still set to 1. Should the value be 0 instead?

darthShadow on 4 Sep 2018

@arvidn thank you so much for your prompt response to all our inputs! To sum it up (please correct me if I am wrong), the following will land to libtorrent ~~1.1.9~~ 1.1.10 as a result of this issue report:

improved torrent_connect_boost setting, now with 80 as default value and 255 as max value, which attempts to maximize the number of firstly connected peers to the number given
new ignore_min_interval setting, with 0 as default, for ignoring the minimum interval by trackers for force announces. Settings this to 1 requires an implementation on the client side of some logic to handle the case, like the script provided here. Only relevant to Deluge: this requires a manual install of the update-tracker console command [root required].

quarreldazzle on 4 Sep 2018

@darthShadow This is what I was referring to. i.e. the call to force_reannounce() won't ignore min-interval, unless you explicitly specify the flag to do so.

arvidn on 4 Sep 2018

👍1

@quarreldazzle sounds right. The update-tracker.py should probably test for which libtorrent version, and if it's > 1.1.9 it can pass in the flag to force_reannounce().

The patch has landed in the RC_1_1 branch in libtorrent, which is the stable branch for the 1.1.x releases. The next release this will make it into will be 1.1.10.

arvidn on 4 Sep 2018

👍2

I consider the issue closed, but I leave it to you in case you have any pending related tasks. Thanks again!

@arvidn I see the revised patch now.

Do I get it right that ignore_min_interval is now _not_ part of the options that libtorrent enables us to set with ltconfig (for Deluge), but it is now an argument to the function force_reannounce()?

quarreldazzle on 4 Sep 2018

There is no option to disable it client-wide, just the flag to be passed in to force_reannounce()

arvidn on 9 Sep 2018

libtorrent 1.1.{7,8,9} + Deluge 1.3.15, connecting only to small fraction of peers over dozens and only on certain trackers

Most helpful comment

All 82 comments

Related issues