Rtorrent: [feature request] Prevent rtorrent from rehashing everything after it crashes

[2600box]

rtorrent crashes fairly frequently, which is fine. the pain is that it rehashes everything after restarting.

rtorrent should have an option to prevent rehashing known good torrents and only rehash those that were incomplete.

this is my number one feature request

[chros73]

rehashes everything after restarting

Have you tried this in your config?

# Check hash for finished torrents. (check_hash)
pieces.hash.on_completion.set = no

[2600box]

yes, i have always set this and this only prevents rehashing when a download finishes to check it downloaded correctly. it has no effect on the insane startup rehash.

[chros73]

... it has no effect on the insane startup rehash.

It should have! Although rtorrent fires up the hash check event at startup for every torrent, but none of the completed torrents should be affected by this.

It can mean 2 things:

• problem with your session files (unreadable/unwriteable, damaged)
• other util(s) forces hash checking

I bet you use rutorrent (I don't): just ditch it completely for couple of days and see whether your problems still exist (same applies to your other issue as well: https://github.com/rakshasa/libtorrent/issues/145).

[2600box]

I do normally use rutorrent, mostly to keep downloads sorted by tracker name. I disabled scgi because of the crashing problems. It doesn't seem to have had any effect.

My crashing has actually gotten worse and now rtorrent is unresponsive at startup while hashing until it crashes with the standard SIGBUS as seen here: https://github.com/rakshasa/rtorrent/issues/551 . I have tried debian source, feature-bind, latest stable source, rtorrent-ps script and pimp-my-box. I have tried changing config files. I have tried local DNS, building latest stable curl with c-ares.

I am tearing my hair out. It is only a small VPS with 2GB RAM and a single 3.5ghz cpu thread with 200GB of disk.

Of course, if I start with a blank .session it starts normally, but as soon as I start adding them back, the same behaviour appears.

[2600box]

and to bring it back to the subject of this feature request, i have experienced crashing of rtorrent over the years on ubuntu, debian and os x and it always rehashes on startup after a crash, even though the data has not changed and always using "no hash on complete".

[chros73]

I do normally use rutorrent, mostly to keep downloads sorted by tracker name.

You can achieve the same thing (even more complex ones) with rtorrent-ps as well, although it's a matter of preference.

My crashing has actually gotten worse ...

How often do they accure? In 10 mins / 10 hours / etc. ?

i have experienced crashing of rtorrent over the years on ubuntu, debian ...

I've only seen crashes in the last year when it ran out of space. :) I use Ubuntu 14.04 on a laptop (physical hardware).

it always rehashes on startup after a crash, even though the data has not changed ...

Well, it shouldn't have, and it works fine for me: couple of days ago I crashed it by accident (made it to run out of disk space), and only 2 torrents were rechecked on next startup, one that hasn't been completed yet and an other one that was complete but session.save hasn't run yet.

session.save runs in every 20 mins by default, until you haven't overriden in your config. It can happen that if this interval is large (e.g. 12 hours) and you have a crash in every hour, so rtorrent will never save (update) the session files.

So, probably your session files are faulty somehow.
Make quick test:

• wait until all's good in rtorrent, then fire up manually session.save= in rtorrent
• quit rtorrent normally ^q
• make a backup of your session dir
• start rtorrent again (you shouldn't see any hash recheck), don't add any new torrents, just wait for / trigger the same crash
• make an another copy of your session dir and compare it with the good backup one: they should be different in some way
• finally, to test whether the session files are the problem: replace your session dir with your good backup session dir, and start rtorrent: you shouldn't see any hash recheck

[2600box]

the rehashing may be a red herring - my thinking was that it starts up and does all its io and tracker connections at once and is thrashing the disk on what seems to be a single thread.

i think you are right that it is related to my .session - i have had the problem before and the solution was to find the offending torrent and remove it. in this case however, the behaviour is different in that if I add a few, everything is ok, but adding more and it crashes. i have tried with different sets and had the same effect which points to unfamiliar territory.

i wish that rtorrent handled these "corruptions" better, and that they did not lead to the slow rehashing when the data is unchanged.

i cannot get to the point where everything is stable in rtorrent anymore to use your suggested methodology. the crash now happens some minutes after startup, before the hashing finishes.

i am thinking to re-add them to a fresh config and not via the .session, they are not so many... but it is still annoying and slow

[2600box]

i just did a fresh install of Debian 8. Installed rtorrent-ps with Ansible. Download a few things to test. All good.

Then I download a torrent I have the data for and go to reshash, it rehashes for a while, and then rtorrent crashes.

I am beginning to think the problem is with my vps.

[pyroscope]

rtorrent is quite sensitive to IO errors, since they basically happen in the kernel (mmap).

The least painful way out of such things is stopping all items via sed (state = 1 → 0), and then starting them slowly in a bash "while true" loop with sleep using "rtcontrol --from stopped // -/1 --start" (until a crash, and then the last started torrent is the culprit).

[2600box]

I moved everything to a fresh instance with rtorrent-ps on debian8. it worked great for a week and then it crashed and now it crashes soon after startup, the same as before. typical output is:

Caught SIGBUS, dumping stack:
/opt/rtorrent/bin/rtorrent() [0x413d0f]
/lib/x86_64-linux-gnu/libpthread.so.0(+0xf890) [0x7f72b0465890]
/usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0(+0x7e91c) [0x7f72aee9691c]

Error: Success
Signal code '2': Non-existent physical address.
Fault address: 0x7f72a3c1c000
The fault address is not part of any chunk.

I changed the state or every torrent with sed -i 's/statei1e13/statei0e13/g' *.rtorrent and it all starts up nicely until I start hashing and then the UI becomes unresponsive while starting torrents, which it hashes every one, even though they are all complete and I have pieces.hash.on_completion.set = no.

i would also appreciate more detail on the while true loop with some example, if you don't mind.
how is this?

#!/bin/bash
while true
do
rtcontrol --from stopped // -/1 --start
sleep 300
done

but where will i see which torrent caused the problem? does it need particular verbose logging somewhere?

[pyroscope]

Add a set -e, and rtcontrol will abort the script as soon as rT doesn't react anymore (i.e. crashed).

And add rtcontrol --from stopped // -/1 -qo name above the start call to get diagnostics.

[2600box]

script seems to do its job, but rtorrent still crashes after a few hashes. I have done this about ten times now, removing the last attempted torrent each time from the session directory now and it still continues to crash.

for reference i used sed like this sed -i 's/statei1e13/statei0e13/g' *.rtorrent and my script is like this:

#!/bin/bash
set -e
while true
do
rtcontrol --from stopped // -/1 -qo name
rtcontrol --from stopped // -/1 --start
sleep 300
done

[jef]

I have an external drive (raid box) that's connected via USB. Rutorrent/rtorrent is started in Docker automatically on reboot, but sometimes the drive isn't connected right away. When it does connect and I start up the torrent sessions again, it looses it's information on being completed.

How do I prevent it from loosing that session or fast resuming the torrents that are definitely there?

edit: This comment seems to be relevant. Maybe I should make sure my rTorrent doesn't auto start or at least check if the drive is mounted before booting.

[pyroscope]

do not add to 3 years old tickets, this is pure chaos.