(EDIT) modified the steps and added a variable. It should reproduce the issue every time now; at least it does on my end.
v4.2.1 (64-bit) / Windows 10 1909 (18363.535)
"Recheck" doesn't work for moved-then-unmoved torrents when the ".!qB extension" option is enabled.
"Recheck" should always work even with the ".!qB extension" option enabled.
(EDIT) Note: The _"Recheck torrents on completion"_ and _"Keep incomplete torrents in"_ options are disabled.
If I start the torrent, it'd start from 0.0%. The interesting part is that it would create a second file but with the .!qB extension, e.g. the original complete file "A.exe" and another file named "A.exe.!qB".
Workaround:
There are two methods:
a. If the ".!qB extension" option is enabled, recheck works if I add the .!qB extension manually to the original completed file/s.
b. Disable the ".!qB extension" option, exit qbt, launch it again and recheck should now work.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Old and frequently reported bug. Nobody seems to have the gumption to approach the issue of how a recheck should attempt to locate the files to be checked.
The client SHOULD check for the filenames both WITHOUT and then WITH the .!qB extension.
The client SHOULD check for the filenames in both the torrent's "Save Path" as well as the "Incomplete" temp path.
The client SHOULD check for the filenames WITHIN and WITHOUT a subfolder.
In all, Qbittorrent should grope around through no fewer than 6 distinct paths for each file within a torrent. Until then, we have to shuffle the files around ourselves; move and rename them manually before performing a Force Recheck.
Sounds like a problem in the fastresume files. the file extension is probably not being correctly updated.
@glassez thoughts?
Anyway, what is the purpose/use case of the "蛣append .!qB extension to incomplete files" feature? Seems like bloat that's more trouble that it's worth. If one needs to check whether or not a file is complete:
btcheckAnyway, what is the purpose/use case of the "蛣append .!qB extension to incomplete files" feature? Seems like bloat that's more trouble that it's worth. If one needs to check whether or not a file is complete:
* Check in qBittorrent itself, force recheck if needed * Use an external tool like `btcheck`
Why not? Say I'm downloading a torrent with 20 large files that I need to work on in no particular order ASAP. With the extension option enabled I can simply open the folder, immediately tell which file is completed by merely looking at the thumbnail, open it and do my work. Also, as the torrent keeps getting downloaded and more files become completed, I wouldn't have to leave the folder to check.
Without the extension enabled I'd have to constantly open up qbt to check.
The .!qB extension is pretty necessary to me. It tells me in no uncertain terms that a torrent is incomplete, regardless of what happens in hard drive swaps, data moves, and archive digging 20 years from now. I'm terribly glad my p2p clients from 20 years ago used "part file" incomplete extensions, just as Firefox and Chrome do today.
I think for the common user, the .!qB extension is intended to prevent multimedia players from automatically indexing the incomplete files and throwing errors.
Want to get deeper in the matter, qBittorrent should generate an SHA checksum file for each completed torrent that it checks. If it's going to re-read the file during a check, just generate us some .SHA256 or .SHA512 files to go with them for extra peace of mind.
@eddlang
Why not? Say I'm downloading a torrent with 20 large files that I need to work on in no particular order ASAP. With the extension option enabled I can simply open the folder, immediately tell which file is completed by merely looking at the thumbnail, open it and do my work. Also, as the torrent keeps getting downloaded and more files become completed, I wouldn't have to leave the folder to check.
Without the extension enabled I'd have to constantly open up qbt to check.
In your scenario you say you are downloading, so qbt is already opened. How is bringing up qbt's window different from navigating in your filesystem to your downloads folder and opening it?
@a-raccoon
It tells me in no uncertain terms that a torrent is incomplete, regardless of what happens in hard drive swaps, data moves, and archive digging 20 years from now. I'm terribly glad my p2p clients from 20 years ago used "part file" incomplete extensions, just as Firefox and Chrome do today.
If you are digging up old files, to be sure of their integrity "in no uncertain terms", you simply recheck the torrent using qbt or even a standalone tool like btcheck. Actual cryptographic hash verification gives you much stronger guarantees of completion (and integrity) than "hey, it does not have the !qB extension, so it must be complete".
Want to get deeper in the matter, qBittorrent should generate an SHA checksum file for each completed torrent that it checks. If it's going to re-read the file during a check, just generate us some .SHA256 or .SHA512 files to go with them for extra peace of mind.
A .torrent file is already a list of hashes of the various file pieces. This is one of the main features of Bittorrent itself, automatic integrity checking with cryptographic hashes. If you read the pieces, calculate their hashes, and those hashes match the ones on the .torrent file, then the files are good. This is what qBittorrent's recheck functionality does (via the underlying libtorrent code), as well as standalone tools like btcheck.
Granted, the current version of the BIttorrent protocol uses SHA-1 hashes instead of SHA-256 (or 512), but this poses no practical problem in terms of integrity checking.
If you are curious, the use of SHA-1 hashes is only a concern right now for torrents will very small piece sizes due to the BitErrant attack. This is one of the things that the v2 specification fixes.
In your scenario you say you are downloading, so qbt is already opened. How is bringing up qbt's window different from navigating in your filesystem to your downloads folder and opening it?
What makes you think the qbt window is always open? For me the window is closed and the program simply runs in the background and lives in the notification area.
Besides, even if the window was always open, constantly checking the program to see which one of those many files in that single torrent have been completed is user unfriendly and a waste of time. Why do unnecessary steps when I can easily tell from the folder which I'm working on?
None of this is relevant though. The program should be able to recheck regardless of the extension option being enabled or not. It's a bug. Utorrent has the option and I've never encountered this issue with it.
@eddlang
What makes you think the qbt window is always open? For me the window is closed and the program simply runs in the background and lives in the notification area.
By "open" I meant "application running". Whether it is _minimized_ to tray or taskbar (or equivalent) in Windows is basically the same. Takes roughly the same amount of effort to bring it up in either case as alt+tabbing to the folder you are manually monitoring.
Besides, even if the window was always open, constantly checking the program to see which one of those many files in that single torrent have been completed is user unfriendly and a waste of time. Why do unnecessary steps when I can easily tell from the folder which I'm working on?
I'm confused as to why you say it's "user unfriendly". Isn't the purpose of a (Bittorrent protocol) _download manager_ exactly that? To download files and report on the progress of the operation?
Do you have a suggestion of how qBittorrent's UI could be made more user friendly?
Either way, you are either "constantly" checking a folder or "constantly" checking qbt. qbt gives you more information, like current download speed, ETA, global and individual progress in a tree view all in one glance, if you have the content view opened. In comparison, looking at a folder only tells you if it is ready or not. Plus if a given torrent has multiple, possibly nested files, you also have to check each subfolder (though for the sake of argument I recognize you can use some kind of tree view in your file manager to ameliorate that problem).
None of this is relevant though. The program should be able to recheck regardless of the extension option being enabled or not. It's a bug. Utorrent has the option and I've never encountered this issue with it.
It is relevant. I would prefer if this feature were available without bugs as well, in a perfect world. But qbt is a free software project lacking in manpower. Sometimes the decision must be made whether supporting and attempting to fix some features of limited usefulness that generate a lot of problems for the users is worth it vs. simply scrapping it. Thanks to version control, one can always revive the feature later, the removed code is not lost forever.
@FranciscoPombal Your arguments make the assumption that qBittorrent is better or more ideal that the OS file explorer, or third party file explorers, or remote file explorers and file transfer protocols. Thus, most of your arguments fall flat.
Everyone requires affirmative signaling outside of the qBittorrent client. An entire world exists that precludes this software and I will not make my library dependent on it for discovering which files are complete, incomplete, valid or corrupt. I have Tens of Thousands of files that are spat through qBittorrent across multiple machines.
@a-raccoon
@FranciscoPombal Your arguments make the assumption that qBittorrent is better or more ideal that the OS file explorer, or third party file explorers, or remote file explorers and file transfer protocols. Thus, most of your arguments fall flat.
Name 1 (one) file manager (remote or local) that somehow reports download progress of a torrent, when looking at the folders that contain the torrent's files(*). The Windows explorer file manager, which is what the OP is almost certainly using, certainly doesn't.
I never claimed that qBittorrent was "the best at everything", like you seem to imply I claimed. I only said that it is objectively better to bring up qBIttorrent's window to check the progress of a download than to manually look at a file manager window and look at the file extensions.
I also never claimed that qBittorrent had the best GUI out of any file transfer application, in fact I encourage anyone to explain and point out the flaws, with screenshots if necessary, so that it may be improved by someone with interest.
Everyone requires affirmative signaling outside of the qBittorrent client.
Can you clarify what you mean by "affirmative signaling"? I don't know what it means.
An entire world exists that precludes this software and I will not make my library dependent on it for discovering which files are complete, incomplete, valid or corrupt. I have Tens of Thousands of files that are spat through qBittorrent across multiple machines.
Not sure if I completely understand these sentences either ("entire world that precludes" and "spat through"), but if you want to check file integrity outside of qBItorrent, you can use btcheck like I already mentioned. There are also many other alternatives written in every language imaginable, from python scripts to probably even haskell. You can even easily implement one yourself. You just have to read the files, calculate the hashes of their pieces, and compare them to the hashes you get from parsing the torrent file, which is a simple bencoded structure.
Alternatively, if you want to make software that needs to check for download completion in real time (since you mention "my library"), you can use qBittorrent's API. It's HTTP and spits out JSON, can't get more interoperable than that.
(*) for the sake of my faith in humanity, I really hope such bloat does not exist.
By "open" I meant "application running". Whether it is _minimized_ to tray or taskbar (or equivalent) in Windows is basically the same. Takes roughly the same amount of effort to bring it up in either case as alt+tabbing to the folder you are manually monitoring.
I'm confused as to why you say it's "user unfriendly". Isn't the purpose of a (Bittorrent protocol) _download manager_ exactly that? To download files and report on the progress of the operation?
Do you have a suggestion of how qBittorrent's UI could be made more user friendly?
Either way, you are either "constantly" checking a folder or "constantly" checking qbt. qbt gives you more information, like current download speed, ETA, global and individual progress in a tree view all in one glance, if you have the content view opened. In comparison, looking at a folder only tells you if it is ready or not. Plus if a given torrent has multiple, possibly nested files, you also have to check each subfolder (though for the sake of argument I recognize you can use some kind of tree view in your file manager to ameliorate that problem).
It is relevant. I would prefer if this feature were available without bugs as well, in a perfect world. But qbt is a free software project lacking in manpower. Sometimes the decision must be made whether supporting and attempting to fix some features of limited usefulness that generate a lot of problems for the users is worth it vs. simply scrapping it. Thanks to version control, one can always revive the feature later, the removed code is not lost forever.
It IS an unnecessary extra step/s, hence it wastes time.
With the extension option enabled:
It might not be much for you, but it is for me. It is user unfriendly in my case. Just because you don't use it, doesn't mean it's worthless.
It's not relevant because an existing feature should work without bugs; that's the point of it, but if no one's going to fix it, then so be it. I'm just reporting a bug, but please don't go around removing it because you personally don't want it. I'd rather have a bugged feature that can be worked around instead of no such feature at all.
@eddlang
It IS an unnecessary extra step/s, hence it wastes time.
1. I have to switch to qbt. 2. I have to go through the files list to see which files are 100%. 3. I have to switch to the folder. 4. I have to go through the file names to match the ones that are completed. 5. Work on another completed file.With the extension option enabled:
1. I switch to the folder and .!qB files can immediately be distinguished from the completed files. Also, since my files are sorted by type, they are even more easily distinguished. 2. Work on another completed file.
Fine. I concede this point. I overlooked the fact that you are working on the files themselves that you are downloading. Though to be fair, the 5 step procedure can be roughly shortened to 2 steps if you have qbt and the folder window side-by-side.
In qbt look at which files are 100%.
In the folder, with the columns sorted by file names, you can select them and work on them
It might not be much for you, but it is for me. It is user unfriendly in my case. Just because you don't use it, doesn't mean it's worthless.
It's not relevant because an existing feature should work without bugs; that's the point of it. If no one's going to fix it, then so be it. I'm just reporting a bug, but please don't go around removing it because you personally don't want it.
I never said it is worthless because I don't use it. I thought it was kind of worthless because I could not find any good reason for it to exist, compared to the effort of maintaining it and fixing its bugs. I agree that a feature should work without bugs. But if you leave a buggy feature for too long, users will start distrusting the program, especially bugs that have to do with data integrity.
I guess this is where we fundamentally disagree now:
I'd rather have a bugged feature that can be worked around instead of no such feature at all.
I think users expect a certain level of quality-assurance for functionality in the stable releases.
I am simply entertaining the thought that maybe this feature has reached the point of being too buggy and having caused too many issue reports over time to be kept in its current state.
TL;DR:
You wrote:
Name 1 (one) file manager (remote or local) that somehow reports download progress of a torrent.
Answer: None. That's exactly why we need the .!qB extension.
Fine. I concede this point. I overlooked the fact that you are working on the files themselves that you are downloading. Though to be fair, the 5 step procedure can be roughly shortened to 2 steps if you have qbt and the folder window side-by-side.
1. In qbt look at which files are 100%. 2. In the folder, with the columns sorted by file names, you can select them and work on themI never said it is worthless because I don't use it. I thought it was kind of worthless because I could not find any good reason for it to exist, compared to the effort of maintaining it and fixing its bugs. I agree that a feature should work without bugs. But if you leave a buggy feature for too long, users will start distrusting the program, especially bugs that have to do with data integrity.
I think users expect a certain level of quality-assurance for functionality in the stable releases.
I am simply entertaining the thought that maybe this feature has reached the point of being too buggy and having caused too many issue reports over time to be kept in its current state.
I don't keep the folder or qbt in view when I work because they take up display space that I need for the said work, so even if I keep both the folder and qbt open side-by-side, it won't reduce the steps, because they are behind the work application's window, which means I still need to switch to them to see them.
I can go with minimizing the work window at step 1 which would bring the folder and qbt into view. That would remove step 3, but still in total It'd only reduce it to four steps, not two.
Also, these steps are not equal in effort. Trying to find a completed file based on filename in a list, where completed and uncompleted files look the same, is far more time consuming than simply looking at the thumbnail which immediately tells you if the file is completed.
The extension feature is not as buggy as you think. This recheck bug only occurs with moved files that are then unmoved to the original download location. It doesn't happen if you don't move the files, even if the extension option is enabled.
I'm hoping someone might come along and fix it, hence the report.
@eddlang
I don't keep the folder or qbt in view when I work because they take up display space that I need for the said work, so even if I keep both the folder and qbt open side-by-side, it won't reduce the steps, because they are behind the work application's window, which means I still need to switch to them to see them.
I can go with minimizing the work window at step 1 which would bring the folder and qbt into view. That would remove step 3, but still in total It'd only reduce it to four steps, not two.
Also, these steps are not equal in effort. Trying to find a completed file based on filename in a list, where completed and uncompleted files look the same, is far more time consuming than simply looking at the thumbnail which immediately tells you if the file is completed.
Again, I conceded the point. The step reduction stuff I said was just for the sake of argument. I can see that this feature is better suited for your workflow now.
The extension feature is not as buggy as you think. This recheck bug only occurs with moved files that are then unmoved to the original download location. It doesn't happen if you don't move the files, even if the extension option is enabled.
I'm hoping someone might come along and fix it, hence the report.
And I was arguing that we could consider removing the feature if it is found to be not that useful. After all, removing a feature is a valid, albeit extreme way to fix a bug with said feature. No feature = no bug. However, you have convinced me that the feature has some legitimately useful purpose. So I no longer consider its removal due to lack of merit of its existence.
My concern right now is this feature being a systematic source of issues on this issue tracker. You say it is "not as buggy as you think.". But try searching for extension-related issues (gihub search sucks though). There are dozens. So, on the account of being such a persistent source of trouble, maybe it could be removed. I am not outright saying it should be removed, I am just saying it should be a possibility to consider. This would come at the cost of users like you though. So it is not really a good option.
Ultimately, the contributors with write-access will decide. Let's hope someone comes along and fixes it. I personally am not inclined to do so, at least not anytime soon, as I am working on other things.
@a-raccoon
Answer: None. That's exactly why we need the .!qB extension.
@eddlang convinced me the feature is useful in certain situations for distinguishing between complete/incomplete. You should know though, that this feature does not give you any direct guarantee of integrity. For that, you really need to recheck, either with qbt or an external tool.
That's why we need a means for a Recheck (upon torrent completion, or forced recheck) to generate SHA checksums on each of the files, so the files can be independently verified outside of qBittorrent / btcheck. It's mildly out of scope, but not that difficult to implement in the grand scheme of things.
@a-raccoon
That's why we need a means for a Recheck (upon torrent completion, or forced recheck) to generate SHA checksums on each of the files, so the files can be independently verified outside of qBittorrent / btcheck. It's mildly out of scope, but not that difficult to implement in the grand scheme of things.
If you don't trust any torrent verification method, including qBItorrent's, how can you put more trust into the SHA checksums it would generate for the files? The recheck process already does what you want by design. the recheck capabilities in libtorrent and btcheck use known-good crypto hash implementations. If you don't trust that, you can't trust anything.
Nobody said anything about _trust_. This is about the life and evolution of a file once it grows up and leaves the cradle of bittorrent. Creating a clean transition from the verification that qbt provides, to standardized verification that still exists 20 years later when all the lame NFO and DIZ files have been deleted. When a file is ready to leave the nest, qBittorrent must let it go, and provide it with the tools it needs to flourish in the big scary world.
@a-raccoon
Nobody said anything about _trust_. This is about the life and evolution of a file once it grows up and leaves the cradle of bittorrent. Creating a clean transition from the verification that qbt provides, to standardized verification that still exists 20 years later when all the lame NFO and DIZ files have been deleted. When a file is ready to leave the nest, qBittorrent must let it go, and provide it with the tools it needs to flourish in the big scary world.
You don't seem to understand how Bittorrent verification works. The "verification that qbt provides" is the modern standard for integrity verification, using SHA-1 hashes.
Hashing pieces of a file and comparing to the values on the torrent file gives you exactly the same integrity guarantees as hashing the whole file and comparing that to a known good value.
You don't think you'll be able to verify file integrity (and trust said verification) with SHA-1 (SHA-256 in the case of Bittorrent v2) hashes in 20 years?
You just have to keep the .torrent file around, which is basically the same as keeping a .txt file with the SHA-1/SHA-256/SHA-512 sums of each file in the torrent.
Guys, I prefer not to participate in your pointless holiwars (I have neither the time nor the desire to do so), so I just skipped most of the comments.
I'll try to fix the issue, but I need some feedback. First of all, can someone (other then issue author) confirm the problem? I couldn't reproduce it the first time.
Make sure the "Append .!qB extension to incomplete files" option is checked.
Done.
Download a torrent to completion, exit qbittorrent and then move the torrent to another location.
Done (unless you're putting some special meaning into "to completion").
Launch qbittorrent and then exit it.
Done. Torrent is in "Missing files" state (as expected) when I launched qBittorrent.
Move the torrent back to its original download location.
Done.
Launch qbittorrent and recheck the torrent. It won't do anything and the percentage sits at 0.0%.
Done (partially). I just launched qBittorrent and see the torrent having 100%. Why I need to recheck it?
Some note: I tested it with my custom build that is very likely ahead of reported version by some commits so it can contains some related fixes.
Also I have "Recheck on completion" option enabled.
@glassez
Download a torrent to completion, exit qbittorrent and then move the torrent to another location.
Done (unless you're putting some special meaning into "to completion").
The OP mentions "(seeding job finished, although I don't know it plays a role or not)" which I assume means the torrent was paused, thus going into the "Completed" state before qBittorrent was closed. This is what I did. ~Still, @eddlang can you clarify this please?~ EDIT: yes, this is indeed the case.
Then I followed the same steps as you up to:
Launch qbittorrent and recheck the torrent. It won't do anything and the percentage sits at 0.0%.
Done (partially). I just launched qBittorrent and see the torrent having 100%. Why I need to recheck it?
My percentage was also reported at 100% upon opening. Recheck worked as expected up to 100%.(*)
So I cannot reproduce the issue.
Extra info: I used a recent Xubuntu torrent (info hash 6b1fdc1d893f79ab1bdc8099e811c006df21eceb), which is a single file. Maybe this bug is only reproducible with a multi-file/folders torrent? I will try and report back.
Some note: I tested it with my custom build that is very likely ahead of reported version by some commits so it can contains some related fixes.
Also I have "Recheck on completion" option enabled.
I tested with current master. "Recheck on completion" option was disabled (as is the default).
(*)Side note:
Why I need to recheck it?
Whether or not you need to recheck it is besides the point, if this issue were happening reproducibly as described in the OP, it would be clearly wrong behavior, and the user would think qbittorrent ate/corrupted their data.
Update: tried with a multi file torrent and still could not reproduce (torrent in question: http://www.legittorrents.info/index.php?page=torrent-details&id=436859e8dddf4d8bd22d9ecc826139b6749a9a4a)
@FranciscoPombal : I don't have the opportunity to test right now. I know there were some recent fixes to how qbt handles torrent states between reopening.
I would test having some completed torrents on an external drive. The torrents should be regularly Queued / Seeding and not Paused. Close qbt(1), remove the external drive, restart qbt(1), close qbt(2), restart qbt(2) again, note the states. Close qbt(3), re-insert the external drive, restart qbt(3), note the states again. Are they Queued / Seeding or are they zero'd out / errored / downloading / paused?
The "Recheck torrents on completion" is disabled on my end.
After some digging it's possible this isn't exactly what I think it is.
When you move a completed torrent, qbt would list it as "Errored". This is the expected behavior.
In some cases I've noticed that qbt would list a completed-but-moved torrent in the "Downloading" section at 0.0% again, even though it has done the job already. It basically forgets the torrent's status. I'm yet to pinpoint what causes this.
It's possible that this causes the recheck issue. I have to do more testing.
@a-raccoon That looks like the steps to reproduce a different kind of issue. Is it already reported? If so, link it here and I'll check it out, if not, please create one and post the contents of that comment there. Let's keep this comment chain focused on the specific issue in the OP.
@eddlang
The "Recheck torrents on completion" is disabled on my end.
After some digging it's possible this isn't exactly what I think it is.
When you move a completed torrent, qbt would list it as "Errored". This is the expected behavior.
In some cases I've noticed that qbt would list a completed-but-moved torrent in the "Downloading" section at 0.0% again, even though it has done the job already. It basically forgets the torrent's status. I'm yet to pinpoint what causes this.
It's possible that this causes the recheck issue. I have to do more testing.
So this issue can be closed, right? If you are able to reproduce the situation you are describing, post another issue with the steps to reproduce it.
I modified the steps. Now I can reproduce it every time.
The extension option does seem to play the main role here. It causes completed-but-moved torrents to be wrongly listed under the "Downloading" section at 0.0%, instead of "Errored".
The recheck option doesn't work when this happens.
I have to point out that sometimes there isn't even any need to enable/disable the option to trigger this issue. Leaving the option enabled without touching it is enough to cause the Errored/Downloading bug. It seems to be random.
I can reproduce with the new steps but only if incomplete path is also used. It rechecks just fine with just .!qB.
Shouldn't we be locking Completed torrents as, well, Completed. Forever. Unless the user clicks some "Download Again" option that doesn't exist yet? Or unless a Force Recheck yields 0.1% to 99% (but not 0%) completeness.