Tdesktop: More options when re-downloading existing files [feature request]

[DRSDavidSoft]

Steps to reproduce

1. Save media to the specified 'Telegram Desktop' folder
2. Move the folder to a different path; Change the 'Download path' to the new location
3. Open the already downloaded data
   

   Expected behaviour

The files which are already downloaded should be found in the directory and thus be opened.

Actual behaviour

1. Telegram Desktop thinks that the file in the path is not the intended file
2. Instead of opening the file, it downloads a new copy (but otherwise exact content)

3. Since the file actually exists in the location, Telegram labels the new download to "Filename (2).ext".

   Requests

4. Telegram should be able to understand if the requested file already exists in the 'Download path'. Also, it should check for the checksum hashes (if any)

5. Telegram _should not_ download the file, if it already exists and have the correct checksum hash.
   

   TL;DR

When saving a file, if the file already exists and contains the correct data, _Telegram should open the file,_ instead of _downloading it again, and naming it "File (2).ext"_.

Thank you. Please +1 this if you agree.

[john-preston]

@DRSDavidSoft There is no such checksum got from server to check that the file on the hard drive is the one that we start downloading.

[DRSDavidSoft]

@john-preston Thank you for your quick reply. Here's what would I do if I were in the development team:

1. Add support for checksums. Files already exist on the cloud-server, so a way to get quick checksums in possible, and would actually make sense for file integrity verification.
   Of course, I'm not sure how would this work on end-to-end encryptions, but it could be also done, if not as quite easily.
2. Or just check for the filename, and file size. Doesn't matter if the file is corrupted or not, if the file with the exact name and size exists, open it instead of downloading a new copy.
   In most cases, a file with the exact name and size mean it's good and completed. If there is anything wrong with the data, user could easily delete and re-download it.
3. Add a simple option is settings to prevent downloading files.
   It can be disabled by default, and when enabled it can use the # 2 mechanism.

Please consider this. I think this would be a pretty useful feature for people like me, who has downloaded ~50GBs of data from groups, and moved the download directory because of lack of space; And now Telegram doesn't recognise the downloaded files and retries to re-download them.

---

_On a side note,_ I would fork the the project, modify the code to meet my requirements, compile it and use it. However,

1. Unfortunately, I don't know where to start.
2. If I did, maintaining this feature without a Pull Request would not be reasonable through version updates.

I think Telegram saves the location of downloaded data in a database-like query, named TDIF. I don't know what TDIF is and how to modify it, otherwise I think it would be easy just to replace the path for those files to the new location.

[auchri]

There is no such checksum got from server to check that the file on the hard drive is the one that we start downloading.

Since there is no checksum, it can't be implemented (until the server developer adds such a checksum)

[DRSDavidSoft]

@auchri

1. Or just check for the filename, and file size. Doesn't matter if the file is corrupted or not, if the file with the exact name and size exists, open it instead of downloading a new copy.
   In most cases, a file with the exact name and size mean it's good and completed. If there is anything wrong with the data, user could easily delete and re-download it.
2. [Or] Add a simple option is settings to prevent downloading files.
   It can be disabled by default, and when enabled it can use the # 2 mechanism.

Read before closing :|

[auchri]

Read before closing :|

My fault, sorry

[DRSDavidSoft]

Please remove the api issue label already.
Just add the simple, standard "overwrite, rename, skip" option which is available in many apps.
Telegram Desktop uses "rename" by default, and this is very annoying.
I would set it to "skip", and wouldn't care about the file's integrity. I mean, I can always remove the any corrupted downloads, can't I?

[stek29]

What about creating hashes just after download and storing them locally?

@DRSDavidSoft well, it's really an api issue. But it's also an enhancement.

[DRSDavidSoft]

@stek29 It's actually a good idea to store hashes locally! Though a quick hash algorithm should be chosen for this, because large files take longer on older computers to generate hashes. Also, the hash should only be generated on a successful download.

Please implement this sooner, I am waiting for this to happen before I continue my file transmission in Telegram.

[Royal2000H]

@stek29 How would creating hashes locally help? You'd still need to download the file from the server to generate the hash since the server isn't giving you a hash.

[DRSDavidSoft]

@Royal2000H You're right, we still need the hash from the server. It slipped my mind :|
However, using locally hashes, we already know the correct hash to a downloaded file. Now if the downloaded file still has the same hash, then the data is correct. If not, Telegram should re-download it.

Please imagine this scenario:

1. File 'a' is downloaded from the servers, and saved locally in default folder 'b'.
2. File 'a' is then moved from folder 'b' to folder 'c'.
3. Telegram's default folder is then changed from 'b' to 'c'.
4. File 'a' is requested to open.

Now, as of writing this in the current Telegram version, it will try to re-download file 'a' and name it file 'a (2)' in directory 'c', even though the file already exist and has the same data.
If the hashes were generated, Telegram could avoid re-downloading it by check hash of the local file 'a' in directory 'c', and match it with the one in the db, which is known to be the correct match (because Telegram had saved it on a successful download).

I hope it was clear!

If hashes are not going to be supported by the api, all i am asking is to implement a simple feature to allow skipping files with the same name. Almost every standard download manager have this option!

Edit: I have updated this message.

[Royal2000H]

I can certainly understand the motive and that would be a step in the right direction into solving your problem. However, I don't foresee telegram going that route because after all, they're a messaging system not a download manager. The reason I don't see it happening is because if someone sends you a bunch of photos that are actually all originally named "image.jpg", you can be certain that 99% of users would just want to see the photo, not deal with naming it. Telegram has frequently said they want to steer away from "more options" in the settings menu because that causes more different behaviors and more testing required. That being said I have no affiliation with Telegram whatsoever so this is all speculation on my part.

However, the idea to hash files and pass the hash from the server is a great one, in my opinion. It's definitely something Telegram should implement and include in their API since it won't add any complexity to the user interface and it will potentially save them bandwidth. It would also allow them to save on cloud space by just storing 1 copy of every hashed file rather than 1 for each chat that the same identical file is in (assuming they don't already do that and that the hash is robust enough). Further, if you send a file to a friend that some other Telegram user has already uploaded, it could use the hash and file size to match the file so you don't even have to upload the file at all. Again, the hash must be long enough to ensure no collisions - and this should only apply to cloud chats as the e2e encryption would prevent hashes from matching across different chats. I suggest you make that feature request to Telegram directly rather than just to the Telegram Desktop github.

[Aokromes]

https://github.com/telegramdesktop/tdesktop/issues/747#issuecomment-132626731

[TodeDragos]

Brothers please help. I'm in a telegram group with ~6k audio file uploads.
At this moment i'm forced to re-download all of them :((. Don't know the reason but telegram doesn't know aymore that i've downloaded already exactly same files in the past.
Also the auto-download feature for "flac" files doesn't work at all. Only .aiff and .wav works. It's a pain in the ass because i have tot take each file manually and press the download button.
And as if that weren't enough, this "re-download" problem appeared from nowhere :|
I'm on macOS Mojave right now.

[RememberTheAir]

@TodeDragos use the export tool. Three dots menu -> export chat history

[stale[bot]]

Hey there!

This issue will be automatically closed in 7 days if there would be no activity. We therefore assume that the user has lost interest or resolved the problem on their own.

Don't worry though; if this is an error, let us know with a comment and we'll be happy to reopen the issue.

Thanks!