Gdrive: Sync upload creates name collisions

Created on 5 Apr 2016 · 34Comments · Source: prasmussen/gdrive

I tried to sync upload a rather large folder, and it got interrupted. After attempting to restart, it reports name collisions between several files and sub folders.

Source

ryanhle

Most helpful comment

just write a script to fix that

#!/bin/bash
while [ true ]; do
    status=`gdrive sync upload /path/to/folder gdriveFolderID`
    if grep "Found name collision between" <<< $status; then
        statusLine=`grep "Found name collision between" <<< $status | sed "s/.*Found name/Found name/g"`
        echo "found same file name!!"
        echo $statusLine
        file=`sed "s/Found name collision between //g" <<< $statusLine | sed "s/\ and\ /,/g"`
        file1=`cut -d ',' -f 1 <<< $file`
        # file2=`cut -d ',' -f 2 <<< $file`

        echo "same file: $file"
        echo "deleting file: $file1"
        gdrive delete $file1
    else
        echo "not filename collision"
        echo "message:"
        echo $status
        break
    fi
    sleep 10
done

slow but works

dd-han on 23 Nov 2016

👍10 🎉2 ❤1

All 34 comments

Hi, i'm having almost the same issue. I synced a rather large folder with recursion and all went fine, but when i wanted to do another sync of the same folder i encountered name collisions. I checked the google drive and found there were files there with identical names but different file id. Could they have been synced twice?

robertthiemann on 6 Apr 2016

@ryanhle you need to provide a conflict resolution flag. You probably want to use --keep-local in your case, which will overwrite the file on drive.

prasmussen on 9 Apr 2016

😕1

@robertthiemann are you using sync upload or just upload -r? upload -r does not have any syncing features and will always create new files on drive.

prasmussen on 9 Apr 2016

😕1

Hi Petter,
I'm using sync upload. Somehow during the first upload, two files with the same name but different id's got created on the google drive.

robertthiemann on 10 Apr 2016

@prasmussen let me know if you need any logs of screenshots or something. I like this software a lot but at the moment i cant sync with it. Let me know if i can help fix this ok?

robertthiemann on 15 Apr 2016

I am having this issue
Collecting local and remote file information...
Found name collision between 0Bxjv_mh2V96OTm1wdl9EU1ZjckU and 0Bxjv_mh2V96OaGNoSDJTRGV4akU
and here is command im using gdrive -c /home/alonebfg/.gdrive/ sync upload --keep-local --timeout 300 /home/alonebfg 0Bxjv_mh2V96OcG41TGRBaG5TRTg
here is file info
looking at the file info both file are in same location sam md5 and also tring to upload the same file.

alonebfg on 7 Jun 2016

👍1

Hello there,

I'm having the same issue. While doing 'sync upload' the process stops and then when retrying I receive the error:

Found name collision between X and Y

Doing 'gdrive info X' and 'gdrive info Y' the files have the same name, path and m5sum.

Using --keep-local does not make any difference.

Any updates on this?

pdelteil on 1 Jul 2016

I have the same problem. Would be great if there was a flag to overwrite the file when a collision was detected

jeroenqui on 6 Jul 2016

👍2

Same issue here.

MDXDave on 16 Jul 2016

Same issue here. Would love a resolution to this. --keep-local or --keep-remote doesn't help.

/usr/local/bin/gdrive2 sync upload --keep-local dirname 0B4p_knkv9C1UWHZfYTYxV3h1TWc Starting sync... Collecting local and remote file information... Found name collision between 0B4p_knkv9C1UMXJsNzlDQ2swdXM and 0B4p_knkv9C1USzd4Y3N4bHkyUjQ

petercoxphoto on 21 Jul 2016

Same here: providing --keep-local didn't help. I just did a sync upload that went on for some hours. Then the connection was interrupted. When I restarted (without changing any files on both local or online) the "Collecting local and remote file information..." process is interrupted after some seconds with "Found name collision between x and y" I'd really love to get that back to work: The directory is too big to sync it again all the time if something fails.

proligde on 26 Jul 2016

I've got some additional info that might(?) help debugging this issue. I've came across this problem several times now. The setup is quite basic:

initial sync upload to a fresh and empty target on google drive
The sync runs for quite some hours without problem
Then the sync suddenly hangs without any error message, without network traffic and without exiting. I cancelled the sync manually.
After trying to recover just by entering the same sync command again the last file (where it stopped) is on gdrive twice with the same name producing that name collission.
Deleting one of the files (you have to delete and remove it from trash as well) fixes the issue:

Here is my shell output where it hung

[0801/23191] Uploading Fotografie/2014/2014-06-07/IMG_0838.CR2 -> photo/Fotografie/2014/2014-06-07/IMG_0838.CR2 [0802/23191] Uploading Fotografie/2014/2014-06-07/IMG_0839.CR2 -> photo/Fotografie/2014/2014-06-07/IMG_0839.CR2 [0803/23191] Uploading Fotografie/2014/2014-06-07/IMG_0840.CR2 -> photo/Fotografie/2014/2014-06-07/IMG_0840.CR2 [0804/23191] Uploading Fotografie/2014/2014-06-07/IMG_0841.CR2 -> photo/Fotografie/2014/2014-06-07/IMG_0841.CR2 [0805/23191] Uploading Fotografie/2014/2014-06-07/IMG_0842.CR2 -> photo/Fotografie/2014/2014-06-07/IMG_0842.CR2

trying to recover shows:

Starting sync... Collecting local and remote file information... Found name collision between 0B1JkX4q8SB67NG9odmtsMjdvdVE and 0B1JkX4q8SB67dHRFYWdxQjRYS0E

completey deleting one of the two files (IMG_0842.CR2) fixes that:

`Starting sync...
Collecting local and remote file information...
Found 39595 local files and 17209 remote files

22386 remote files are missing
[0001/22386] Uploading Fotografie/2014/2014-06-07/IMG_0843.CR2 -> photo/Fotografie/2014/2014-06-07/IMG_0843.CR2`
[...]

proligde on 10 Aug 2016

👍1

Any news on this? I'm in the same situation and I'm a bit worried there were no follow up in months

dflasse on 24 Oct 2016

After a lot of issues with gdrive ended up using 'syncdocs'. It's slow but it's 100% accurate.

pdelteil on 24 Oct 2016

Hi! I had the same issue... I tried to use --keep-remote and --keep-local but didn't work

mauricioss777 on 27 Oct 2016

same problem
it show Found name collision between 0B8n4VsDeE1egMks4aUlUdnpYUUU and 0B8n4VsDeE1ega2UycXZDTUhJaDQ

then I run gdrive info0B8n4VsDeE1egMks4aUlUdnpYUUU and gdrive info 0B8n4VsDeE1ega2UycXZDTUhJaDQ for getting information

that I found already two file with same file name.Then I delete (not move to trash can) one of them, It Works find

dd-han on 28 Oct 2016

Same issue here. I suggest adding a new function to auto scan and delete another file. eg gdrive merge.
Or make a argument called --ignore-conflict.

ALiangLiang on 16 Nov 2016

@ALiangLiang the problem is that gdrive itself creates these two files having different IDs but the same name. If this bug gets fixed there wouldn't be a name collision/conflict in the first place. I'd try to fix it but I'm not into go (yet).

proligde on 17 Nov 2016

just write a script to fix that

#!/bin/bash
while [ true ]; do
    status=`gdrive sync upload /path/to/folder gdriveFolderID`
    if grep "Found name collision between" <<< $status; then
        statusLine=`grep "Found name collision between" <<< $status | sed "s/.*Found name/Found name/g"`
        echo "found same file name!!"
        echo $statusLine
        file=`sed "s/Found name collision between //g" <<< $statusLine | sed "s/\ and\ /,/g"`
        file1=`cut -d ',' -f 1 <<< $file`
        # file2=`cut -d ',' -f 2 <<< $file`

        echo "same file: $file"
        echo "deleting file: $file1"
        gdrive delete $file1
    else
        echo "not filename collision"
        echo "message:"
        echo $status
        break
    fi
    sleep 10
done

slow but works

dd-han on 23 Nov 2016

👍10 🎉2 ❤1

@dd-han: Yes! This worked for me.

pincushman on 30 Nov 2016

This is a pretty serious problem. Can someone fix it?

valkjsaaa on 20 Jan 2017

I'm not familiar with Go but I suspect there's something wrong with the following line:

https://github.com/prasmussen/gdrive/blob/master/drive/sync_upload.go#L271

Maybe it needs to check whether the folder (in this case) is already exist before tries again?

valkjsaaa on 20 Jan 2017

I am having this same problem. I have an IP camera shell script that grabs images, uses imagemagick to make a 3 image to one snapshot, and does this every 1/2 second. I had a sync setup using gdrive with the --delete-extraneous option, which worked fine for 2 weeks then just completely stopped removing the remote files. Even though I have each file uniquely named with the format "push_YYMMDDHHMM.mp4" (see https://drive.google.com/open?id=0B6ScTEIoK3tPZTVpUnRlZ3dBZ28 ) -- gdrive is (1) failing to delete any files when the local files are deleted after the 3 AM job that merges them all into a single MP4 and removes the temp MP4s, and (2) fails to upload new files due to this name error.

f00dl3 on 24 Feb 2017

If someone is looking for how to reproduce this:

For me this error was caused by stopping the script (ctrl+c) twice in the middle of a large sync upload. I.e. start, wait for a few files to upload, stop, start, wait, stop, start. The third start is where I encountered this issue. Although I suspect it is a bit random doing this repeatedly might produce this error eventually.

7yl4r on 9 Mar 2017

Serious issue needs to be fixed.

3noch on 21 Jun 2017

Moved to rclone

MDXDave on 26 Jun 2017

👍1

also moved to rclone. Just follow the this tutorial: https://rclone.org/drive/

lukas-vollmer on 29 May 2019

👍1

This still remains an issue... . 3 years later.. :(

kooscode on 16 Jul 2019

I found the source:
https://github.com/gdrive-org/gdrive/blob/master/drive/sync.go

Looking into changing this, and perhaps need to fork this project, as it feels to be dead.

Power2All on 16 Jul 2019

Nevermind, I switched to rclone as well.

Power2All on 17 Jul 2019

I found the way to solve this problem.
Just need to login to the google drive in the browser, and find the problem folder or file .
If it is folder, use the below URL
https://drive.google.com/drive/u/0/folders/*****

If it is a file, use the below URL
https://drive.google.com/open?id=*****

** is the folder or file parent id like 0B8n4VsDeE1egMks4aUlUdnpYUUU
And then delete two folder or file as the error message mention.
That will fix the problem.