Gdrive: Constant 403s

[hamptonmoore]

Failed to get file: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
Pretty much anytime I run gdrive download I end up with this

[jvfiel]

Same issue, any ideas?

[hamptonmoore]

I just switched to rclone and it fixed everything :P

[LINKIWI]

The author hardcoded his or her own client ID and secret directly in the source:

https://github.com/prasmussen/gdrive/blob/c3cbcceedd6beb1fcff30f06ea7be7c29558d181/handlers_drive.go#L17

Which means, if you're using the compiled binary published in this repository's releases page, you are effectively sharing the same quota with everyone else who downloaded a binary. Google APIs make a distinction between rate limit errors and user rate limit errors. In this case the error is most likely "not your fault" but rather a symptom of a lot of people using gdrive at the same time.

You can enable the Drive API in your own account, create a set of credentials, replace it in the source above, and recompile. Then the API-level quota will be consumed by only yourself.

[BaconCatBug]

This is weird because I have the same problem when I run the command via crontab, but if I run it manually it works.

[calisir]

@LINKIWI I did the same things as you instructed. However, after I changed my ClientID and ClientSecret I got this error:

Failed to create directory: Post https://www.googleapis.com/drive/v3/files?alt=json: oauth2: cannot fetch token: 401 Unauthorized
Response: {
  "error": "invalid_client",
  "error_description": "Unauthorized"
}

Note that before compilation I moved _build-all.sh,print_usage_markdown.sh_, and _upload.sh_ to the gdrive folder where I can succesfully compile with the old ClientID and ClientSecret.

[internalkernel]

Seeing this issue also... Even seems to happen for simple operations like creating a folder. Also noticing that this bug has been lingering since 2017... I guess Rclone is the answer... Thanks anyways.

[smx06]

@gdrive + @LINKIWI:

"Drive API in your own account, create a set of credentials"
Which kind of app need I create?
1) web app creation gives oauth2 client ID and secret (like in handlers_drive.go) but when replacing with em - "gdrive about" says web app can't access - "you need a native app"
2) Android app creation gives oauth2 client ID but doesn't give secret!
3) Other app type - doesn't work

So which is correct? or am I missing something?

[smx06]

NB: @gdrive + @LINKIWI:
2) if creating Android app and placing just client-ID (no valid secret)
it says
Enter verification code: 4/4ABWn1qjGi5pbgxk9wWpkWGnl66GKkcwg0a47_RW22hnyUXlSlmfP9s
Failed getting oauth client: Failed to exchange auth code for token: oauth2: cannot fetch token: 401 Unauthorized
Response: {
"error": "invalid_client",
"error_description": "Unauthorized"
}

[LINKIWI]

What do you mean by "doesn't work"?

[smx06]

Yesterday "other" app type produced google error on creating.
Today works fine and new gdrive works! Thank you!

[smx06]

pps To speed things up binary editing can be used (to replace Id and secret in the built executable)

[scrumbee]

Would anyone be abel to help me out either explain step by step how edit the binary or how to compile a version for me that would work with ubuntu 16.04. im willing to even pay a little for help if your willing talk me trough and help privately. Like the amount of 403 errors these days just to point of this tool not being useful anymore. But yea at end of the day best solution would be to move this portion to a config file so everyone change change it themselves.

[smx06]

You can build from sources like all do. About editing just use any binary editor (like ghex2, hexedit, etc.)
Find string
1qsNodXNaWq1mQuBjUjmvhoO
and replace it with your Secret string
find string
367116221053-7n0vf5akeru7on6o2fjinrecpdoe99eg.apps.googleusercontent.com
and replace it with your Google clientId

[mbenlioglu]

Available workaround described in #426. Possible solutions will be investigated.

[grandeto]

Rebuild the repo with your own credentials.
Tested on Ubuntu 16.04

**use sudo if needed on your enviroment

apt-get install git
apt-get update
apt install software-properties-common
add-apt-repository ppa:git-core/ppa # apt update; apt install git
// Install go
wget https://dl.google.com/go/go1.12.linux-amd64.tar.gz
mv go1.12.linux-amd64.tar.gz /usr/local/
cd /usr/local
tar -zxvf go1.12.linux-amd64.tar.gz
cd ~
nano .profile
// Add these global variables as follow at the end of the file:
export GOROOT=/usr/local/go
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH
// Update current shell session:
source ~/.profile
// Check go version
go version
// Login to google developers and create a new project. Give it name for instance Gdrive-my-project
https://console.developers.google.com/apis/dashboard
// Go to google drive api and enable it. Then click on Credentials
https://console.developers.google.com/apis/library/drive.googleapis.com
// From dropdown choose Other UI and check User data
// Then click on "What credentials do I need?"
// Give again same name Gdrive-my-project and save
// If old gdrive installation exists:
cd ~
rm -r gdrive
rm -r .gdrive
rm /usr/local/bin/gdrive
// Get the repo
go get github.com/gdrive-org/gdrive
cd go/src/github.com/gdrive-org/gdrive/
nano handlers_drive.go
// On line 17 and 18 change the credentials (ClientId and ClientSecret) with those created in google drive api
// Compile the executable by run ‘go build’ in go/src/github.com/gdrive-org/gdrive/
go build
install gdrive /usr/local/bin/gdrive
install gdrive ~/go/bin/gdrive
// Run some gdrive command to setup the gdrive permissions
gdrive about

[southbot]

I rebuilt gdrive and still got the 403 error.
One of the possible causes is that your machines network speed is very fast and exceeds the maximum speed allowed for uploads. I wrote a bash script that fixed it for me. Note that I first create a list of file names that I wish to upload using ls *.jpg >list.txt. The are other ways of doing it without a list but by using a list you can pick up where you have left off by deleting the names that have already been uploaded, if the upload does not succeed.

[southbot]

!/bin/bash

FILES="list.txt"
while read f ; do
MAXTRIES=4
COUNT=0
BACKOFF=1
while [ $COUNT -lt $MAXTRIES ];do
gdrive upload --parent "put your google drive directory id here without quotes" $f
if [ $? -eq 0 ];then
break
else
let COUNT=COUNT+1
let BACKOFF=BACKOFF*2
echo "Attempt $COUNT failed retrying after $BACKOFF seconds"
sleep $BACKOFF
fi
done
if [ $COUNT -ge $MAXTRIES ];then
echo "Upload failed at $f"
break
fi
done < $FILES

[southbot]

Use this if your list contains directories...

!/bin/bash

FILES="list.txt"
OLD_DIR=""
while read f ; do
MAXTRIES=4
COUNT=0

BACKOFF=1 
DIR=$(dirname "${f}")
if [ "$DIR" != "$OLD_DIR" ];then
    DCOUNT=0
    while [ $DCOUNT -lt $MAXTRIES ];do
        abc=$(gdrive mkdir $DIR)
        echo abc = $abc
        outt=$(awk -F " " '{print $2}' <<< "$abc")  
        chk=$(awk -F " " '{print $1}' <<< "$abc")
        if [ "$chk" == "Directory" ];then
            break
        else
            let DCOUNT=DCOUNT+1
            sleep 4
        fi
    done
    if [ $DCOUNT -ge $MAXTRIES ];then
        echo "Upload failed at creating $DIR directory" 
        break
    fi      
    OLD_DIR=$DIR
    echo Directory = $DIR
    echo outt = $outt
fi  
while [ $COUNT -lt $MAXTRIES ];do 
    gdrive upload --parent $outt $f
    if [ $? -eq 0 ];then
        break
    else
        let COUNT=COUNT+1 
        let BACKOFF=BACKOFF*2 
        echo "Attempt $COUNT failed retrying after $BACKOFF seconds" 
        sleep $BACKOFF 
    fi 
done
if [ $COUNT -ge $MAXTRIES ];then
    echo "Upload failed at $f" 
    break
fi

done < $FILES

[pablodc00]

@grandeto
thanks for the walkthrough,
I have not experience with go language and I'm geting this error when I try to run
go get github.com/gdrive-org/gdrive
_package context: unrecognized import path "context" (import path does not begin with hostname)_
any clue?
thanks in advance!

[grandeto]

@pablodc00 It seems to be an old go version or go env variable not set correctly issue. checkout here:
https://github.com/golang/dep/issues/1985 or google it

[SalimF]

// Login to google developers and create a new project. Give it name for instance Gdrive-my-project
https://console.developers.google.com/apis/dashboard
// Go to google drive api and enable it. Then click on Credentials
https://console.developers.google.com/apis/library/drive.googleapis.com
// From dropdown choose Other UI and check User data

For people who cant find these instruction easy this is screen shot of how it will looks like .