Server: Google Drive 403 user rate exceeded error

[thefirstofthe300]

I am having an issue with Google Drive not allowing a large number of files to be uploaded in a short time span. Whenever I end up syncing a bunch of files with the desktop client, I get 403 user rate exceeded errors. I believe that Owncloud pushed a fix for this in an upgrade to the google-api-php-client-1.0.17 (??? I know it was from previously 1.0.6-beta).

Steps to reproduce

1. Add Google Drive external storage
2. Upload a large number of files (I am using audiobook files)
3. Watch the sync process from desktop client
   

   Expected behaviour

Files should be uploaded without issue.

Actual behaviour

Nextcloud throws 403 user rate exceeded errors.

Server configuration

Ubuntu 16.04

Web server: apache
Database: mariadb 10.0.25

PHP version: PHP 7.0.4-7ubuntu2.1

Nextcloud version: Nextcloud 9.0.50 (stable)

Updated from an older Nextcloud/ownCloud or fresh install: update from ownCloud 9.0.2

Where did you install Nextcloud from: download page

*_Signing status (Nextcloud 9.0 and above): *_

for another issue

List of activated apps:

Enabled:

• activity: 2.2.1
• admin_audit: 1.0.0
• calendar: 1.2.2
• comments: 0.2
• contacts: 1.3.1.0
• dav: 0.1.6
• documents: 0.12.0
• federatedfilesharing: 0.1.0
• federation: 0.0.4
• files: 1.4.4
• files_external: 0.5.2
• files_pdfviewer: 0.8.1
• files_sharing: 0.9.1
• files_texteditor: 2.1
• files_trashbin: 0.8.0
• files_versions: 1.2.0
• files_videoplayer: 0.9.8
• firstrunwizard: 1.1
• gallery: 14.5.0
• music: 0.3.11
• notifications: 0.2.3
• provisioning_api: 0.4.1
• systemtags: 0.2
• tasks: 0.9.1
• templateeditor: 0.1
• updatenotification: 0.1.0
  Disabled:
• encryption
• external
• user_external
• user_ldap

The content of config/config.php:

{
"system": {
"updatechecker": false,
"instanceid": "ocg20z26yv0r",
"passwordsalt": "_REMOVED SENSITIVE VALUE_",
"secret": "_REMOVED SENSITIVE VALUE_",
"trusted_domains": [
"localhost",
"_REMOVED SENSITIVE VALUE_"
],
"datadirectory": "\/var\/www\/owncloud\/data",
"overwrite.cli.url": "http:\/\/localhost",
"dbtype": "mysql",
"version": "9.0.50.0",
"dbname": "owncloud",
"dbhost": "localhost",
"dbtableprefix": "oc_",
"dbuser": "_REMOVED SENSITIVE VALUE_",
"dbpassword": "_REMOVED SENSITIVE VALUE_",
"logtimezone": "UTC",
"installed": true,
"memcache.local": "\OC\Memcache\APCu",
"mail_from_address": "noreply",
"mail_smtpmode": "php",
"mail_domain": "_REMOVED SENSITIVE VALUE_",
"theme": "",
"loglevel": 2,
"maintenance": false,
"htaccess.RewriteBase": "\/"
}
}

Are you using external storage, if yes which one: Google drive

Are you using disk encryption: no

Are you using an external user-backend, if yes which one: nope

Client configuration

Browser:

Operating system: Linux

Logs

Nextcloud log (data/owncloud.log)

    {"reqId":"SpvcNyIVOBuzVSIMhjF5","remoteAddr":"***REMOVED SENSITIVE VALUE***","app":"index","message":"Exception: {\"Exception\":\"Google_Service_Exception\",\"Message\":\"Error calling GET https:\\\/\\\/www.googleapis.com\\\/drive\\\/v2\\\/files?q=title%3D%2733+-+Track+33.mp3%27+and+%270BzU7zWzP6q-TTjBBQjVRLTE3MzA%27+in+parents+and+trashed+%3D+false: (403) User Rate Limit Exceeded\",\"Code\":403,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/google-api-php-client\\\/src\\\/Google\\\/Http\\\/REST.php(44): Google_Http_REST::decodeHttpResponse(Object(Google_Http_Request))\\n#1 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/google-api-php-client\\\/src\\\/Google\\\/Client.php(556): Google_Http_REST::execute(Object(Google_Client), Object(Google_Http_Request))\\n#2 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/google-api-php-client\\\/src\\\/Google\\\/Service\\\/Resource.php(195): Google_Client->execute(Object(Google_Http_Request))\\n#3 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/google-api-php-client\\\/src\\\/Google\\\/Service\\\/Drive.php(1783): Google_Service_Resource->call('list', Array, 'Google_Service_...')\\n#4 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/lib\\\/google.php(123): Google_Service_Drive_Files_Resource->listFiles(Array)\\n#5 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/lib\\\/google.php(332): OC\\\\Files\\\\Storage\\\\Google->getDriveFile('Audiobooks\\\/Suza...')\\n#6 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/storage\\\/common.php(101): OC\\\\Files\\\\Storage\\\\Google->filetype('Audiobooks\\\/Suza...')\\n#7 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/lib\\\/google.php(642): OC\\\\Files\\\\Storage\\\\Common->is_file('Audiobooks\\\/Suza...')\\n#8 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/storage\\\/wrapper\\\/wrapper.php(367): OC\\\\Files\\\\Storage\\\\Google->hasUpdated('Audiobooks\\\/Suza...', 1465627086)\\n#9 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/storage\\\/wrapper\\\/wrapper.php(367): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->hasUpdated('Audiobooks\\\/Suza...', 1465627086)\\n#10 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/storage\\\/wrapper\\\/availability.php(390): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->hasUpdated('Audiobooks\\\/Suza...', 1465627086)\\n#11 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/storage\\\/wrapper\\\/wrapper.php(367): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Availability->hasUpdated('Audiobooks\\\/Suza...', 1465627086)\\n#12 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/storage\\\/wrapper\\\/wrapper.php(367): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->hasUpdated('Audiobooks\\\/Suza...', 1465627086)\\n#13 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/cache\\\/watcher.php(124): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->hasUpdated('Audiobooks\\\/Suza...', 1465627086)\\n#14 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/view.php(1277): OC\\\\Files\\\\Cache\\\\Watcher->needsUpdate('Audiobooks\\\/Suza...', Object(OC\\\\Files\\\\Cache\\\\CacheEntry))\\n#15 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/view.php(1315): OC\\\\Files\\\\View->getCacheEntry(Object(OCA\\\\Files_Trashbin\\\\Storage), 'Audiobooks\\\/Suza...', '\\\/GoogleDrive\\\/Au...')\\n#16 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/files\\\/filesystem.php(874): OC\\\\Files\\\\View->getFileInfo('\\\/dansey\\\/files\\\/G...', true)\\n#17 \\\/var\\\/www\\\/owncloud\\\/core\\\/ajax\\\/preview.php(51): OC\\\\Files\\\\Filesystem::getFileInfo('\\\/GoogleDrive\\\/Au...')\\n#18 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/route\\\/route.php(154) : runtime-created function(1): require_once('\\\/var\\\/www\\\/ownclo...')\\n#19 [internal function]: __lambda_func()\\n#20 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/route\\\/router.php(276): call_user_func('\\\\x00lambda_1568', Array)\\n#21 \\\/var\\\/www\\\/owncloud\\\/lib\\\/base.php(950): OC\\\\Route\\\\Router->match('\\\/core\\\/preview.p...')\\n#22 \\\/var\\\/www\\\/owncloud\\\/index.php(39): OC::handleRequest()\\n#23 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/google-api-php-client\\\/src\\\/Google\\\/Http\\\/REST.php\",\"Line\":79}","level":3,"time":"2016-06-14T21:29:47+00:00","method":"GET","url":"\/core\/preview.png?file=%2FGoogleDrive%2FAudiobooks%2FSuzanne+Collins+-+The+Hunger+Games%2F33+-+Track+33.mp3&c=575bb20df37d9&x=32&y=32&forceIcon=0","user":"dansey"}

[blizzz]

Maybe you refer to this fix https://github.com/owncloud/core/pull/24799 ? This (quite huge PR) is present in master only. Normally, 3rd party libs were not being updated in stable releases.

[thefirstofthe300]

Yes. That is the fix I am referring to. Are you saying that commit is not
included in thr Nextcloud stable release? If so, I will hold off until you
guys get a fix for it pushed to stable.

On Tue, Jun 14, 2016, 3:24 PM blizzz [email protected] wrote:

Maybe you refer to this fix owncloud/core#24799
https://github.com/owncloud/core/pull/24799 ? This (quite huge PR) is
present in master only. Normally, 3rd party libs were not being updated in
stable releases.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/nextcloud/server/issues/99#issuecomment-226034471,
or mute the thread
https://github.com/notifications/unsubscribe/ADA88hkZOcjeRGy6xalQJiVrV7E0ECCIks5qLyoAgaJpZM4I1zW6
.

[LukasReschke]

The referenced PR seems in fact to do that: https://github.com/owncloud/core/issues/20481#issuecomment-222988250

Considering that this basically breaks Google Drive integration and is a very limited subset I'd advocate a backport to stable9 in this case. This can be easily tested by verifying whether the Google Drive integration works or not.

Tentatively setting to 9.1

[LukasReschke]

@karlitschek Any objections? Currently Google Drive integration is basically borked for a lot of users, we can pretty safely just backport the update of the Google Drive connector library and check if it continues to work then. More than broken isn't possible so the situation won't get worse…

[karlitschek]

agreed. please backport 👍

[MorrisJobke]

Nothing for 9.0.54: moved to .55

[mreiland]

Has there been an update to this? It looks like it wsa removed from the nextcloud 9 milestone, are there still plans on fix this issue?

This basically borks my entire reason for using nextcloud, so I'm very interested in what can be done to get this fixed, even if it means doing it myself.

[ChristophWurst]

Could you please check out https://apps.nextcloud.com/apps/files_external_gdrive?

[MorrisJobke]

As this is a separate app: please report it in that repo https://github.com/NastuzziSamy/files_external_gdrive/