Server: S3 storage has folder size "pending"

Created on 7 Dec 2017  路  23Comments  路  Source: nextcloud/server

Steps to reproduce

  1. Mount an S3 bucket
  2. Upload Data via Windows client

Expected behaviour

The correct folder size should be displayed.

Actual behaviour

Folder size says "pending" on all levels in the file structure.

image

image

Server configuration

Operating system:
Ubuntu 16.04.3 64bit

Web server:
Apache/2.4.18
nginx/1.11.10

Database:
mysql Ver 14.14 Distrib 5.7.20

PHP version:
PHP-FPM 5.6.32

Nextcloud version: (see Nextcloud admin page)
12.0.4

Updated from an older Nextcloud/ownCloud or fresh install:
Updated to the newest release since ownCloud 9

Where did you install Nextcloud from:
php installation script

Signing status:


Signing status

No errors have been found.

List of activated apps:


App list

Enabled:
  - activity: 2.5.2
  - bruteforcesettings: 1.0.2
  - calendar: 1.5.6
  - comments: 1.2.0
  - contacts: 2.0.1
  - dav: 1.3.0
  - federatedfilesharing: 1.2.0
  - federation: 1.2.0
  - files: 1.7.2
  - files_external: 1.3.0
  - files_pdfviewer: 1.1.1
  - files_sharing: 1.4.0
  - files_texteditor: 2.4.1
  - files_trashbin: 1.2.0
  - files_versions: 1.5.0
  - files_videoplayer: 1.1.0
  - firstrunwizard: 2.1
  - gallery: 17.0.0
  - logreader: 2.0.0
  - lookup_server_connector: 1.0.0
  - mail: 0.7.5
  - nextcloud_announcements: 1.1
  - notifications: 2.0.0
  - oauth2: 1.0.5
  - password_policy: 1.2.2
  - provisioning_api: 1.2.0
  - serverinfo: 1.2.0
  - sharebymail: 1.2.0
  - survey_client: 1.0.0
  - systemtags: 1.2.0
  - theming: 1.3.0
  - twofactor_backupcodes: 1.1.1
  - updatenotification: 1.2.0
  - workflowengine: 1.2.0
Disabled:
  - admin_audit
  - encryption
  - user_external
  - user_ldap

Nextcloud configuration:


Config report

{
    "system": {
        "instanceid": "oc2fyztrch2c",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "domain.com"
        ],
        "datadirectory": "\/home\/andy\/oc-data",
        "overwrite.cli.url": "https:\/\/domain.com",
        "dbtype": "mysql",
        "version": "12.0.4.3",
        "dbname": "cloud_domain_com_1",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "Europe\/Berlin",
        "installed": true,
        "knowledgebaseenabled": false,
        "enable_avatars": true,
        "mail_from_address": "cloud",
        "mail_smtpmode": "php",
        "mail_domain": "domain.com",
        "allow_user_to_change_display_name": true,
        "loglevel": 3,
        "maintenance": false,
        "theme": "",
        "singleuser": false,
        "appstore.experimental.enabled": true,
        "htaccess.RewriteBase": "\/",
        "updater.secret": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "filelocking.enabled": false,
        "redis": {
            "host": "localhost",
            "port": 6379
        },
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "auth.bruteforce.protection.enabled": true
    },
    "apps": {
        "activity": {
            "enabled": "yes",
            "installed_version": "2.5.2",
            "types": "filesystem"
        },
        "announcementcenter": {
            "enabled": "no",
            "installed_version": "1.1.1",
            "ocsid": "173921",
            "signed": "true",
            "types": ""
        },
        "backgroundjob": {
            "lastjob": "39"
        },
        "bruteforcesettings": {
            "enabled": "yes",
            "installed_version": "1.0.2",
            "types": ""
        },
        "calendar": {
            "enabled": "yes",
            "installed_version": "1.5.6",
            "signed": "true",
            "types": ""
        },
        "comments": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "logging"
        },
        "contacts": {
            "enabled": "yes",
            "installed_version": "2.0.1",
            "ocsid": "168708",
            "types": ""
        },
        "core": {
            "OC_Channel": "production",
            "backgroundjobs_mode": "cron",
            "incoming_server2server_share_enabled": "no",
            "installed.bundles": "[\"CoreBundle\"]",
            "installedat": "1459469183.8086",
            "lastcron": "1512672301",
            "lastupdateResult": "[]",
            "lastupdatedat": "1512672131",
            "moveavatarsdone": "yes",
            "oc.integritycheck.checker": "[]",
            "outgoing_server2server_share_enabled": "no",
            "previewsCleanedUp": "1",
            "public_files": "files_sharing\/public.php",
            "public_webdav": "dav\/appinfo\/v1\/publicwebdav.php",
            "remote_caldav": "dav\/appinfo\/v1\/caldav.php",
            "remote_calendar": "dav\/appinfo\/v1\/caldav.php",
            "remote_carddav": "dav\/appinfo\/v1\/carddav.php",
            "remote_contacts": "dav\/appinfo\/v1\/carddav.php",
            "remote_dav": "dav\/appinfo\/v2\/remote.php",
            "remote_files": "dav\/appinfo\/v1\/webdav.php",
            "remote_webdav": "dav\/appinfo\/v1\/webdav.php",
            "repairlegacystoragesdone": "yes",
            "scss.variables": "28b56eb75bff149ec8df6d6bdbcaacfa",
            "shareapi_allow_mail_notification": "yes",
            "shareapi_allow_public_notification": "yes",
            "shareapi_default_expire_date": "yes",
            "shareapi_expire_after_n_days": "30",
            "umgmt_send_email": "false",
            "umgmt_show_backend": "false",
            "umgmt_show_email": "false",
            "umgmt_show_last_login": "true",
            "umgmt_show_storage_location": "true",
            "updater.secret.created": "1506461277",
            "vendor": "nextcloud"
        },
        "dav": {
            "OCA\\DAV\\Migration\\ValueFixInsert_ran": "true",
            "buildCalendarSearchIndex": "yes",
            "enabled": "yes",
            "installed_version": "1.3.0",
            "types": "filesystem"
        },
        "federatedfilesharing": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": ""
        },
        "federation": {
            "autoAddServers": "0",
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "authentication"
        },
        "files": {
            "cronjob_scan_files": "500",
            "enabled": "yes",
            "installed_version": "1.7.2",
            "types": "filesystem"
        },
        "files_antivirus": {
            "av_chunk_size": "1024",
            "av_cmd_options": "",
            "av_host": "",
            "av_infected_action": "only_log",
            "av_mode": "socket",
            "av_path": "\/usr\/bin\/clamscan",
            "av_port": "0",
            "av_socket": "\/var\/run\/clamav\/clamd.ctl",
            "enabled": "no",
            "installed_version": "0.8.0.2",
            "ocsid": "157439",
            "signed": "true",
            "types": "filesystem"
        },
        "files_external": {
            "enabled": "yes",
            "installed_version": "1.3.0",
            "ocsid": "166048",
            "types": "filesystem"
        },
        "files_pdfviewer": {
            "enabled": "yes",
            "installed_version": "1.1.1",
            "ocsid": "166049",
            "types": ""
        },
        "files_sharing": {
            "enabled": "yes",
            "incoming_server2server_share_enabled": "no",
            "installed_version": "1.4.0",
            "outgoing_server2server_share_enabled": "no",
            "types": "filesystem"
        },
        "files_texteditor": {
            "enabled": "yes",
            "installed_version": "2.4.1",
            "ocsid": "166051",
            "types": ""
        },
        "files_trashbin": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "filesystem"
        },
        "files_versions": {
            "enabled": "yes",
            "installed_version": "1.5.0",
            "types": "filesystem"
        },
        "files_videoplayer": {
            "enabled": "yes",
            "installed_version": "1.1.0",
            "types": ""
        },
        "firstrunwizard": {
            "enabled": "yes",
            "installed_version": "2.1",
            "types": "logging"
        },
        "gallery": {
            "enabled": "yes",
            "installed_version": "17.0.0",
            "types": ""
        },
        "galleryplus": {
            "enabled": "no",
            "installed_version": "15.1.1",
            "ocsid": "169116",
            "signed": "true",
            "types": ""
        },
        "logreader": {
            "enabled": "yes",
            "installed_version": "2.0.0",
            "ocsid": "170871",
            "types": ""
        },
        "lookup_server_connector": {
            "enabled": "yes",
            "installed_version": "1.0.0",
            "types": "authentication"
        },
        "mail": {
            "enabled": "yes",
            "installed_version": "0.7.5",
            "signed": "true",
            "types": ""
        },
        "nextcloud_announcements": {
            "enabled": "yes",
            "installed_version": "1.1",
            "types": "logging"
        },
        "notifications": {
            "enabled": "yes",
            "installed_version": "2.0.0",
            "types": "logging"
        },
        "oauth2": {
            "enabled": "yes",
            "installed_version": "1.0.5",
            "types": "authentication"
        },
        "password_policy": {
            "enabled": "yes",
            "installed_version": "1.2.2",
            "types": ""
        },
        "provisioning_api": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "prevent_group_restriction"
        },
        "serverinfo": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": ""
        },
        "sharebymail": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "filesystem"
        },
        "spreedme": {
            "OWNCLOUD_TEMPORARY_PASSWORD_LOGIN_ENABLED": "true",
            "OWNCLOUD_TEMPORARY_PASSWORD_SIGNING_KEY": "",
            "SPREED_WEBRTC_BASEPATH": "\/",
            "SPREED_WEBRTC_IS_SHARED_INSTANCE": "false",
            "SPREED_WEBRTC_ORIGIN": "https:\/\/lm.domain.com:8082\/",
            "SPREED_WEBRTC_SHAREDSECRET": "",
            "enabled": "no",
            "installed_version": "0.3.5",
            "is_set_up": "true",
            "ocsid": "174436",
            "types": ""
        },
        "survey_client": {
            "apps": "no",
            "database": "no",
            "enabled": "yes",
            "encryption": "no",
            "files_sharing": "no",
            "installed_version": "1.0.0",
            "php": "no",
            "server": "no",
            "stats": "no",
            "types": ""
        },
        "systemtags": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "logging"
        },
        "templateeditor": {
            "enabled": "no",
            "installed_version": "0.2",
            "types": ""
        },
        "theming": {
            "cachebuster": "4",
            "color": "#1D2D44",
            "enabled": "yes",
            "installed_version": "1.3.0",
            "name": "1n3xCloud",
            "slogan": "nur wir sind cool!",
            "types": "logging",
            "url": "https:\/\/domain.com"
        },
        "twofactor_backupcodes": {
            "enabled": "yes",
            "installed_version": "1.1.1",
            "types": ""
        },
        "updatenotification": {
            "calendar": "1.4.1",
            "contacts": "1.5.1",
            "core": "9.1.2.2",
            "enabled": "[\"admin\"]",
            "installed_version": "1.2.0",
            "spreedme": "0.3.5",
            "types": ""
        },
        "workflowengine": {
            "enabled": "yes",
            "installed_version": "1.2.0",
            "types": "filesystem"
        }
    }
}

Are you using external storage, if yes which one: local/smb/sftp/...
S3

Are you using encryption: yes/no
no

Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/...
no

Client configuration

Browser:
any

Operating system:
any

Logs

Web server error log


Web server error log

No errors

Nextcloud log (data/nextcloud.log)


Nextcloud log

{"reqId":"XA4tA1FzLYXAm8STtEoS","level":3,"time":"2017-12-07T19:53:05+01:00","remoteAddr":"88.130.52.26","user":"a.martin","app":"PHP","method":"PROPFIND","url":"\/remote.php\/webdav\/Dokumente%20-%20S3","message":"Invalid argument supplied for foreach() at \/var\/www\/vhosts\/domain.com\/httpdocs\/apps\/files_external\/lib\/Lib\/Storage\/AmazonS3.php#291","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/62.0.3202.94 Safari\/537.36","version":"12.0.4.3"}
{"reqId":"XA4tA1FzLYXAm8STtEoS","level":3,"time":"2017-12-07T19:53:05+01:00","remoteAddr":"88.130.52.26","user":"a.martin","app":"PHP","method":"PROPFIND","url":"\/remote.php\/webdav\/Dokumente%20-%20S3","message":"Invalid argument supplied for foreach() at \/var\/www\/vhosts\/domain.com\/httpdocs\/apps\/files_external\/lib\/Lib\/Storage\/AmazonS3.php#291","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/62.0.3202.94 Safari\/537.36","version":"12.0.4.3"}
{"reqId":"XA4tA1FzLYXAm8STtEoS","level":3,"time":"2017-12-07T19:53:05+01:00","remoteAddr":"88.130.52.26","user":"a.martin","app":"PHP","method":"PROPFIND","url":"\/remote.php\/webdav\/Dokumente%20-%20S3","message":"Invalid argument supplied for foreach() at \/var\/www\/vhosts\/domain.com\/httpdocs\/apps\/files_external\/lib\/Lib\/Storage\/AmazonS3.php#291","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/62.0.3202.94 Safari\/537.36","version":"12.0.4.3"}
{"reqId":"ivLnfMFaRFsTZlIbQMSJ","level":3,"time":"2017-12-07T19:53:06+01:00","remoteAddr":"88.130.52.26","user":"a.martin","app":"PHP","method":"GET","url":"\/index.php\/apps\/files\/ajax\/getstoragestats.php?dir=%2FDokumente%20-%20S3","message":"Invalid argument supplied for foreach() at \/var\/www\/vhosts\/domain.com\/httpdocs\/apps\/files_external\/lib\/Lib\/Storage\/AmazonS3.php#291","userAgent":"Mozilla\/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/62.0.3202.94 Safari\/537.36","version":"12.0.4.3"}

1. to develop bug external storage
Source
picture In-eX
👍2

Most helpful comment

I am running into the same problem, running 13 b4

Error

Invalid argument supplied for foreach() at /###REDACTED###/apps/files_external/lib/Lib/Storage/AmazonS3.php#290
picture lesaff on 10 Jan 2018
👍3

All 23 comments

cc @icewind1991

MorrisJobke picture MorrisJobke on 14 Dec 2017

S3 should be a lot smoother with 13. Could you test it again?

MorrisJobke picture MorrisJobke on 14 Dec 2017

After upgrading from 12.0.4 to 13beta3 the problem persisted. A installation with the web installer (12.0.3) fixes the problem, but when I upgrade to 12.0.4 I'm back to my old problem. Note, that I'm using the same database for all versions (not at the same time ofc). I don't have any issues with 13beta3 and a new database.

Is there something wrong with the updater? Or did something get lost after all those small updates I did in the past?

In-eX picture In-eX on 20 Dec 2017

I am running into the same problem, running 13 b4

Error

Invalid argument supplied for foreach() at /###REDACTED###/apps/files_external/lib/Lib/Storage/AmazonS3.php#290
lesaff picture lesaff on 10 Jan 2018
👍3

After trying some stuff out, I noticed that when the root of the S3 bucket only contains folders, then the "Contents" attribute in the result paginator is not present. It should therefore be checked whether the "Contents" attribute is an array before iterating it with foreach similar to the CommonPrefixes.

@TecJon @lesaff For a quick workaround, you can add a dummy file .dummy with conents placeholder to the root of the S3 bucket.

MickVanDuijn picture MickVanDuijn on 17 Feb 2018

Sorry, I just noticed it still does not always show the correct size (it did at least once), but at least the logs don't overflow with Invalid argument supplied for foreach()

Edit: it _sometimes_ shows the correct size on the shared folder, but for folders inside it is still pending.

MickVanDuijn picture MickVanDuijn on 17 Feb 2018

cc @icewind1991

MorrisJobke picture MorrisJobke on 17 Feb 2018

@MickVanDuijn thank you, unfortunately it didn't change for me.

This issue might only be a side issue to a bigger problem with s3 that I am facing. #8299

TecJon picture TecJon on 17 Feb 2018

Issue is still persist with nextCloud 13.0.1.

k0ste picture k0ste on 31 Mar 2018

Same problem here: NC 13.0.2, external storage is DigitalOcean's Spaces. All external folders showing "Pending" and log is flooded with error:

Invalid argument supplied for foreach() at /......./apps/files_external/lib/Lib/Storage/AmazonS3.php#295

jaakj picture jaakj on 29 Apr 2018

Same problem here: NC 13.0.2, external storage is DigitalOcean's Spaces. All external folders showing "Pending" and log is flooded with error:

Invalid argument supplied for foreach() at /......./apps/files_external/lib/Lib/Storage/AmazonS3.php#295

See #8842 and #9403

MorrisJobke picture MorrisJobke on 7 May 2018

I tested 13.0.4. Still in pending state.

selection_001

k0ste picture k0ste on 15 Jun 2018

Same issue on 13.0.4 using snap installation on Ubuntu server.

patricksebastien picture patricksebastien on 30 Jul 2018

Same for 14.0.0.13 for an empty folder

TecJon picture TecJon on 30 Jul 2018
👍2

It seems with a GCP bucket with around 14000 objects, Nextcloud is stuck and can't access the files. Maybe a UTF8 problem?

y0hnn picture y0hnn on 6 Jan 2019

I applied the following patch to fix a bug, where root-level folders have not been identified in case they have been directly uploaded to S3 storage instead of upload via Nextcloud. This also fixed the "pending" size issue in Nextcloud 15.0.5.

https://github.com/nextcloud/server/compare/bugfix/6954/scan-external-s3

mfridge picture mfridge on 18 Mar 2019
❤2

cc @kesselb

MorrisJobke picture MorrisJobke on 19 Mar 2019

Thanks for your feedback @mfridge

There are a few positive reports for this patch. I'll try to do some more testing and get it into 16.

kesselb picture kesselb on 19 Mar 2019

There are a few positive reports for this patch. I'll try to do some more testing and get it into 16.

Is there some PR for that already, because the beta is planned for tomorrow ;)

MorrisJobke picture MorrisJobke on 20 Mar 2019
👍1

Hi, I just want to report the same issue of pending size for S3 external storage.
This bug also makes tones of PUT (400,000 requests per day) requests for even only one user.
The patch @mfridge provided solved this issue for me.

cerebellum92 picture cerebellum92 on 15 Apr 2019
👍1

How can I apply this patch?

crypdick picture crypdick on 2 May 2019

How can I apply this patch?

@mfridge provided the link.
You just edit that file, delete (or command) the line with "-" sign in front of it, and add the line with "+" in front of it.

cerebellum92 picture cerebellum92 on 2 May 2019
👍1

I am using Nextcloud on Hostio.cloud and added S3 as external storage and still see the Pending on the folders

cinghaman picture cinghaman on 12 Jan 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

mxschmitt picture mxschmitt  路  68Comments
hartundweich picture hartundweich  路  71Comments
alvVDC picture alvVDC  路  117Comments
cdauth picture cdauth  路  121Comments
j-ed picture j-ed  路  75Comments