Django-storages: Clear fails with S3Boto3Storage

[WhyNotHugo]

Trying to clear a S3Boto3Storage fails (at least when using it for static files):

$ django-admin collectstatic  -c
[    INFO 17:08:26] raven.contrib.django.client.DjangoClient:215  Raven is not configured (logging is disabled). Please see the documentation for more information.

You have requested to collect static files at the destination
location as specified in your settings.

This will DELETE ALL FILES in this location!
Are you sure you want to do this?

Type 'yes' to continue, or 'no' to cancel: yes
Traceback (most recent call last):
  File "/usr/local/bin/django-admin", line 11, in <module>
    sys.exit(execute_from_command_line())
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 199, in handle
    collected = self.collect()
  File "/usr/local/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 106, in collect
    self.clear_dir('')
  File "/usr/local/lib/python3.6/site-packages/django/contrib/staticfiles/management/commands/collectstatic.py", line 233, in clear_dir
    if not self.storage.exists(path):
  File "/usr/local/lib/python3.6/site-packages/storages/backends/s3boto3.py", line 478, in exists
    self.connection.meta.client.head_object(Bucket=self.bucket_name, Key=name)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 312, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 579, in _make_api_call
    api_params, operation_model, context=request_context)
  File "/usr/local/lib/python3.6/site-packages/botocore/client.py", line 634, in _convert_to_request_dict
    api_params, operation_model)
  File "/usr/local/lib/python3.6/site-packages/botocore/validate.py", line 291, in serialize_to_request
    raise ParamValidationError(report=report.generate_report())
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid length for parameter Key, value: 0, valid range: 1-inf

[aguynamedben]

+1. I experienced this error and it sent me on a 1-hour red herring because I suspected the cause was more intricate logic in collectstatic. Then I realized it was just the --clear option.

FWIW, I was using a hybrid backend like this one

```py

storage.py

class StaticStorage(ManifestFilesMixin, S3Boto3Storage):
pass

settings.py

STATICFILES_STORAGE = 'storage.StaticStorage'
``

[WhyNotHugo]

Just to update this: I didn't find a simple way around this.

I ended up using whitenoise and cloudfront for static content.

[aguynamedben]

@WhyNotHugo Thanks for the tip, whitenoise + cloudfront and just skipping S3 looks like a good option, I'll look at that.

[petrboros]

Is there any development on this issue? We are still experiencing the problem. Thank you very much.

[serjant]

The problem here too, so I am currently using manual aws command to clear the assets on the CDN:

aws s3 rm s3://<bucket name> --recursive

[jcarizza]

I'm using django-storages==1.9 and Python 3.6 with Boto3.

I investigated this issue and for unknown reasons, a file with the name '' (empty) gets until this function .exists().

When the function try to check the object with Key='' in the bucket it raises an error that is not captured by the except ClientError, this exception is botocore.exceptions.ParamValidationError.

Why an empty name gets so far? I think it should be validated before calling.exists()

https://github.com/jschneier/django-storages/blob/0a93eef19c96b2f7705d23251ecb1ca918fc6b40/storages/backends/s3boto3.py#L556