Django-storages: 'SpooledTemporaryFile' object has no attribute 'readinto'

Created on 17 Dec 2020 · 3Comments · Source: jschneier/django-storages

Hi there,

after upgrading django-storages from v1.10.1 to v1.11 I've got the following error:

AttributeError: 'SpooledTemporaryFile' object has no attribute 'readinto'
  File "core/tasks.py", line 349, in issues_progress_report
    url = svc.upload(report)
  File "core/services/issues_progress.py", line 356, in upload
    name = self.storage.save(f'issues-progress/{org}/{file.name}', file)
  File "django/core/files/storage.py", line 52, in save
    return self._save(name, content)
  File "storages/backends/s3boto3.py", line 446, in _save
    obj.upload_fileobj(reader, ExtraArgs=params)
  File "boto3/s3/inject.py", line 619, in object_upload_fileobj
    return self.meta.client.upload_fileobj(
  File "boto3/s3/inject.py", line 539, in upload_fileobj
    return future.result()
  File "s3transfer/futures.py", line 106, in result
    return self._coordinator.result()
  File "s3transfer/futures.py", line 265, in result
    raise self._exception
  File "s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "s3transfer/upload.py", line 692, in _main
    client.put_object(Bucket=bucket, Key=key, Body=body, **extra_args)
  File "botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "botocore/client.py", line 652, in _make_api_call
    handler, event_response = self.meta.events.emit_until_response(
  File "botocore/hooks.py", line 360, in emit_until_response
    return self._emitter.emit_until_response(aliased_event_name, **kwargs)
  File "botocore/hooks.py", line 243, in emit_until_response
    responses = self._emit(event_name, kwargs, stop_on_response=True)
  File "botocore/hooks.py", line 211, in _emit
    response = handler(**kwargs)
  File "botocore/utils.py", line 2219, in conditionally_calculate_md5
    md5_digest = calculate_md5(body, **kwargs)
  File "botocore/utils.py", line 2196, in calculate_md5
    binary_md5 = _calculate_md5_from_file(body)
  File "botocore/utils.py", line 2208, in _calculate_md5_from_file
    for chunk in iter(lambda: fileobj.read(1024 * 1024), b''):
  File "botocore/utils.py", line 2208, in <lambda>
    for chunk in iter(lambda: fileobj.read(1024 * 1024), b''):
  File "s3transfer/utils.py", line 480, in read
    data = self._fileobj.read(amount_to_read)
  File "s3transfer/upload.py", line 86, in read
    return self._fileobj.read(amount)
  File "django/core/files/utils.py", line 17, in <lambda>
    readinto = property(lambda self: self.file.readinto)

I guess it may be caused by this commit. Reverting django-storages back to v1.10.1 fixed the issue. Other package versions:

python==3.9.0
Django==3.1.4
boto3==1.16.38
botocore==1.19.38

if I can provide anything else that may help - please let me know. Thanks!

Source

stebunovd

😕1 👍1

Most helpful comment

Ugh, thanks. I need to invest in an integration test.

On Dec 17, 2020, at 15:57, Denis Stebunov notifications@github.com wrote:

Hi there,

after upgrading django-storages from v1.10.1 to v1.11 I've got the following error:

AttributeError: 'SpooledTemporaryFile' object has no attribute 'readinto'
File "core/tasks.py", line 349, in issues_progress_report
url = svc.upload(report)
File "core/services/issues_progress.py", line 356, in upload
name = self.storage.save(f'issues-progress/{org}/{file.name}', file)
File "django/core/files/storage.py", line 52, in save
return self._save(name, content)
File "storages/backends/s3boto3.py", line 446, in _save
obj.upload_fileobj(reader, ExtraArgs=params)
File "boto3/s3/inject.py", line 619, in object_upload_fileobj
return self.meta.client.upload_fileobj(
File "boto3/s3/inject.py", line 539, in upload_fileobj
return future.result()
File "s3transfer/futures.py", line 106, in result
return self._coordinator.result()
File "s3transfer/futures.py", line 265, in result
raise self._exception
File "s3transfer/tasks.py", line 126, in __call__
return self._execute_main(kwargs)
File "s3transfer/tasks.py", line 150, in _execute_main
return_value = self._main(kwargs)
File "s3transfer/upload.py", line 692, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, *extra_args)
File "botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "botocore/client.py", line 652, in _make_api_call
handler, event_response = self.meta.events.emit_until_response(
File "botocore/hooks.py", line 360, in emit_until_response
return self._emitter.emit_until_response(aliased_event_name, *kwargs)
File "botocore/hooks.py", line 243, in emit_until_response
responses = self._emit(event_name, kwargs, stop_on_response=True)
File "botocore/hooks.py", line 211, in _emit
response = handler(kwargs)
File "botocore/utils.py", line 2219, in conditionally_calculate_md5
md5_digest = calculate_md5(body, **kwargs)
File "botocore/utils.py", line 2196, in calculate_md5
binary_md5 = _calculate_md5_from_file(body)
File "botocore/utils.py", line 2208, in _calculate_md5_from_file
for chunk in iter(lambda: fileobj.read(1024 * 1024), b''):
File "botocore/utils.py", line 2208, in
for chunk in iter(lambda: fileobj.read(1024 * 1024), b''):
File "s3transfer/utils.py", line 480, in read
data = self._fileobj.read(amount_to_read)
File "s3transfer/upload.py", line 86, in read
return self._fileobj.read(amount)
File "django/core/files/utils.py", line 17, in
readinto = property(lambda self: self.file.readinto)
I guess it may be caused by this commit https://github.com/jschneier/django-storages/pull/955/files. Reverting django-storages back to v1.10.1 fixed the issue. Other package versions:

python==3.9.0
Django==3.1.4
boto3==1.16.38
botocore==1.19.38
if I can provide anything else that may help - please let me know. Thanks!

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/jschneier/django-storages/issues/967, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAREDWAE2M6HEQOPRPPEUSLSVJWDBANCNFSM4VAHD7CA.

jschneier on 17 Dec 2020

👍4

All 3 comments

Ugh, thanks. I need to invest in an integration test.

On Dec 17, 2020, at 15:57, Denis Stebunov notifications@github.com wrote:

Hi there,

after upgrading django-storages from v1.10.1 to v1.11 I've got the following error:

AttributeError: 'SpooledTemporaryFile' object has no attribute 'readinto'
File "core/tasks.py", line 349, in issues_progress_report
url = svc.upload(report)
File "core/services/issues_progress.py", line 356, in upload
name = self.storage.save(f'issues-progress/{org}/{file.name}', file)
File "django/core/files/storage.py", line 52, in save
return self._save(name, content)
File "storages/backends/s3boto3.py", line 446, in _save
obj.upload_fileobj(reader, ExtraArgs=params)
File "boto3/s3/inject.py", line 619, in object_upload_fileobj
return self.meta.client.upload_fileobj(
File "boto3/s3/inject.py", line 539, in upload_fileobj
return future.result()
File "s3transfer/futures.py", line 106, in result
return self._coordinator.result()
File "s3transfer/futures.py", line 265, in result
raise self._exception
File "s3transfer/tasks.py", line 126, in __call__
return self._execute_main(kwargs)
File "s3transfer/tasks.py", line 150, in _execute_main
return_value = self._main(kwargs)
File "s3transfer/upload.py", line 692, in _main
client.put_object(Bucket=bucket, Key=key, Body=body, *extra_args)
File "botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "botocore/client.py", line 652, in _make_api_call
handler, event_response = self.meta.events.emit_until_response(
File "botocore/hooks.py", line 360, in emit_until_response
return self._emitter.emit_until_response(aliased_event_name, *kwargs)
File "botocore/hooks.py", line 243, in emit_until_response
responses = self._emit(event_name, kwargs, stop_on_response=True)
File "botocore/hooks.py", line 211, in _emit
response = handler(kwargs)
File "botocore/utils.py", line 2219, in conditionally_calculate_md5
md5_digest = calculate_md5(body, **kwargs)
File "botocore/utils.py", line 2196, in calculate_md5
binary_md5 = _calculate_md5_from_file(body)
File "botocore/utils.py", line 2208, in _calculate_md5_from_file
for chunk in iter(lambda: fileobj.read(1024 * 1024), b''):
File "botocore/utils.py", line 2208, in
for chunk in iter(lambda: fileobj.read(1024 * 1024), b''):
File "s3transfer/utils.py", line 480, in read
data = self._fileobj.read(amount_to_read)
File "s3transfer/upload.py", line 86, in read
return self._fileobj.read(amount)
File "django/core/files/utils.py", line 17, in
readinto = property(lambda self: self.file.readinto)
I guess it may be caused by this commit https://github.com/jschneier/django-storages/pull/955/files. Reverting django-storages back to v1.10.1 fixed the issue. Other package versions:

python==3.9.0
Django==3.1.4
boto3==1.16.38
botocore==1.19.38
if I can provide anything else that may help - please let me know. Thanks!

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub https://github.com/jschneier/django-storages/issues/967, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAREDWAE2M6HEQOPRPPEUSLSVJWDBANCNFSM4VAHD7CA.