Django-storages: GCP and NamedTemporaryFile: file-like object only had 0 bytes remaining.

Created on 14 Dec 2018  路  2Comments  路  Source: jschneier/django-storages

import tempfile
import requests
from django.core import files

def process_image(url):
    """Process a single image."""
    r = requests.get(url, stream=True)
    file_name = url.split('/')[-1]

    # read response stream into lf
    lf = tempfile.NamedTemporaryFile()
    for block in r.iter_content(1024 * 8):
        if not block:
            break
        lf.write(block)
    lf.flush()

    # save to google cloud
    media_image = BrandMediaImage()
    media_image.image.save(file_name, files.File(lf), save=False) # error here
    media_image.save()

Error:

  File "/app/apps/jobs/management/commands/mycommand.py", line 89, in process_image
    media_image.image.save(file_name, files.File(lf), save=False)
  File "/usr/local/lib/python3.6/site-packages/django/db/models/fields/files.py", line 87, in save
    self.name = self.storage.save(name, content, max_length=self.field.max_length)
  File "/usr/local/lib/python3.6/site-packages/django/core/files/storage.py", line 49, in save
    return self._save(name, content)
  File "/usr/local/lib/python3.6/site-packages/storages/backends/gcloud.py", line 167, in _save
    content_type=file.mime_type)
  File "/usr/local/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 975, in upload_from_file
    size, num_retries, predefined_acl)
  File "/usr/local/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 888, in _do_upload
    size, num_retries, predefined_acl)
  File "/usr/local/lib/python3.6/site-packages/google/cloud/storage/blob.py", line 683, in _do_multipart_upload
    raise ValueError(msg)
ValueError: Size 37319 was specified but the file-like object only had 0 bytes remaining.

Most helpful comment

The file object needs to be at beginning of the file.
Try adding:
lf.seek(0)

All 2 comments

The file object needs to be at beginning of the file.
Try adding:
lf.seek(0)

@sww314 replacing lf.flush() with lf.seek(0) fixes this problem, thanks!

Also files.File(lf) is not necessary, just passing lf directly is enough.

Was this page helpful?
0 / 5 - 0 ratings