Google-cloud-python: Storage: 'Blob.download_as_string' fails with strange error message

[ekeleshian]

Environment details

Using Ubuntu 18.04, python 3.6.5, google-cloud-storage 1.13.1

Steps to reproduce

I created a bucket to store a file that I will access later with google cloud functions. I authenticated to a Cloud API just as explained in Google's Getting Started documentation. I made sure the bucket creation actually happened by running gsutil du -s gs://[bucket_id] on the terminal and indeed the byte size is correct. I tried downloading the content as a string, but it failed with the message below. The "NotFound" error links to a URL that I think has a bug.

Code example

storage_client = storage.Client()
bucket = storage_client.get_bucket([bucket_id])
blob = bucket.blob
result = blob.download_as_string()

Stack trace

---------------------------------------------------------------------
InvalidResponse                     Traceback (most recent call last)
~/anaconda3/lib/python3.6/site-packages/google/cloud/storage/blob.py in download_to_file(self, file_obj, client, start, end)
    559         try:
--> 560             self._do_download(transport, file_obj, download_url, headers, start, end)
    561         except resumable_media.InvalidResponse as exc:

~/anaconda3/lib/python3.6/site-packages/google/cloud/storage/blob.py in _do_download(self, transport, file_obj, download_url, headers, start, end)
    497             )
--> 498             download.consume(transport)
    499         else:

~/anaconda3/lib/python3.6/site-packages/google/resumable_media/requests/download.py in consume(self, transport)
    170 
--> 171         self._process_response(result)
    172 

~/anaconda3/lib/python3.6/site-packages/google/resumable_media/_download.py in _process_response(self, response)
    169         _helpers.require_status_code(
--> 170             response, _ACCEPTABLE_STATUS_CODES, self._get_status_code)
    171 

~/anaconda3/lib/python3.6/site-packages/google/resumable_media/_helpers.py in require_status_code(response, status_codes, get_status_code, callback)
     92             response, u'Request failed with status code',
---> 93             status_code, u'Expected one of', *status_codes)
     94     return status_code

InvalidResponse: ('Request failed with status code', 404, 'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 206>)

During handling of the above exception, another exception occurred:

NotFound                            Traceback (most recent call last)
<ipython-input-19-7f68b8f7e283> in <module>()
----> 1 result = blob.download_as_string()

~/anaconda3/lib/python3.6/site-packages/google/cloud/storage/blob.py in download_as_string(self, client, start, end)
    619         """
    620         string_buffer = BytesIO()
--> 621         self.download_to_file(string_buffer, client=client, start=start, end=end)
    622         return string_buffer.getvalue()
    623 

~/anaconda3/lib/python3.6/site-packages/google/cloud/storage/blob.py in download_to_file(self, file_obj, client, start, end)
    560             self._do_download(transport, file_obj, download_url, headers, start, end)
    561         except resumable_media.InvalidResponse as exc:
--> 562             _raise_from_invalid_response(exc)
    563 
    564     def download_to_filename(self, filename, client=None, start=None, end=None):

~/anaconda3/lib/python3.6/site-packages/google/cloud/storage/blob.py in _raise_from_invalid_response(error)
   1964     )
   1965 
-> 1966     raise exceptions.from_http_status(response.status_code, message, response=response)
   1967 
   1968 

NotFound: 404 GET https://www.googleapis.com/download/storage/v1/b/[bucket_id]/o/[bucket_id]?alt=media: ('Request failed with status code', 404, 'Expected one of', <HTTPStatus.OK: 200>, <HTTPStatus.PARTIAL_CONTENT: 206>)

[tseaver]

@ekeleshian Thanks for the report. The 404 error indicates that the blob does not exist with the given name inside your bucket. Your example code doesn't show how you set the blob's name. E.g.:

from google cloud import storage

BUCKET_NAME = "<your-bucket-name>"
BLOB_NAME = "<your-blob-name>"

storage_client = storage.Client()
bucket = storage_client.get_bucket(BUCKET_NAME)
blob = bucket.blob(BLOB_NAME)
assert blob.exists()  # Will be false if the next line would raise NotFound
result = blob.download_as_string()

You can check the name(s) of the blobs in your bucket via Bucket.list_blobs(), e.g.:

blob_names = [blob.name for blob in bucket.list_blobs()]

Or via gsutil:

$ gsutil ls -l gs://[bucket_id]

[kylefoley76]

The 404 error indicates that the blob does not exist with the given name inside your bucket. -- Then why didn't the developer write that in his error message. It would have saved everyone a lot of time.

[GunK194]

I too get the same error, even though the file exists. Can anyone check the code
storage_client = storage.Client()
bucket = storage_client.get_bucket(bucketname)

destination_blob_pathname = zipfilename_with_path

blob = bucket.blob(destination_blob_pathname)
blob_names = [blob.name for blob in bucket.list_blobs()]
#zipbytes = io.BytesIO(blob.download_as_string())
for i in blob_names:
    print(i)
zipbytes = io.BytesIO(blob.download_as_string()) --- This gives error

[shkumesh]

I'm having the same error as GunK194 mentioned, any solution for that?

[siphon-app]

Same issue here.

[reynierhdez]

Same issue here.

[jenildesai25]

Still seeing same issue even though file is present. Any workaround?

[arafatsh]

I have the same issue though the file exists , what I have noticed is that when I write the file name manually it downloads but when the filename is in a dict or list it gives the error,,,,

[tseaver]

@arafatsh, @jenildesai25, @siphon-app, @shkumesh, @GunK194, @kylefoley76 The google-cloud-storage library is no longer being maintained in this repository: please use the new repository's tracker to report issues.