Google-cloud-python: Google cloud storage needs requests >= 2.18.0 but the setup.py only requires >= 2.4.0

[newt0311]

According to the requests changelog the Response class was turned into a context manager only in version 2.18.0. However in google/resumable_media/requests/download.py line 62 (inside _write_to_stream) the code tries to use "response" as a context manager. Some digging reveals that this response makes its way from Client._http which is defined in google/cloud/client.py and gets the Response object from google/auth/transport/requests.py:AuthorizedSession.

So... it definitely feels like something in here needs to require requests >= 2.18.0 (maybe all of them?).

Thoughts?

Detailed info:

1. OS type and version: N/A
2. Python version and virtual environment information: 2.7 but I don't think it matters.
3. google-cloud-python version: google-cloud-storage v 1.3.1
4. Stacktrace if available: below.
5. Steps to reproduce: Install a version of the requests library _less than 2.18.0_. Get some blob that exists. Try blob.download_as_string(). I personally re-pro'd this with version 2.10.0 to check. Note that the setup.py only requires 2.0.0 for storage and 2.4.0 for client, etc...

-- PG.

[lukesneeringer]

Aha, thanks for reporting @newt0311.

I had a theory that this might have been the root cause of #3736 as I said there this morning (although the subsequent pip freeze he gave shows requests 2.18). You are getting a different error, but I bet it is the same issue -- we are being too loose on our lower bound.

[lukesneeringer]

@jonparrott @dhermes Any issue with bumping our minimum bound to 2.18.0?

[dhermes]

1. I have no issue with bumping the minimum bound
2. I am so grateful to @newt0311 for putting in this legwork.

[theacodes]

I have no issues with bumping the minimum bound.

[sonlac]

@lukesneeringer Yes, thank you. It could be related to my issue #3736. I re-checked and the fact is that I used requests 2.12.3 (I forced it in my requirements.txt) which is required (> 2.0.0) by google-cloud 0.27.0

[dhermes]

@sonlac That would certainly explain the lack of __exit__

[newt0311]

Oh yeah: that __exit__ AttributeError is what I was seeing on my end. I should have attached the stack trace. Must have slipped my mind (apologies). Anyway thanks for taking a look and for the fast fix.

-- PG.

Update: I only noticed this because I was running things on google-cloud-dataflow. I suspect people's local setups tend to be fairly up-to-date so bugs like this get masked...

[lukesneeringer]

@sonlac Yeah, the older version of requests was _almost definitely_ your problem. I am super glad now that I wrote fauxquests back in the day, because I recognized the error immediately (which is why I asked you about it).

[lukesneeringer]

I only noticed this because I was running things on google-cloud-dataflow. I suspect people's local setups tend to be fairly up-to-date so bugs like this get masked...

Yep. pip by default will do the newest it can, so it would only appear for people with an older version already installed in their virtualenv.

[sonlac]

I only noticed this because I was running things on google-cloud-dataflow. I suspect people's local setups tend to be fairly up-to-date so bugs like this get masked...

Not always true. Sometimes, people fix _google-cloud_ to a specific version in _requirements.txt_. So, pip tried to install the latest packages it can. For my case, I fixed _google-cloud_ to _0.24.0_ but pip installed _google-cloud-storage 1.3.0_ for all versions older than google-cloud 0.27.0. The fact that the _google-cloud-storage 1.3.0_ only works with _google-cloud-core 0.26.0_ (google cloud 0.27.0) https://github.com/GoogleCloudPlatform/google-cloud-python/releases/tag/storage-1.3.0. It's is something about "user-aware", so just be careful.