urllib3 1.24 Just came out a couple days ago and some other influential projects are updating their requirements to allow it. requests just released a version allowing it today. Since pip isn't incredibly graceful in its dependency negotiation (basically doesn't do it) this can cause some confusing installation conflicts for projects that use both boto3 and requests depending on the order that dependencies are resolved. Since both libraries seem pretty popular I'm thinking that's a lot of projects.
The specific error we were seeing was
botocore 1.12.26 has requirement urllib3<1.24,>=1.20, but you'll have urllib3 1.24 which is incompatible.
Thoughts?
kjschiroo
All 4 comments
Looking at the change logs I don't think bumping our range should be an issue.
joguSD
on 18 Oct 2018
I'm curious as to why this version pin exists at all. Once every couple of months, this breaks our requires.io deployment automation; I've never seen it prevent a real problem.
urllib3 seems to purport to some kind of semver-ish scheme. Could this pin be more broadly relaxed to, for example, <2.0 if an upper bound is really necessary?
glyph
on 19 Oct 2018
@glyph This version pin is a relatively new addition. Up until a couple months ago we vendored both urllib3 and requests. Now botocore has a direct dependency on urllib3, and this is the first time urllib3 has released an update since we added it. Ultimately, this pin is to protect from implicit upgrades breaking things. Dropping support for 2.6 or 3.3 is something that we consider a breaking change. This pin gives us time to ensure that upgrades to dependencies doesn't break anything intentionally or not. We've been bit quite a few times by implicit dependency upgrades breaking our packaged installers for the AWS CLI, so we're a little cautious when it comes to version ranges. Ideally, this wouldn't be an error as there is a version that both packages support.
joguSD
on 20 Oct 2018
@joguSD Thanks for the clarification. Perhaps I'm confusing it with a different upper bound.
Personally my belief is that a better way to enforce this is to have a version pin (or set of pins) in requirements.txt which your CI uses to test things, and then something like requires.io to send you automated PRs which bump all the versions of everything. This way you can do development against deterministic dependencies and don't get caught unawares by having to suddenly fix a dependency in order to get an unrelated PR to pass its CI, but also you get a regular PR every time one of your dependencies does a release, so you can fix it immediately.
Then, don't bother setting upper bounds in setup.py, because in the absence of a pip dependency resolver, this constraint doesn't properly constrain users' installed environments, it just causes random errors depending on which order they happen to install things in.
glyph
on 20 Oct 2018
Related issues
ThisGuyCodes
路
5Comments
kapilt
路
5Comments
thehesiod
路
5Comments
stig
路
3Comments
stephanebruckert
路
3Comments
Most helpful comment
@joguSD Thanks for the clarification. Perhaps I'm confusing it with a different upper bound.
Personally my belief is that a better way to enforce this is to have a version pin (or set of pins) in
requirements.txtwhich your CI uses to test things, and then something like requires.io to send you automated PRs which bump all the versions of everything. This way you can do development against deterministic dependencies and don't get caught unawares by having to suddenly fix a dependency in order to get an unrelated PR to pass its CI, but also you get a regular PR every time one of your dependencies does a release, so you can fix it immediately.Then, don't bother setting upper bounds in
setup.py, because in the absence of a pip dependency resolver, this constraint doesn't properly constrain users' installed environments, it just causes random errors depending on which order they happen to install things in.