Pip-tools: pip-sync broken with pip 8.1.2

[beedub]

pip-sync doesn't work with pip==8.1.2

Steps to replicate

1. Upgrade to pip 8.1.2
2. Run pip-sync
   

   Expected result

• No error
• Dependencies installed
  

  Actual result

Error: AttributeError: 'Requirement' object has no attribute 'key'

[nhanb]

Same issue with pip-compile:

$ pip-compile --output-file requirements.txt requirements.in
Traceback (most recent call last):
  File "/home/vagrant/virtualenvs/myapp/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/piptools/scripts/compile.py", line 164, in cli
    results = resolver.resolve()
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/piptools/resolver.py", line 78, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/piptools/resolver.py", line 147, in _resolve_one_round
    constraints = sorted(self.constraints, key=_dep_key)
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/piptools/resolver.py", line 49, in constraints
    self.their_constraints)))
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/piptools/resolver.py", line 118, in _group_constraints
    for _, ireqs in full_groupby(constraints, key=_dep_key):
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/piptools/utils.py", line 97, in full_groupby
    return groupby(sorted(iterable, key=key), key=key)
  File "/home/vagrant/virtualenvs/myapp/lib/python3.4/site-packages/piptools/resolver.py", line 27, in _dep_key
    return ireq.req.key
AttributeError: 'Requirement' object has no attribute 'key'

I'm using:

• pip-tools 1.6.4
• pip 8.1.2
• python 3.4.2

[georgexsh]

with pip 8.1.2 under python 2.7.9 breaks too.

[sivabudh]

+1

[nvie]

pip made some changes to its internal data structures in 8.1.2 it seems. I'm having a look to see if there's a quick fix for this or if it's more fundamental.

[nvie]

Hmm, yeah these changes affect pip-tools at the core. We need to carefully inspect how these data structures were refactored and how we need to follow those new patterns. For now, I'll address this by issuing a warning that 8.1.2 is not supported in pip-tools 1.6.*, since I lack the time to properly fix this now.

I'll change the codebase to match 8.1.2, but it won't be a backward compatible fix, so I'll roll a new minor release for it when it's done.

This will be the situation:

• pip-tools < 1.7 will only support pip < 8.1.2
• pip-tools >= 1.7 will only support pip >= 8.1.2

[nvie]

pip-tools==1.6.5 is now available which errors with a more helpful message than a random stack trace with internals.

[cjerdonek]

@nvie Out of curiosity, have you considered articulating to the pip folks what needs to be part of their public API so that pip-tools can have forward compatibility?

[cjerdonek]

@kiorky Do you have a sense for whether the information needed by pip-tools and salt can sensibly be made available through pip's stable command-line API to prevent issues like this in the future? Or through some other stable interface?

[voidus]

in pypa/pip#3573, it is suggested to use pypa/packaging directly, as pip is not meant to be used programmatically. Maybe that would be an option?
https://github.com/pypa/pip/issues/3673#issuecomment-218539154

[gjcooper]

I've come across this before, and downgrading pip has fixed it for me, but just updated to the latest version of Ubuntu and am getting the same error (AttributeError: 'Requirement' object has no attribute 'key') with versions:

Python 3.5.1+
pip-compile, version 1.6.5
pip 8.1.1

Within a pyvenv virtual environment. Possibly related to being on new python release.

[jmbowman]

@nvie Think it's safe to close this now, or giving it some time to hear feedback on the new release?

[cjerdonek]

Worked for me. Thanks! 🎉

[nvie]

Should be fixed in pip-tools==1.7, which was released yesterday!

[pdecat]

Works for me, thanks!

[aaugustin]

Works for me as well, thanks @jmbowman and @nvie!

[timmyomahony]

I'm not sure if I'm doing something wrong, but I'm still getting this error with pip==8.1.2 and pip-tools==1.6 (and python==3.5):

Traceback (most recent call last):
  File "/Users/user/Development/Sites/myproject/bin/pip-compile", line 11, in <module>
    sys.exit(cli())
  File "/Users/user/Development/Sites/myproject/lib/python3.5/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/user/Development/Sites/myproject/lib/python3.5/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/user/Development/Sites/myproject/lib/python3.5/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/user/Development/Sites/myproject/lib/python3.5/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/user/Development/Sites/myproject/lib/python3.5/site-packages/piptools/scripts/compile.py", line 128, in cli
    existing_pins[ireq.req.project_name.lower()] = ireq
AttributeError: 'Requirement' object has no attribute 'project_name'

Downgrading pip to 8.1.1 seems to fix it.

[jmbowman]

@timmyomahony The fix was released with pip-tools 1.7, so if you're still on 1.6 I wouldn't expect it to work.