Pip: have_directory_for_build AssertionError when installing with --no-cache-dir (19.0.1)

[cjerdonek]

• pip version: 19.0.1

With the 19.0.1 release, some people are still reporting an AssertionError when installing with --no-cache-dir similar to the one in #6158. This issue is for the 19.0.1-specific issue as it is narrower in scope.

Here are details for a couple reports: https://github.com/pypa/pip/issues/6158#issuecomment-456882072 and https://github.com/pypa/pip/issues/6158#issuecomment-456949839

The traceback looks like this:

Exception:
Traceback (most recent call last):
  File "/mnt/jenkins/workspace/venv/local/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/mnt/jenkins/workspace/venv/local/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 346, in run
    session=session, autobuilding=True
  File "/mnt/jenkins/workspace/venv/local/lib/python2.7/site-packages/pip/_internal/wheel.py", line 886, in build
    assert have_directory_for_build
AssertionError

Some work-arounds (feel free to add to the list):

• Use pip version 18.1.
• Pass --no-use-pep517 in your invocation.
• See issue #6158 for some other possibilities.

[pradyunsg]

ISTM this is only happening on Alpine Linux -- could someone confirm whether that is the case?

[vinayan3]

@pradyunsg I had problems as well. Please see my dup issue #6192

[dknepprath]

Same but Python 3.7

Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
    status = self.run(options, args)
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 346, in run
    session=session, autobuilding=True
  File "/usr/local/lib/python3.7/site-packages/pip/_internal/wheel.py", line 886, in build
    assert have_directory_for_build
AssertionError

Reverting to pip 18.1 worked for me.

[hrw]

ISTM this is only happening on Alpine Linux -- could someone confirm whether that is the case?

debian:stretch-backports official image:

INFO:kolla.common.utils.openstack-base:Traceback (most recent call last):
INFO:kolla.common.utils.openstack-base:  File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 176, in main
INFO:kolla.common.utils.openstack-base:    status = self.run(options, args)
INFO:kolla.common.utils.openstack-base:  File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 346, in run
INFO:kolla.common.utils.openstack-base:    session=session, autobuilding=True
INFO:kolla.common.utils.openstack-base:  File "/var/lib/kolla/venv/local/lib/python2.7/site-packages/pip/_internal/wheel.py", line 886, in build
INFO:kolla.common.utils.openstack-base:    assert have_directory_for_build
INFO:kolla.common.utils.openstack-base:AssertionError

[gaborbernat]

@pfmoore I ran into this myself, basically just passing --no-cache breaks pip when discovering and installing a PEP-517 package. We need either to fallback to disable PEP-517 build, or instead use the TMPDIR env var.

Hitting this from jython that does not support pip cache mechanism inside virtualenv.

[eliasbrange]

Easily reproducable in docker:

# Same for 3.5.6, 3.6.8, etc.
FROM python:3.7.2-alpine

RUN pip3 install -U pip

RUN pip3 install --no-cache-dir pendulum

[gaborbernat]

The error is very cryptic here, I think pip should check before invoking wheel if the folder exists and raise a better readable error what folder is missing. At the moment this fails with an assertion error leaving the user clueless what folder is missing, making it harder to track down (e.g. incorrectly set pip cache dir, HOME env var, etc). It also does not state for which package build it failed (important if the install pulls in a lot of dependencies first).

[cosmith]

Using --no-use-pep517 didn't work for me, downgrading to 18.1 did (using FROM python:3.6-alpine)

[pradyunsg]

Using --no-use-pep517 didn't work for me, downgrading to 18.1 did (using FROM python:3.6-alpine)

Could you provide us a reproducing case for this issue?

[cjerdonek]

FYI, for the provided Dockerfile example, it is failing for:
<Link https://files.pythonhosted.org/packages/5b/57/71fc910edcd937b72aa0ef51c8f5734fbd8c011fa1480fce881433847ec8/pendulum-2.0.4.tar.gz#sha256=cf535d36c063575d4752af36df928882b2e0e31541b4482c97d63752785f9fcb (from https://pypi.org/simple/pendulum/) (requires-python:>=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*)>
This means, for example, that the link.is_artifact check returns True:
https://github.com/pypa/pip/blob/e5f4bbb7ddff87f48f2b5815513e4211ccdde83a/src/pip/_internal/wheel.py#L876-L879
If it were instead a VCS url, it looks like the assertion would succeed.

What is the purpose of the assertion, btw? Should the above case be working for a link like that?

[cjerdonek]

So I'm wondering if this logic might be incorrect / is the culprit:
https://github.com/pypa/pip/blob/d9de37d20ee35a8afd0e3c24834809b8c7c37ee8/src/pip/_internal/wheel.py#L884-L886

It seems like ephem_cache should also be set to True if that condition isn't true and no cache directory is available.

[cjerdonek]

FYI, I added a failing unit test for this issue here: https://github.com/pypa/pip/pull/6219 (based on the Dockerfile example provided in this comment above)

Does anyone familiar with this part of the code know how the logic in compute_ephem_cache() should be changed to make the have_directory_for_build assertion succeed?

[pradyunsg]

I responded on the PR -- I'll try to make time to look into this sometime soon.

[ncoghlan]

@pradyunsg May be worth pinning this one as well, as I ran into it while trying to test the PRs for #6163 (see https://github.com/pypa/pip/pull/6212#issuecomment-458166386)

[pradyunsg]

Donezo.

[pradyunsg]

@cjerdonek figured out a nice fix for this in #6219 and that's now merged. I'll make a 19.0.2 in the coming week, with that fix and more.

I'll go ahead and close this issue. I'll ping here when the release is made.

[pradyunsg]

pip 19.0.2 has been released with a fix for this.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.