Pipenv: Confusing error message when run in Dockerfile

[OddBloke]

When experimenting in a Dockerfile, I hit an odd error message:

Creating a virtualenv for this project…
usage: pew [-h] [-p PYTHON] [-i PACKAGES] [-r REQUIREMENTS] [-d] [-a PROJECT]
           envname
pew: error: the following arguments are required: envname

Virtualenv location: 
Creating a Pipfile for this project…
Creating a virtualenv for this project…
usage: pew [-h] [-p PYTHON] [-i PACKAGES] [-r REQUIREMENTS] [-d] [-a PROJECT]
           envname
pew: error: the following arguments are required: envname

Virtualenv location: 
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
An unexpected error occurred while accessing your virtualenv's python installation!
Please run $ pipenv --rm to re-create your environment.

Obviously a Dockerfile is a pathological environment to be running in (and I've moved on to resolve the issue), but it feels like pipenv should abort much earlier than it did

Describe your environment

1. OS Type: ubuntu:16.04
2. Python version: 3.5
3. Pipenv version: latest from PyPI

Expected result

Error message explaining the actual issue.

Actual result

Above error message.

Steps to replicate

Attempt to build this Dockerfile:

FROM ubuntu:16.04

RUN apt-get update && apt-get install -y python3-pip
RUN pip3 install pipenv

ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8

RUN pipenv install --deploy --verbose

[kennethreitz42]

closing and will see if this comes up with anyone else.

[aljp]

I had a similar issue and it was just that PYTHONPATH and PYTHON_VERSION environment variables weren't set in the docker container.

An example of my dockerfile can be found at: https://gist.github.com/aljp/811030019a711532eda77cf07408458a

I'm currently using this docker image in a circleci build and it works there, however, when I attempt to run the circleci builds or docker run locally I get the same error.

If you build that image and run docker run -it --rm my/repo:circleci-latest /bin/bash and then run pipenv install --dev you'll see the error:

Creating a virtualenv for this project…
⠋usage: pew [-h] [-p PYTHON] [-i PACKAGES] [-r REQUIREMENTS] [-d] [-a PROJECT]
           envname
pew: error: the following arguments are required: envname

Virtualenv location: 
Creating a Pipfile for this project…
Creating a virtualenv for this project…
⠙usage: pew [-h] [-p PYTHON] [-i PACKAGES] [-r REQUIREMENTS] [-d] [-a PROJECT]
           envname
pew: error: the following arguments are required: envname

Virtualenv location: 
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
An unexpected error occurred while accessing your virtualenv's python installation!
Please run $ pipenv --rm to re-create your environment.

Turns out the docker container was just missing the PYTHONPATH and PYTHON_VERSION environment variables, so running the container with docker run -it --rm -e PYTHONPATH=/usr/local/bin -e PYTHON_VERSION="3.6.1" my/repo:circleci-latest /bin/bash seems to work.

[mccalluc]

Perhaps reopen this? Here's a minimal reproducer:

$ cat Dockerfile 
FROM python:3.6.3
RUN pip install pipenv
RUN pipenv install

$ docker build .
Sending build context to Docker daemon  4.608kB
Step 1/3 : FROM python:3.6.3
 ---> a8f7167de312
Step 2/3 : RUN pip install pipenv
 ---> Using cache
 ---> 14db7304c3a0
Step 3/3 : RUN pipenv install
 ---> Running in b9253f7914a2
Creating a virtualenv for this project…
usage: pew [-h] [-p PYTHON] [-i PACKAGES] [-r REQUIREMENTS] [-d] [-a PROJECT]
           envname
pew: error: the following arguments are required: envname

Virtualenv location: 
Creating a Pipfile for this project…
Creating a virtualenv for this project…
usage: pew [-h] [-p PYTHON] [-i PACKAGES] [-r REQUIREMENTS] [-d] [-a PROJECT]
           envname
pew: error: the following arguments are required: envname

Virtualenv location: 
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
An unexpected error occurred while accessing your virtualenv's python installation!
Please run $ pipenv --rm to re-create your environment.
The command '/bin/sh -c pipenv install' returned a non-zero code: 1

I can't control how it is run, ie I need to make the Dockerfile work on its own, but so far I haven't fixed the problem with ENV statements.

[techalchemy]

@mccalluc @aljp We have an example dockerfile in our project here where we recommend using set -ex and passing the the --system flag to pipenv, because by default pipenv uses pew to manage virtual environments which requires you to change your WORKDIR in your dockerfile as well as to set your WORKON_HOME environment variable.

[tsiq-oliver]

Can we reopen this?

I disagree that Docker is "a pathological environment"! In particular, if Pipenv doesn't work out of the box with an official Python Docker image, something is wrong (though I'm not sure whether it's the image or Pipenv that should be fixed.)

FWIW, here's my repro:

FROM python:3.6.4

RUN pip install pipenv
RUN pipenv --three

Output:

Sending build context to Docker daemon  14.19MB
Step 1/3 : FROM python:3.6.4
 ---> 336d482502ab
Step 2/3 : RUN pip install pipenv
 ---> Using cache
 ---> 91428d6b95e1
Step 3/3 : RUN pipenv --three
 ---> Running in 492b2b8a1779
Creating a virtualenv for this project…
Using /usr/local/bin/python3 (3.6.4) to create virtualenv…
usage: __main__.py [-h] [-p PYTHON] [-i PACKAGES] [-r REQUIREMENTS] [-d]
                   [-a PROJECT]
                   envname
__main__.py: error: the following arguments are required: envname

Virtualenv location:
Creating a Pipfile for this project…
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1043, in invoke
    return Command.invoke(self, ctx)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/pipenv/cli.py", line 152, in cli
    core.ensure_project(three=three, python=python, warn=True, site_packages=site_packages)
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 672, in ensure_project
    ensure_pipfile(validate=validate, skip_requirements=skip_requirements)
  File "/usr/local/lib/python3.6/site-packages/pipenv/core.py", line 321, in ensure_pipfile
    project.create_pipfile(python=python)
  File "/usr/local/lib/python3.6/site-packages/pipenv/project.py", line 416, in create_pipfile
    config_parser = ConfigOptionParser(name=self.name)
  File "/usr/local/lib/python3.6/site-packages/pipenv/vendor/pip9/baseparser.py", line 149, in __init__
    assert self.name
AssertionError
The command '/bin/sh -c pipenv --three' returned a non-zero code: 1

[techalchemy]

@tsiq-oliverc do you have a pipfile that you installed? Pipenv can’t just _run_, it’s a dependency manager, it needs to install things in order to compute dependencies

[tsiq-oliver]

@techalchemy - But pipenv --three is what you run to create a new env in the first place, I think? i.e. equivalent to pipenv --python 3.6 (which also causes this issue, FWIW). At least that's my understanding.

This works fine outside of Docker, so I assumed that was an expected workflow.

[sgrowe]

In case it helps others I was able to solve this issue by setting WORKDIR to /opt/app:

FROM python:3.6.4-stretch

WORKDIR /opt/app

# build your stuff....

[uranusjr]

@sgrowe You made something click for me. This is probably an issue with Python not getting the correct os.getcwd() inside Docker. Actually worth investigating 🤔 Unfortunately we are pretty overloaded already, and this is a low-priority issue to start with. Hopefully this can point contributors to the correct direction if there is interest.

[jtratner]

I thought the core issue is that os.path.basename(‘/‘) == ‘’
On Fri, Mar 30, 2018 at 9:32 AM Tzu-ping Chung [e-mail]
wrote:

@sgrowe https://github.com/sgrowe You made something click for me. This
is probably an issue with Python not getting the correct os.getcwd()
inside Docker. Actually worth investigating 🤔 Unfortunately we are
pretty overloaded already, and this is a low-priority issue to start with.
Hopefully this can point contributors to the correct direction if there is
interest.

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/pypa/pipenv/issues/1328#issuecomment-377564160, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ABhjq6ApzbEeR-CnpaRzOqxe5h8oZLIrks5tjl4SgaJpZM4RlBL-
.

[techalchemy]

@tsiq-oliverc

usage: __main__.py [-h] [-p PYTHON] [-i PACKAGES] [-r REQUIREMENTS] [-d]
                   [-a PROJECT]

You got this error message. From the documentation:

$ pipenv install <package>

You can't just have pipenv --three, that's not a valid command. You haven't told pipenv to install, or run, or uninstall, or <verb> anything. That's why it told you what the usage is as a response. This would be the same as simply typing pip, or docker. Sure, you can tell it what executable or image you want to use, but unless you tell it what you want to do you're not going to get very far.

[tsiq-oliver]

@techalchemy - Also from the docs (the landing page, no less):

To initialize a Python 3 virtual environment, run $ pipenv --three.

It's also featured in the demo GIF in the README.

The CLI help also says:

Create a new project using Python 3.6, specifically:
   $ pipenv --python 3.6

So it's unclear that a verb is necessary.

It sounds like a review of the docs is due, if there's this kind of confusion?

[techalchemy]

@tsiq-oliverc thanks for pointing that out! Possibly we should bring the code in line with this to avoid more confusion. Will consider which approach makes sense

[tsiq-oliver]

@techalchemy FWIW, my opinion is that the ability to create a "blank" environment (starting from nothing) is extremely useful. So I'd prefer that you didn't remove that :/

Perhaps that behaviour could be moved to an explicit init command though? And then if the intent is that there should always be a command/verb, perhaps update cli.py to enforce that.

[geoidesic]

I don't think this is a documentation issue. I'm getting the same problem with the following Dockerfile:

# ---- Base Node ----
FROM alpine:3.7 as base
RUN apk add --no-cache python3 && \
    python3 -m ensurepip && \
    rm -r /usr/lib/python*/ensurepip && \
    pip3 install --upgrade pip setuptools && \
    if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && \
    if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && \
    rm -r /root/.cache

# security update
RUN apk update
RUN apk upgrade -U -a
RUN apk add sqlite
RUN adduser -S djangocms
RUN mkdir /home/djangocms -p
RUN chown djangocms /home/djangocms
# add locales
ENV LANG=en_US.UTF-8 \
    LANGUAGE=en_US.UTF-8 \
    LC_CTYPE=en_US.UTF-8 \
    LC_ALL=en_US.UTF-8
RUN apk add --update --no-cache socat curl tzdata findutils
RUN apk add git
RUN pip install git+https://github.com/pypa/pipenv
RUN cd /home/djangocms
RUN pipenv install djangocms-installer --three

@sgrowe: your solution above works btw but I don't understand why.

[jtratner]

1. pipenv --three definitely is supposed to work (I use it too)
2. Docker problems generally boil down to either (1) / has no name and thus cannot be hashed for generating virtualenv or (2) python version and path problems. So for now, the solution is to make the WORKDIR not be / (which is the default). For context - pipenv hashes the / to figure out virtualenv name, and I think when it doesn't exist it hits a pathological codepath...

[techalchemy]

Also there is a fix in the pipeline (pr is open as of yesterday) which likely helps with this so there isn't much more we can do