pipenv install --system cannot find python path

[Gronis]

When executing pipenv install --system using pipenv 8.1.2 or older, everything works as normal. When using pipenv version 8.1.3 or newer, pipenv cannot find the path to python.

1. Environment: I run this inside a docker container: tensorflow:1.3.0-gpu-py3 using nvidia-docker
2. Python version: 3.5.2
3. Python location: /usr/bin/python
4. Pipenv version: 8.1.3 or newer

Expected result

No thrown exception

Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock!
Installing dependencies from Pipfile.lock…

Actual result

The following exception is thrown:

Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Error [Errno 2] No such file or directory: '/bin/python' while executing command python setup.py egg_info
Traceback (most recent call last):
  File "/usr/local/bin/pipenv", line 11, in <module>
    sys.exit(cli())
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/vendor/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/cli.py", line 1738, in install
    do_init(dev=dev, allow_global=system, ignore_pipfile=ignore_pipfile, system=system, skip_lock=skip_lock, verbose=verbose, concurrent=concurrent, deploy=deploy)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/cli.py", line 1250, in do_init
    do_lock(system=system)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/cli.py", line 1069, in do_lock
    pre=pre
  File "/usr/local/lib/python3.5/dist-packages/pipenv/utils.py", line 425, in resolve_deps
    resolved_tree.update(resolver.resolve())
  File "/usr/local/lib/python3.5/dist-packages/piptools/resolver.py", line 101, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/usr/local/lib/python3.5/dist-packages/piptools/resolver.py", line 198, in _resolve_one_round
    for dep in self._iter_dependencies(best_match):
  File "/usr/local/lib/python3.5/dist-packages/piptools/resolver.py", line 284, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
  File "/usr/local/lib/python3.5/dist-packages/piptools/repositories/pypi.py", line 170, in get_dependencies
    self._dependencies_cache[ireq] = reqset._prepare_file(self.finder, ireq)
  File "/usr/local/lib/python3.5/dist-packages/pipenv/patched/pip/req/req_set.py", line 634, in _prepare_file
    abstract_dist.prep_for_dist()
  File "/usr/local/lib/python3.5/dist-packages/pipenv/patched/pip/req/req_set.py", line 129, in prep_for_dist
    self.req_to_install.run_egg_info()
  File "/usr/local/lib/python3.5/dist-packages/pipenv/patched/pip/req/req_install.py", line 447, in run_egg_info
    command_desc='python setup.py egg_info')
  File "/usr/local/lib/python3.5/dist-packages/pipenv/patched/pip/utils/__init__.py", line 667, in call_subprocess
    cwd=cwd, env=env)
  File "/usr/lib/python3.5/subprocess.py", line 947, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.5/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: '/bin/python'

Steps to replicate

pip install pipenv==8.1.3
pipenv install --system

[erinxocon]

@Gronis Since we haven't had other reports about this I'm thinking this might be a problem with docker and it's $PATH. Can you tell me the contents of the $PATH in the docker container you are using?

[Gronis]

The path is /usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

[erinxocon]

can you do a which python? I haven't had a chance to download the docker and test.

[Gronis]

/usr/bin/python

[erinxocon]

Hmm I'm going to be doing some docker work later today where I can look into this. Strange!

[erinxocon]

Have you tried with the most recent version

[erinxocon]

I downloaded the docker image and launched a bash shell inside it. I think the issue is due to encodings and environment variables. Can you make sure the following env variables are set? Once I did that, the error went away.

export LC_ALL=C.UTF-8
export LANG=C.UTF-8

[erinxocon]

it should be noted that I didn't get the error you reported, but I am using the latest release of pipenv.

[Gronis]

Hi again, sorry if I might not have provided enough details about the problem. The issue has not been resolved, since I already had those environment variables set.

I'm building a docker image based on tensorflow:1.3.0-gpu-py3
This is the Dockerfile:

FROM tensorflow/tensorflow:1.3.0-gpu-py3

ENV LC_ALL=C.UTF-8 \
    LANG=C.UTF-8 \
    PIPENV_HIDE_EMOJIS=1

COPY Pipfile /notebooks/Pipfile

# Installs all dependencies
RUN pip install pipenv==8.2.7 && \
    pipenv install --system

Using this dockerfile and then executing docker build result in the following error in my initial post. Using pip install pipenv==8.1.2 instead of pip install pipenv==8.2.7 works however.

This is my Pipfile

[[source]]
verify_ssl = true
url = "https://pypi.python.org/simple"

[requires]
python_version = '3.6'

[packages]
scipy = "==0.18.1"
xlrd = "==1.1.0"
xlsxwriter = "==0.9.9"

[dev-packages]
tensorflow = "==1.3.0"

[techalchemy]

So this is fixed in the latest release of Pipenv?

On Oct 20, 2017, at 4:27 AM, Robin Grönberg [e-mail] wrote:

Hi again, sorry if I might not have provided enough details about the problem. The issue has not been resolved, since I already had those environment variables set.

I'm building a docker image based on tensorflow:1.3.0-gpu-py3
This is the Dockerfile:

FROM tensorflow/tensorflow:1.3.0-gpu-py3

ENV LC_ALL=C.UTF-8 \
LANG=C.UTF-8 \
PIPENV_HIDE_EMOJIS=1

COPY Pipfile /notebooks/Pipfile

Installs all dependencies

RUN pip install pipenv==8.2.7 && \
pipenv install --system
Using this dockerfile and then executing docker build result in the following error in my initial post. Using pip install pipenv==8.1.2 instead of pip install pipenv==8.2.7 works however.

This is my Pipfile

[[source]]
verify_ssl = true
url = "https://pypi.python.org/simple"

[requires]
python_version = '3.6'

[packages]
scipy = "==0.18.1"
xlrd = "==1.1.0"
xlsxwriter = "==0.9.9"

[dev-packages]
tensorflow = "==1.3.0"
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[erinxocon]

No, provided with the above two files, I can reproduce the issue. Looking for the cause.

[erinxocon]

Try using this dockerfile, it works for me.

FROM tensorflow/tensorflow:1.3.0-gpu-py3

ENV LC_ALL=C.UTF-8 \
    LANG=C.UTF-8 \
    PIPENV_HIDE_EMOJIS=1

RUN set -ex && pip install pipenv --upgrade

COPY Pipfile notebooks/Pipfile
RUN set -ex && pipenv install --system

[Gronis]

Hi @erinxocon , the reason it works for you, is because you are missing the slash before notebooks, and pipenv does not find the Pipfile (since it is not in working directory). For your Dockerfile, it does not crash, but the result is a docker container without any dependencies from the Pipfile.

This Dockerfile does not work: (e.g with /notebooks rather than notebooks)

FROM tensorflow/tensorflow:1.3.0-gpu-py3

ENV LC_ALL=C.UTF-8 \
    LANG=C.UTF-8 \
    PIPENV_HIDE_EMOJIS=1

RUN set -ex && pip install pipenv --upgrade

COPY Pipfile /notebooks/Pipfile
RUN set -ex && pipenv install --system

I have also tested this Dockerfile with pipenv version 8.3.2 and the bug still persist.

[rdehouss]

I also have the issue. Using 8.1.2 does not have the issue.

[techalchemy]

Pretty sure this will relate to #1002

[cegprakash]

should reopen this issue

[Jitsusama]

Ok, I've done some investigating, and I've been able to make it work in a way that's not too hacky. As an aside, my main reason for doing this is because pipenv doesn't work on my Windows PC due to a corporate firewall that thinks it's a virus :-P

Here are my steps for reproducing the issue:

# docker run --rm -it --entrypoint /bin/bash python:3.6-slim
root@adbc1b6c3752:/# mkdir app && cd app
root@adbc1b6c3752:/app# echo '[[source]]
> url = "https://pypi.python.org/simple"
> verify_ssl = true
> name = "pypi"
>
> [packages]
> requests = "*"
>
> [requires]
> python_version = "3.6"' > Pipfile
root@adbc1b6c3752:/app# pip install -q --upgrade pipenv
root@adbc1b6c3752:/app# pipenv install --system
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
/python: not found

I tried to track down what was giving this error and had no luck, then I discovered that there is a verbose option to pipenv install. I used that and had the following results:

root@adbc1b6c3752:/app# pipenv install -v --system
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…

/bin/sh: 1: /bin/python: not found

I tried to figure out which script referenced /bin/python, but to no avail. So I thought, well, I can fix this easy enough:

root@adbc1b6c3752:/app# ln -sf $(which python) /bin/python
root@adbc1b6c3752:/app# pipenv install --system
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (7b8df8)!
Installing dependencies from Pipfile.lock (7b8df8)…
  🐍   ▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉▉ 5/5 —

When adding the symbolic link function into my Dockerfile, I'm able to do this all dynamically. I understand from @kennethreitz that this goes against the very principle of pipenv, mainly being that when you deploy your application, you have the versions already pinned. I agree with him. In my case, I'm simply using this method to generate my Pipfile.lock since I can't generate it on my development workstation.

If you are interested though, here is what the Dockerfile could look like:

FROM python:3.6-slim

WORKDIR /app
COPY Pipfile ./
RUN set -x \
 && ln -sf /usr/local/bin/python /bin/python \
 && pip install --upgrade pipenv==11.9.0 \
 && pipenv install --system