Pipenv: Pipenv dockerfile fails to install
Looks like my dockerfiles using pipenv are failing at the moment when I try to build them.
Describe your environment
- OS Type: Docker pipenv docker file
- Python version:
$ python -V - Pipenv version:
$ pipenv --version
Expected result
Expected pipenv dockerfile to build and install properly.
Actual result
Docker build fails at pipenv install --deploy --system
---> Running in e0eedb80adab
+ pipenv install --deploy --system
Installing dependencies from Pipfile.lock (1c8a85)β¦
Traceback (most recent call last):
File "/usr/bin/pipenv", line 11, in <module>
load_entry_point('pipenv==11.1.4', 'console_scripts', 'pipenv')()
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3/dist-packages/pipenv/vendor/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3/dist-packages/pipenv/cli.py", line 197, in install
selective_upgrade=selective_upgrade
File "/usr/lib/python3/dist-packages/pipenv/core.py", line 1836, in do_install
pre=pre, requirements_dir=requirements_directory
File "/usr/lib/python3/dist-packages/pipenv/core.py", line 1373, in do_init
requirements_dir=requirements_dir.name)
File "/usr/lib/python3/dist-packages/pipenv/core.py", line 857, in do_install_dependencies
requirements_dir=requirements_dir
File "/usr/lib/python3/dist-packages/pipenv/core.py", line 1474, in pip_install
c = delegator.run(pip_command, block=block)
File "/usr/lib/python3/dist-packages/pipenv/vendor/delegator.py", line 267, in run
c.run(block=block, binary=binary)
File "/usr/lib/python3/dist-packages/pipenv/vendor/delegator.py", line 156, in run
s = PopenSpawn(self._popen_args, **pexpect_kwargs)
File "/usr/lib/python3/dist-packages/pipenv/vendor/pexpect/popen_spawn.py", line 46, in __init__
self.proc = subprocess.Popen(cmd, **kwargs)
File "/usr/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/bin/pip': '/bin/pip'
Steps to replicate
docker build . with a Dockerfile containing
FROM kennethreitz/pipenv
ghop02
All 19 comments
seems plausible, I had an eye on this bug somewhere else. I'll try to get to it today
techalchemy
on 8 Mar 2018
Please provide your Dockerfile, if possible.
kennethreitz42
on 8 Mar 2018
FROM kennethreitz/pipenv
COPY logging_elk/configure_backups.py /app
COPY logging_elk/run_backup.sh /app
The contents are pretty bare :)
Also, here's my Pipfile:
[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"
[requires]
python_version = "3.6.4"
[packages]
arrow = "*"
PyYAML = "*"
invoke = "*"
requests = "*"
[dev-packages]
nose = "*"
mock = "*"
HTTPretty = "*"
freezegun = "*"
The dockerfile recently changed the base image from python3 to ubuntu 17. This broke our build, but in a different way. pipenv install --system no longer uses python3 on ubuntu 17.
jarshwah
on 8 Mar 2018
let's get that sorted.
kennethreitz42
on 8 Mar 2018
FYI you should probably only be using our dockerfile as an example, not as part of your infrastructure :)
kennethreitz42
on 8 Mar 2018
Yes, totally agree! We're just updating our image to use the python 3.6 image now :)
jarshwah
on 8 Mar 2018
I couldn't duplicate this in a fresh container with a fresh build. I have a dockerfile that installs pipenv from the PPA we recently added and I'm now using 11.1.5. Are you guys including a lockfile in your deployments? And yes, note that you need system python to be the one you intend to use, so manipulate your paths accordingly.
techalchemy
on 9 Mar 2018
@techalchemy i think our example Dockerfile needs to update PATH (or make some symlinks) so 'python' == python3.
kennethreitz42
on 9 Mar 2018
I just built and published a new docker image which pre-installs pipenv from the ppa, you can find an example working dockerfile that builds off of it here (just tested it with the pipfile/lockfile in that repository):
https://github.com/techalchemy/pipenv-docker/blob/master/Dockerfile.example
FROM techalchemy/pipenv-base
RUN set -ex && mkdir /app
WORKDIR /app
COPY Pipfile Pipfile
COPY Pipfile.lock Pipfile.lock
RUN set -ex && pipenv install --deploy --system
CMD ['/bin/bash']
I'm also getting a related issue on my Ubuntu 16.04 dockerfile (looking for /bin/pip and /bin/python). To try to fix, I just created symbolic links to the python3.6 and pip3.6 executables. Then, when I go to my project that was previously working, this happens:
$ pipenv install --dev
Creating a virtualenv for this projectβ¦
Using /usr/bin/python3.6m to create virtualenvβ¦
β /usr/bin/python3.6: No module named pipenv.pew
Virtualenv location:
Creating a virtualenv for this projectβ¦
β /usr/bin/python3.6: No module named pipenv.pew
Virtualenv location:
Installing dependencies from Pipfile.lock (b43a47)β¦
An error occurred while installing amqp==2.2.2! Will try again.
An error occurred while installing arxiv-base==0.2.2! Will try again.
An error occurred while installing billiard==3.5.0.3! Will try again.
An error occurred while installing celery==4.1.0! Will try again.
An error occurred while installing click==6.7! Will try again.
An error occurred while installing coverage==4.5.1! Will try again.
An error occurred while installing coveralls==1.3.0! Will try again.
An error occurred while installing docopt==0.6.2! Will try again.
An error occurred while installing flask==0.12.2! Will try again.
An error occurred while installing flask-sqlalchemy==2.3.2! Will try again.
An error occurred while installing flask-wtf==0.14.2! Will try again.
Ignoring isort: markers 'python_version == ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*"' don't match your environment
An error occurred while installing itsdangerous==0.24! Will try again.
An error occurred while installing jinja2==2.10! Will try again.
An error occurred while installing jsonschema==2.6.0! Will try again.
An error occurred while installing kombu==4.1.0! Will try again.
An error occurred while installing markupsafe==1.0! Will try again.
An error occurred while installing mypy==0.570! Will try again.
An error occurred while installing nose2==0.7.4! Will try again.
An error occurred while installing pyjwt==1.6.0! Will try again.
An error occurred while installing pytz==2018.3! Will try again.
An error occurred while installing redis==2.10.6! Will try again.
An error occurred while installing sqlalchemy==1.2.5! Will try again.
Ignoring typing: markers 'python_version < "3.5"' don't match your environment
Ignoring urllib3: markers 'python_version < "3"' don't match your environment
An error occurred while installing uwsgi==2.0.17! Will try again.
An error occurred while installing vine==1.1.4! Will try again.
An error occurred while installing werkzeug==0.14.1! Will try again.
An error occurred while installing wtforms==2.1! Will try again.
An error occurred while installing click==6.7! Will try again.
An error occurred while installing coverage==4.5.1! Will try again.
An error occurred while installing flask==0.12.2! Will try again.
An error occurred while installing flask-testing==0.7.1! Will try again.
Ignoring isort: markers 'python_version == ">=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*"' don't match your environment
An error occurred while installing itsdangerous==0.24! Will try again.
An error occurred while installing jinja2==2.10! Will try again.
π βββββββββββββββββββββββββββββββββ 61/61 β 00:00:21
An error occurred while installing markupsafe==1.0! Will try again.
An error occurred while installing mypy==0.570! Will try again.
An error occurred while installing nose2==0.7.4! Will try again.
An error occurred while installing pydocstyle==2.1.1! Will try again.
An error occurred while installing snowballstemmer==1.2.1! Will try again.
Ignoring typing: markers 'python_version < "3.5"' don't match your environment
An error occurred while installing werkzeug==0.14.1! Will try again.
Installing initiallyβfailed dependenciesβ¦
Collecting amqp==2.2.2 ββββββββββββββ 0/38 β 00:00:00
Using cached amqp-2.2.2-py2.py3-none-any.whl
Installing collected packages: amqp
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/lib/python3.6/dist-packages/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/usr/local/lib/python3.6/dist-packages/pip/req/req_set.py", line 784, in install
**kwargs
File "/usr/local/lib/python3.6/dist-packages/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/usr/local/lib/python3.6/dist-packages/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/usr/local/lib/python3.6/dist-packages/pip/wheel.py", line 345, in move_wheel_files
clobber(source, lib_dir, True)
File "/usr/local/lib/python3.6/dist-packages/pip/wheel.py", line 316, in clobber
ensure_dir(destdir)
File "/usr/local/lib/python3.6/dist-packages/pip/utils/__init__.py", line 83, in ensure_dir
os.makedirs(path)
File "/usr/lib/python3.6/os.py", line 220, in makedirs
mkdir(name, mode)
PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/amqp'
β€ ββββββββββββββββββββββββββββββββ 0/38 β 00:00:00
bbarker
on 9 Mar 2018
Because you are installing pipenv to a /usr/local install but using python in /usr so I suspect you have a path issue here
techalchemy
on 9 Mar 2018
@techalchemy Right, but how can there be a path issue? My understanding is that pipenv is run by some python (could be a system python) and then creates a local python installation in a virtualenv. For some reason, pipenv seems to not be setting the path for such a virtualenv and is using the system python.
bbarker
on 9 Mar 2018
Pipenv doesnβt do magic, _you_ have a path issue. Pipenv doesnβt set the path for you when it runs itself somehow. You need to have your own $PATH set correctly
techalchemy
on 9 Mar 2018
@techalchemy I didn't realize I was suggesting magic happened; pipenv installs packages in a local directory under $HOME normally in some venv. I assume it invokes virtualenv behind the scenes to do so, maybe some extra bits as well to get everything it wants sorted out. Anyway, we are agreed, not magic. The python in the venv is not in the path, I get that. But why is it installing packages in a systems directory (/usr/local) and not in a venv?
I installed pip and pipenv like this (I'm on Ubuntu 16.04 where there seems to be no pip package for the python3.6 I installed from an alternative repo):
RUN ln -s /usr/bin/python3.6m /bin/python && \
wget https://bootstrap.pypa.io/get-pip.py && python3.6 get-pip.py && \
ln -s /usr/local/bin/pip /bin/pip && \
python3.6 -m pip install pipenv
PATH is /bin:/sbin:/usr/bin:/usr/sbin:/usr/local/bin
$ which python
/bin/python
$ ls -last /bin/python
0 lrwxrwxrwx 1 root root 19 Mar 9 14:45 /bin/python -> /usr/bin/python3.6m
$ which pip
/bin/pip
$ ls -last /bin/pip
0 lrwxrwxrwx 1 root root 18 Mar 9 14:45 /bin/pip -> /usr/local/bin/pip
$ pip --version
pip 9.0.1 from /usr/local/lib/python3.6/dist-packages (python 3.6)
$ ls /usr/local/bin/py*
ls: cannot access '/usr/local/bin/py*': No such file or directory
Your observation that there are packages in /usr/local seems to be a result of pip having been installed there, but as we can see there is no actual Python there.
bbarker
on 9 Mar 2018
run this:
$ env
You probably ran sudo python -m pip install --upgrade pip which on distributions never upgrades pip in place, it always upgrades it to /usr/local, you will need to re-link it back to /usr/bin manually for installing pipenv + pip - also, don't link things to /bin/<name> to solve this problem, it will only make it worse (probably).
And yes, once you've done that, the output of env will be informative
techalchemy
on 9 Mar 2018
@techalchemy So, fixed now. I don't know why, but your suggestion of reverting back to what I had earlier without the symbolic links may have been it.
As noted in my earlier message, I installed pip via get-pip.py. Additionally, I can't run sudo in this container as my container user doesn't have sudo privs (I could login as root if I tried, but I know I haven't).
(I changed the above docker snippet back to:)
RUN wget https://bootstrap.pypa.io/get-pip.py && python3.6 get-pip.py && \
python3.6 -m pip install pipenv
This is inspiring me to make a $ python -m pipenv.help command.
kennethreitz42
on 9 Mar 2018
Related issues
AkiraSama
Β·
3Comments
FooBarQuaxx
Β·
3Comments
jakul
Β·
3Comments
ansrivas
Β·
3Comments
bgjelstrup
Β·
3Comments
Most helpful comment
seems plausible, I had an eye on this bug somewhere else. I'll try to get to it today