Pipenv: No such file or directory: 'pew' on Travis CI

[madhavajay]

https://travis-ci.org/madhavajay/nd889/builds/251710323

pipenv used to work on Travis CI, but now its failing with error:

Creating a virtualenv for this project...
Traceback (most recent call last):
  File "/home/travis/virtualenv/python3.6.1/bin/pipenv", line 11, in <module>
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 722, in __call__
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 697, in main
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 1043, in invoke
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 895, in invoke
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/click/core.py", line 535, in invoke
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/click/decorators.py", line 17, in new_func
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/cli.py", line 779, in cli
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/cli.py", line 161, in ensure_project
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/cli.py", line 154, in ensure_virtualenv
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/cli.py", line 140, in ensure_virtualenv
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/cli.py", line 394, in do_create_virtualenv
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/delegator.py", line 258, in run
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/delegator.py", line 150, in run
  File "/home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages/pipenv/vendor/pexpect/popen_spawn.py", line 45, in __init__
  File "/opt/python/3.6.1/lib/python3.6/subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "/opt/python/3.6.1/lib/python3.6/subprocess.py", line 1326, in _execute_child
    raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'pew'
The command "pipenv --three" failed and exited with 1 during .
Your build has been stopped.

I tried adding pip install pew, but it didn't fix it.

My file is .travis.yml file is

language: python
python:
  - 3.6

install:
  - pip install pipenv pew
  - pipenv --three
  - pipenv install git+https://github.com/hmmlearn/hmmlearn.git
  - pipenv install

script:
  - pipenv run pytest
  - pipenv run mypy 1_sudoku --ignore-missing-imports

My Pipfile is

[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true

[packages]
mypy = "*"
pytest = "*"
pylint = "*"
termcolor = "*"
jupyter = "*"
numpy = "*"
matplotlib = "*"
scipy = "*"
scikit-learn = "*"
pandas = "*"
arpa = "*"
sympy = "*"
h5py = "*"
requests = "*"
opencv-python = "*"
tqdm = "*"
tensorflow-gpu = "*"
Keras = "*"
Pillow = "*"

[nateprewitt]

Hmm, so this is interesting. I'm not sure I understand how you're installing pipenv and pew but it's not being included on the path. We ran the test suite for Requests with a very similar .travis.yml for a while without any issues. Is this happening on all of your builds?

The immediate fix here is to set PIPENV_VENV_IN_PROJECT in your environment variables for Travis. That will bypass using pew for now.

[madhavajay]

https://travis-ci.org/madhavajay/nd889

Hrm... now im getting:
The command "pipenv install" failed and exited with 127 during .

Worker information
hostname: i-05f704c-precise-production-2-worker-org-docker.travisci.net:93ee387d-3d2d-4e3d-a692-53035abcae5e
version: v2.5.0 https://github.com/travis-ci/worker/tree/da3a43228dffc0fcca5a46569ca786b22991979f
instance: 5ad2b10:travis:python
startup: 884.205423ms
system_info
Build system information
Build language: python
Build group: stable
Build dist: precise
Build id: 253695273
Job id: 253695277
travis-build version: 15c61be5d
Build image provisioning date and time
Thu Feb  5 15:09:33 UTC 2015
Operating System Details
Distributor ID: Ubuntu
Description:    Ubuntu 12.04.5 LTS
Release:    12.04
Codename:   precise
Linux Version
3.13.0-29-generic
Cookbooks Version
a68419e https://github.com/travis-ci/travis-cookbooks/tree/a68419e
GCC version
gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3
Copyright (C) 2011 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
LLVM version
clang version 3.4 (tags/RELEASE_34/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
Pre-installed Ruby versions
ruby-1.9.3-p551
Pre-installed Node.js versions
v0.10.36
Pre-installed Go versions
1.4.1
Redis version
redis-server 2.8.19
riak version
2.0.2
MongoDB version
MongoDB 2.4.12
CouchDB version
couchdb 1.6.1
Neo4j version
1.9.4
RabbitMQ Version
3.4.3
ElasticSearch version
1.4.0
Installed Sphinx versions
2.0.10
2.1.9
2.2.6
Default Sphinx version
2.2.6
Installed Firefox version
firefox 31.0esr
PhantomJS version
1.9.8
ant -version
Apache Ant(TM) version 1.8.2 compiled on December 3 2011
mvn -version
Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00)
Maven home: /usr/local/maven
Java version: 1.7.0_76, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-oracle/jre
Default locale: en_US, platform encoding: ANSI_X3.4-1968
OS name: "linux", version: "3.13.0-29-generic", arch: "amd64", family: "unix"
fix.CVE-2015-7547
$ export DEBIAN_FRONTEND=noninteractive
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
  libc-bin libc-dev-bin libc6-dev
Suggested packages:
  glibc-doc
The following packages will be upgraded:
  libc-bin libc-dev-bin libc6 libc6-dev
4 upgraded, 0 newly installed, 0 to remove and 264 not upgraded.
Need to get 8,856 kB of archives.
After this operation, 13.3 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc6-dev amd64 2.15-0ubuntu10.18 [2,948 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc-dev-bin amd64 2.15-0ubuntu10.18 [84.5 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc-bin amd64 2.15-0ubuntu10.18 [1,178 kB]
Get:4 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc6 amd64 2.15-0ubuntu10.18 [4,646 kB]
Fetched 8,856 kB in 0s (14.0 MB/s)
Preconfiguring packages ...
(Reading database ... 72431 files and directories currently installed.)
Preparing to replace libc6-dev 2.15-0ubuntu10.10 (using .../libc6-dev_2.15-0ubuntu10.18_amd64.deb) ...
Unpacking replacement libc6-dev ...
Preparing to replace libc-dev-bin 2.15-0ubuntu10.10 (using .../libc-dev-bin_2.15-0ubuntu10.18_amd64.deb) ...
Unpacking replacement libc-dev-bin ...
Preparing to replace libc-bin 2.15-0ubuntu10.10 (using .../libc-bin_2.15-0ubuntu10.18_amd64.deb) ...
Unpacking replacement libc-bin ...
Processing triggers for man-db ...
Setting up libc-bin (2.15-0ubuntu10.18) ...
(Reading database ... 72430 files and directories currently installed.)
Preparing to replace libc6 2.15-0ubuntu10.10 (using .../libc6_2.15-0ubuntu10.18_amd64.deb) ...
Unpacking replacement libc6 ...
Setting up libc6 (2.15-0ubuntu10.18) ...
Setting up libc-dev-bin (2.15-0ubuntu10.18) ...
Setting up libc6-dev (2.15-0ubuntu10.18) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
update_libssl1.0.0
$ sudo apt-get install libssl1.0.0
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following extra packages will be installed:
  libssl-dev
The following packages will be upgraded:
  libssl-dev libssl1.0.0
2 upgraded, 0 newly installed, 0 to remove and 262 not upgraded.
Need to get 2,634 kB of archives.
After this operation, 29.7 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libssl-dev amd64 1.0.1-4ubuntu5.39 [1,580 kB]
Get:2 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libssl1.0.0 amd64 1.0.1-4ubuntu5.39 [1,054 kB]
Fetched 2,634 kB in 0s (15.6 MB/s)
Preconfiguring packages ...
(Reading database ... 72430 files and directories currently installed.)
Preparing to replace libssl-dev 1.0.1-4ubuntu5.21 (using .../libssl-dev_1.0.1-4ubuntu5.39_amd64.deb) ...
Unpacking replacement libssl-dev ...
Preparing to replace libssl1.0.0 1.0.1-4ubuntu5.21 (using .../libssl1.0.0_1.0.1-4ubuntu5.39_amd64.deb) ...
Unpacking replacement libssl1.0.0 ...
Setting up libssl1.0.0 (1.0.1-4ubuntu5.39) ...
Setting up libssl-dev (1.0.1-4ubuntu5.39) ...
Processing triggers for libc-bin ...
ldconfig deferred processing now taking place
3.6 is not installed; attempting download
Downloading archive: https://s3.amazonaws.com/travis-python-archives/binaries/ubuntu/12.04/x86_64/python-3.6.tar.bz2
$ sudo tar xjf python-3.6.tar.bz2 --directory /
git.checkout
4.40s$ git clone --depth=50 --branch=master https://github.com/madhavajay/nd889.git madhavajay/nd889
Cloning into 'madhavajay/nd889'...
remote: Counting objects: 583, done.
remote: Compressing objects: 100% (93/93), done.
remote: Total 583 (delta 35), reused 102 (delta 20), pack-reused 463
Receiving objects: 100% (583/583), 25.31 MiB | 9.48 MiB/s, done.
Resolving deltas: 100% (203/203), done.
Checking connectivity... done.
$ cd madhavajay/nd889
$ git checkout -qf 63f1fb8a9ce3fc9244f0edea3719177b35af7201
0.02s$ source ~/virtualenv/python3.6/bin/activate
$ python --version
Python 3.6.1
$ pip --version
pip 9.0.1 from /home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages (python 3.6)
install.1
10.86s$ export PIPENV_VENV_IN_PROJECT=true; pip install pipenv pew
Collecting pipenv
  Downloading pipenv-5.1.2.tar.gz (1.1MB)
    100% |████████████████████████████████| 1.1MB 801kB/s 
Collecting pew
  Downloading pew-0.1.26-py2.py3-none-any.whl
Collecting virtualenv (from pipenv)
  Downloading virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
    100% |████████████████████████████████| 1.8MB 502kB/s 
Requirement already satisfied: pip in /home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages (from pipenv)
Requirement already satisfied: setuptools>=17.1 in /home/travis/virtualenv/python3.6.1/lib/python3.6/site-packages (from pew)
Collecting virtualenv-clone>=0.2.5 (from pew)
  Downloading virtualenv-clone-0.2.6.tar.gz
Collecting pythonz-bd>=1.10.2 (from pew)
  Downloading pythonz_bd-1.11.4-py3-none-any.whl (117kB)
    100% |████████████████████████████████| 122kB 5.1MB/s 
Collecting resumable-urlretrieve; python_version == "3.6" (from pythonz-bd>=1.10.2->pew)
  Downloading resumable_urlretrieve-0.1.5-py2.py3-none-any.whl
Collecting requests (from resumable-urlretrieve; python_version == "3.6"->pythonz-bd>=1.10.2->pew)
  Downloading requests-2.18.1-py2.py3-none-any.whl (88kB)
    100% |████████████████████████████████| 92kB 6.1MB/s 
Collecting urllib3<1.22,>=1.21.1 (from requests->resumable-urlretrieve; python_version == "3.6"->pythonz-bd>=1.10.2->pew)
  Downloading urllib3-1.21.1-py2.py3-none-any.whl (131kB)
    100% |████████████████████████████████| 133kB 5.8MB/s 
Collecting idna<2.6,>=2.5 (from requests->resumable-urlretrieve; python_version == "3.6"->pythonz-bd>=1.10.2->pew)
  Downloading idna-2.5-py2.py3-none-any.whl (55kB)
    100% |████████████████████████████████| 61kB 5.7MB/s 
Collecting certifi>=2017.4.17 (from requests->resumable-urlretrieve; python_version == "3.6"->pythonz-bd>=1.10.2->pew)
  Downloading certifi-2017.4.17-py2.py3-none-any.whl (375kB)
    100% |████████████████████████████████| 378kB 2.3MB/s 
Collecting chardet<3.1.0,>=3.0.2 (from requests->resumable-urlretrieve; python_version == "3.6"->pythonz-bd>=1.10.2->pew)
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133kB)
    100% |████████████████���███████████████| 143kB 6.3MB/s 
Building wheels for collected packages: pipenv, virtualenv-clone
  Running setup.py bdist_wheel for pipenv ... \/done
  Stored in directory: /home/travis/.cache/pip/wheels/48/db/09/235c870e15bc0c59510f62eb69b7cd47ca3205ff80afa7dfc9
  Running setup.py bdist_wheel for virtualenv-clone ... -done
  Stored in directory: /home/travis/.cache/pip/wheels/24/51/ef/93120d304d240b4b6c2066454250a1626e04f73d34417b956d
Successfully built pipenv virtualenv-clone
Installing collected packages: virtualenv, virtualenv-clone, urllib3, idna, certifi, chardet, requests, resumable-urlretrieve, pythonz-bd, pew, pipenv
Successfully installed certifi-2017.4.17 chardet-3.0.4 idna-2.5 pew-0.1.26 pipenv-5.1.2 pythonz-bd-1.11.4 requests-2.18.1 resumable-urlretrieve-0.1.5 urllib3-1.21.1 virtualenv-15.1.0 virtualenv-clone-0.2.6
install.2
7.93s$ pipenv --three
Virtualenv already exists!
Removing existing virtualenv...
Creating a virtualenv for this project...
⠋Running virtualenv with interpreter /opt/python/3.4.2/bin/python3
Using base prefix '/opt/python/3.4.2'
New python executable in /home/travis/virtualenv/python3.6.1/bin/python3
Also creating executable in /home/travis/virtualenv/python3.6.1/bin/python
Installing setuptools, pip...done.
Virtualenv location: /home/travis/virtualenv/python3.6.1
0.02s$ pipenv install
/home/travis/.travis/job_stages: line 54: /home/travis/virtualenv/python3.6.1/bin/pipenv: No such file or directory
The command "pipenv install" failed and exited with 127 during .
Your build has been stopped.

[nateprewitt]

Well, that wasn't expected. It looks like 8055035 introduced this behaviour. When Travis activates it's virtualenv to run the test suite, PIPENV_USE_SYSTEM is latching onto that. We've essentially broken PIPENV_VENV_IN_PROJECT in this case, unless you also set PIPENV_IGNORE_VIRTUALENVS.

This feels like a regression to me, but that may have been Kenneth's intended use case with this. For now, I'd set both environment variables. You can use the env section in your .travis.yml to avoid exporting before a command.

[kennethreitz42]

Hmm, travis exposes the CI=1 environment variable, we could key off of that...

[madhavajay]

Sorry, I don't quite understand what I need to do, can you show me an example of what I should put in the .travis file?

[nateprewitt]

@madhavajay, I believe a travis.yml like this should give you what you're looking for.

language: python
python:
  - 3.6

env:
  - PIPENV_VENV_IN_PROJECT=1
  - PIPENV_IGNORE_VIRTUALENVS=1

install:
  - pip install pipenv pew
  - pipenv --three
  - pipenv install git+https://github.com/hmmlearn/hmmlearn.git
  - pipenv install

script:
  - pipenv run pytest
  - pipenv run mypy 1_sudoku --ignore-missing-imports

[madhavajay]

Awesome thanks, looks like it needs to be:

env:
  global:
    - PIPENV_VENV_IN_PROJECT=1
    - PIPENV_IGNORE_VIRTUALENVS=1

Otherwise it builds one for each env which doesn't have the desired affect.

[nateprewitt]

Great, glad you got things working @madhavajay! We'll look into handling this better in a future release.

[kennethreitz42]

I'll open an issue for this.

[kennethreitz42]

https://github.com/kennethreitz/pipenv/issues/436