Poetry: `poetry install` fails when installing dependencies, from git repositories, in gitlab CI.

[pawelrubin]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.
• [x] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: python3.8 docker image
• Poetry version: 1.0.5 (installed via pip3 install poetry)
• Link of a Gist with the contents of your pyproject.toml file: link

Issue

poetry install fails when installing dependencies, from git repositories, in gitlab CI.

It does work on my local machine. Dependency has been added via poetry add git+https://github.com/devopshq/artifactory.git@support-python-38-glob
Traceback from poetry install -vvv:

Installing dependencies from lock file

[CalledProcessError]
Command '['git', '--git-dir', '/tmp/pypoetry-git-artifactory60aj4enu/.git', '--work-tree', '/tmp/pypoetry-git-artifactory60aj4enu', 'checkout', 'support-python-38-glob']' returned non-zero exit status 1.

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/clikit/console_application.py", line 131, in run
    status_code = command.handle(parsed_args, io)
  File "/usr/local/lib/python3.8/site-packages/clikit/api/command/command.py", line 120, in handle
    status_code = self._do_handle(args, io)
  File "/usr/local/lib/python3.8/site-packages/clikit/api/command/command.py", line 171, in _do_handle
    return getattr(handler, handler_method)(args, io, self)
  File "/usr/local/lib/python3.8/site-packages/cleo/commands/command.py", line 92, in wrap_handle
    return self.handle()
  File "/usr/local/lib/python3.8/site-packages/poetry/console/commands/install.py", line 63, in handle
    return_code = installer.run()
  File "/usr/local/lib/python3.8/site-packages/poetry/installation/installer.py", line 74, in run
    self._do_install(local_repo)
  File "/usr/local/lib/python3.8/site-packages/poetry/installation/installer.py", line 225, in _do_install
    ops = solver.solve(use_latest=whitelist)
  File "/usr/local/lib/python3.8/site-packages/poetry/puzzle/solver.py", line 36, in solve
    packages, depths = self._solve(use_latest=use_latest)
  File "/usr/local/lib/python3.8/site-packages/poetry/puzzle/solver.py", line 180, in _solve
    result = resolve_version(
  File "/usr/local/lib/python3.8/site-packages/poetry/mixology/__init__.py", line 7, in resolve_version
    return solver.solve()
  File "/usr/local/lib/python3.8/site-packages/poetry/mixology/version_solver.py", line 80, in solve
    next = self._choose_package_version()
  File "/usr/local/lib/python3.8/site-packages/poetry/mixology/version_solver.py", line 355, in _choose_package_version
    packages = self._provider.search_for(dependency)
  File "/usr/local/lib/python3.8/site-packages/poetry/puzzle/provider.py", line 130, in search_for
    packages = self.search_for_vcs(dependency)
  File "/usr/local/lib/python3.8/site-packages/poetry/puzzle/provider.py", line 167, in search_for_vcs
    package = self.get_package_from_vcs(
  File "/usr/local/lib/python3.8/site-packages/poetry/puzzle/provider.py", line 198, in get_package_from_vcs
    git.checkout(reference, tmp_dir)
  File "/usr/local/lib/python3.8/site-packages/poetry/vcs/git.py", line 217, in checkout
    return self.run(*args)
  File "/usr/local/lib/python3.8/site-packages/poetry/vcs/git.py", line 284, in run
    subprocess.check_output(["git"] + list(args), stderr=subprocess.STDOUT)
  File "/usr/local/lib/python3.8/subprocess.py", line 411, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/usr/local/lib/python3.8/site-packages/poetry/utils/_compat.py", line 205, in run
    raise CalledProcessError(

[finswimmer]

Hello @pawelrubin,

it looks a bit like git cannot find the branch. Can you try to clone the repository and checkout the branch in the target machine only with git? At best, copy&paste the branch name to avoid oversee any minor typo.

fin swimmer

[balan-jayavictor]

The target here is a docker container (based on the image we provide) provisioned by Gitlab during the build time.
It works fine when running the docker container locally.
@pawelrubin did you manage to fix it / work around it?

[guhcampos]

I believe this problem is related to something I've experienced.

It looks like poetry will clone the repository with it's default name during poetry add but will try to infer the local repository name from the package name when doing anything else, so in my case I had something like:

poetry add git+ssh://[email protected]:/mycompany-sdk.git

which cloned such repository under $HOME/.cache/pypoetry/virtualenvs/myproject-pbdKQ0bZ-py3.6/src/mycompany-sdk/

However, during a subsequent poetry update it failed with:

[CalledProcessError]
Command '['git', '--git-dir', '$HOME.cache/pypoetry/virtualenvs/myproject-pbdKQ0bZ-py3.6/src/mycompany/.git', '--work-tree', '$HOME/.cache/pypoetry/virtualenvs/myproject-pbdKQ0bZ-py3.6/src/mycompany', 'rev-parse', 'HEAD^{commit}']' returned non-zero exit status 128.

Which makes total sense because there's no mycompany directory there, the repository is called mycompany-sdk, yet the package it contains is, indeed, named mycompany and that's why poetry thinks it should look for that package there.

[johnmee]

There does not appear to be any rush to fix this, so I guess there is a workaround. I wonder what that might be?

Perhaps delete the virtualenv and start over?

rm -rf ~/.cache/pypoetry/virtualenvs/my-project-name-37cdnjD8-py3.8/
poetry install

[floi]

I had a similar issue with a dependency from a private Gitlab repository that used SSH for cloning (same CalledProcessError). When testing in a local Docker container, I noticed that I had to actively confirm the server fingerprint. As git has no option for automatically doing that, I solved poetry's installation of dependencies in the Gitlab CI container by adding the fingerprint to ~/.ssh/known_hosts before running poetry install in my .gitlab-ci.yml. Here's how I did that:

1. Manually clone my dependency's repository in a clean docker container
2. In that docker container, get the two lines (not sure why there's more than one) from ~/.ssh/known_hosts
3. In my .gitlab-ci.yml, add the following lines before calling poetry install:

- mkdir ~/.ssh
- echo "FIRST LINE COPIED IN STEP 2" >> ~/.ssh/known_hosts
- echo "SECOND LINE COPIED IN STEP 2" >> ~/.ssh/known_hosts 

I'm aware that the problem in the original comment might be something else (given that you, @pawelrubin, were trying to clone via HTTPS from a public repository), but maybe this will help other people ending up here when researching that error message.