Poetry: Alpine 3.12.0: System environment fails with "ModuleNotFoundError: No module named 'pip._vendor.requests._internal_utils"

[drcongo]

• [x] I am on the latest Poetry version.
• [x] I have searched the issues of this repo and believe that this is not a duplicate.

• [ ] If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• Alpine 3.12.0:

• Poetry version 1.1.0:

Issue

This is the output from our CI...

• Installing pyrepl (0.9.0)
• Updating requests (2.23.0 -> 2.24.0)
• Installing tablib (2.0.0)
• Installing traitlets (4.3.3)

  EnvCommandError

  Command ['/usr/bin/python', '-m', 'pip', 'install', '--no-deps', '/root/.cache/pypoetry/artifacts/verylongpath/traitlets-4.3.3-py2.py3-none-any.whl'] errored with the following return code 1, and output: 
  Traceback (most recent call last):
    File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
      return _run_code(code, main_globals, None,
    File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
      exec(code, run_globals)
    File "/usr/lib/python3.8/site-packages/pip/__main__.py", line 26, in <module>
      sys.exit(_main())
    File "/usr/lib/python3.8/site-packages/pip/_internal/cli/main.py", line 73, in main
      command = create_command(cmd_name, isolated=("--isolated" in cmd_args))
    File "/usr/lib/python3.8/site-packages/pip/_internal/commands/__init__.py", line 104, in create_command
      module = importlib.import_module(module_path)
    File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
    File "<frozen importlib._bootstrap>", line 991, in _find_and_load
    File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 783, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "/usr/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 24, in <module>
      from pip._internal.cli.req_command import RequirementCommand, with_cleanup
    File "/usr/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 18, in <module>
      from pip._internal.network.download import Downloader
    File "/usr/lib/python3.8/site-packages/pip/_internal/network/download.py", line 9, in <module>
      from pip._vendor.requests.models import CONTENT_CHUNK_SIZE
    File "/usr/lib/python3.8/site-packages/requests/models.py", line 28, in <module>
    File "/usr/lib/python3.8/site-packages/requests/auth.py", line 20, in <module>
    File "/usr/lib/python3.8/site-packages/requests/cookies.py", line 16, in <module>
  ModuleNotFoundError: No module named 'pip._vendor.requests._internal_utils'

  at ~/.poetry/lib/poetry/utils/env.py:948 in _run
   944│                 output = subprocess.check_output(
   945│                     cmd, stderr=subprocess.STDOUT, **kwargs
   946│                 )
   947│         except CalledProcessError as e:
→  948│             raise EnvCommandError(e, input=input_)
   949│ 
   950│         return decode(output)
   951│ 
   952│     def execute(self, bin, *args, **kwargs):


• Installing unidecode (1.1.1)
• Installing willow (1.4)
• Installing xlsxwriter (1.3.3)
Service 'web' failed to build : The command '/bin/sh -c source $HOME/.poetry/env &&     poetry self update &&     poetry install' returned a non-zero code: 1

Error: Process completed with exit code 1.

This was working 2.5 hours ago with the same pyproject.toml

[kupuguy]

Our CI is also broken but in our case it's the auth changes which means poetry is unable to fetch anything from our private (gemfury) repository. Pinning the poetry version to 1.0.10 in our builds is the only workaround we've found so far, you may have to do similar.

[drcongo]

Yeah, I've pinned it to 1.0.10 here too and it's back to working.

[tony]

To downgrade, see if poetry self update 1.0 works

I look forward to 1.1 being stable - and 1.2 going a lot smoother. Sorry for any inconvenience caused.

[drcongo]

No worries @tony - poetry self update 1.0.10 in our Dockerfile is working for now.

[remram44]

Broken for me because poetry export --dev --format requirements.txt fails with an IndexError (printed to stdout instead of stderr, too, like I wasn't piping that...)

 IndexError

  list index out of range

  at ~/.poetry/lib/poetry/utils/exporter.py:76 in _export_requirements_txt
       72│             if not dev
       73│             else self._poetry.package.all_requires,
       74│             with_nested=True,
       75│         ):
    →  76│             package = repository.find_packages(dependency=dependency)[0]
       77│ 
       78│             # If a package is optional and we haven't opted in to it, continue
       79│             if package.optional and package.name not in extra_package_names:
       80│                 continue

Link to CI job

[abn]

@drcongo whats your pip version; looks like the issue is with the system installed pip version there. Can you try running the same within a poetry managed virtualenv please? It is not obvious where the problem lies atm. From what I can tell, poetry handed over the installation of local wheel to pip. The difference is in 1.1.0 we no longer use pip to download the wheels.

@remram44 see #3024

@kupuguy how have you configured your private repository?

[remram44]

@abn Thanks, now what do I do about this one:

$ poetry install
Installing dependencies from lock file

  RuntimeError

  generator didn't stop after throw()

  at /usr/lib/python3.8/contextlib.py:162 in __exit__
      158│                 # by 'except BaseException'.
      159│                 if sys.exc_info()[1] is value:
      160│                     return False
      161│                 raise
    → 162│             raise RuntimeError("generator didn't stop after throw()")
      163│ 
      164│ 
      165│ class _AsyncGeneratorContextManager(_GeneratorContextManagerBase,
      166│                                     AbstractAsyncContextManager):

[edit: filed #3026]

Today is going to be a long day, thanks for that.

[abn]

Today is going to be a long day, thanks for that.

@remram44 It does not have to be, maybe consider locking at 1.0.10 if your workflow is breaking untill all the issues are resolved. Also perhaps test early with pre-releases and let us know of issues. Testing early helps. :)

We are all unpaid volunteers with day jobs here maintaining this project, we do try to do a good job. Sometimes things break, and considering the amount of changes in this realease, some use cases were bound to break.

As for your error, a bit context might be required. Maybe try reproducing it with a clean virtualenv and/or regenerated lock file. Since this is a different issue to the one being reported, I'd recommend creating a new issue and posting information on how to reproduce it, -vvv output and also if possible a pyproject.toml file.

[remram44]

I sympathize with being a small team of volunteers, however you always had the option of not introducing massive breaking changes in this minor release.

In particular, you could have introduced versioning to the lockfile before you made breaking changes to the format, so as to not require an immediate update of all team members. You could have made this release a major version, which would have prevented CI systems from auto-updating and breaking.

[abn]

In particular, you could have introduced versioning to the lockfile before you made breaking changes to the format, so as to not require an immediate update of all team members. You could have made this release a major version, which would have prevented CI systems from auto-updating and breaking.

The lock file has been versioned since 1.0.10. The lockfile from older releases will continue to function with 1.1.0.

[remram44]

I see, nice! Your blog post made it seem like that was added in 1.1.0.

[dvf]

Make sure you run poetry update. See https://github.com/python-poetry/poetry/issues/3028

[drcongo]

@drcongo whats your pip version; looks like the issue is with the system installed pip version there. Can you try running the same within a poetry managed virtualenv please? It is not obvious where the problem lies atm. From what I can tell, poetry handed over the installation of local wheel to pip. The difference is in 1.1.0 we no longer use pip to download the wheels.

I've had to switch to another project, but I'll check when I'm back on it. It _should_ be the latest pip but there's always the chance that it isn't.

By the way, I can't think of a project that has as many possible variables that could break people's workflows as this one, so I'm _hugely_ appreciative of everyone's work on Poetry. You're all amazing.

[matthewarmand]

Broken for me because poetry export --dev --format requirements.txt fails with an IndexError (printed to stdout instead of stderr, too, like I wasn't piping that...)

 IndexError

  list index out of range

  at ~/.poetry/lib/poetry/utils/exporter.py:76 in _export_requirements_txt
       72│             if not dev
       73│             else self._poetry.package.all_requires,
       74│             with_nested=True,
       75│         ):
    →  76│             package = repository.find_packages(dependency=dependency)[0]
       77│ 
       78│             # If a package is optional and we haven't opted in to it, continue
       79│             if package.optional and package.name not in extra_package_names:
       80│                 continue

I've noticed this same behavior, our pyproject.toml doesn't have the packages attribute in the tool.poetry section so I wonder if that check in the error message needs to be different. Export works fine up through 1.0.10, breaks on 1.1.0

[abn]

@matthewarmand please see #3018 for the export issue.